2.5d global positioning method, system, device and medium for exhibition visitors

Abstract

The application provides a 2.5D global positioning method, system, equipment and medium for exhibition visitors, relates to the technical fields of meta universe and artificial intelligence, and comprises the following steps: S1, a zero Z-axis plane of a global coordinate system is arranged in a positioning domain, global grid points are divided for the whole scene, and the global grid points are projected into each camera to obtain image grid points of a video image; S2, a nearest neighbor point is acquired according to the image grid points; and S3, the mean value of the neighbor points acquired by a plurality of cameras is taken as the global position of the target. The application can adjust the positioning accuracy and avoid the algorithm complexity caused by three-dimensional reconstruction.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of meta universe and artificial intelligence, in particular, to a 2.5D global positioning method, system, device and medium for exhibition visitors. BACKGROUND

[0002] At present, in order to realize the digitalization, networking and intelligentization of the meta universe of the cultural and historical museum, the cultural and historical articles and the showcases can be accurately reproduced through digital scanning technology, but the position of the visitors in the scene is complex and variable, which is difficult to describe accurately. The accurate positioning of the visitors can give an automatic warning when the crowd gathers and can timely inform the security department when entering the restricted area. The traditional monitoring system in the cultural and historical museum can only detect the position of the person in the image and cannot locate the global position in the scene.

[0003] It can be known from the prior art Kutulakos, K.N., Seitz, S.M. A Theory of Shape by Space Carving. International Journal of Computer Vision 38, 199-218 (2000) that the space carving method reconstructs the entire scene including the human body by voxel under the assumption of good background segmentation to describe the position of the person, which has a large amount of calculation and needs a scene with a single background, and it is difficult to apply to the actual exhibition scene. The similar method of 3D Yolo lacks camera parameter information and is difficult to analyze and fuse the information of multiple cameras. SUMMARY

[0004] In view of the defects in the prior art, the present application provides a 2.5D global positioning method, system, device and medium for exhibition visitors.

[0005] According to the 2.5D global positioning method, system, device and medium for exhibition visitors provided by the present application, the scheme is as follows:

[0006] In a first aspect, a 2.5D global positioning method for exhibition visitors is provided, which comprises:

[0007] Step S1: placing the zero Z-axis plane of the global coordinate system in the positioning domain, dividing the entire scene into global grid points, and projecting the global grid points into each camera to obtain image grid points of the video image;

[0008] Step S2: obtaining the nearest neighbor points according to the image grid points;

[0009] Step S3: taking the mean value of the neighbor points obtained by multiple cameras as the global position of the target.

[0010] Preferably, the step S2 comprises: in each group of cameras, using a two-dimensional pedestrian detection classification algorithm to measure the bounding box position of the audience in each camera, obtaining the coordinate values of the left and right points of the lower boundary of the bounding box in the video image (bx 3, by3), (bx 4, by4), and finding the nearest neighbor point (Ix bm, Iy bm ) in the image grid point, which corresponds to the midpoint of the lower boundary, and the subscript m is the camera label.

[0011] Preferably, the step S3 comprises: for each target in each frame of video image, taking the mean value of the neighbor points (Ix bm, Iy bm ) obtained by m cameras as the global position of the target, and when occlusion, m is less than the number of cameras in the group; for multiple targets, target association is needed to be performed through the most dynamic nearest neighbor method, and then the mean value is taken after the association.

[0012] Preferably, the method further comprises: if detection is missing in consecutive video images, then inter-frame data association is performed through the nearest neighbor matching method.

[0013] In a second aspect, a 2.5D global positioning system for exhibition audience is provided, the system comprising:

[0014] Module M1: placing the zero Z-axis plane of the global coordinate system in the positioning domain, dividing the global grid points for the entire scene, and projecting the global grid points into each camera to obtain the image grid points of the video image;

[0015] Module M2: obtaining the nearest neighbor point according to the image grid point;

[0016] Module M3: taking the mean value of the neighbor points obtained by multiple cameras as the global position of the target.

[0017] Preferably, the module M2 comprises: in each group of cameras, using a two-dimensional pedestrian detection classification algorithm to measure the bounding box position of the audience in each camera, obtaining the coordinate values of the left and right points of the lower boundary of the bounding box in the video image (bx 3, by3), (bx 4, by4), and finding the nearest neighbor point (Ix bm, Iy bm ) in the image grid point, which corresponds to the midpoint of the lower boundary, and the subscript m is the camera label.

[0018] Preferably, the module M3 comprises: for each target in each frame of video image, taking the mean value of the neighbor points (Ix bm, Iy bmThe global position is the mean value of the target as the goal, and m is less than the number of cameras in the group when occlusion occurs; for multiple targets, target association is needed through the nearest neighbor method first, and then the mean value is taken after association.

[0019] Preferably, the system further comprises: if missing is detected in the continuous video images, then inter-frame data association is performed through the nearest neighbor matching method.

[0020] In a third aspect, an apparatus is provided, the apparatus comprising:

[0021] one or more processors;

[0022] a memory device for storing one or more programs,

[0023] when the one or more programs are executed by the one or more processors, the one or more processors implement the steps in the method.

[0024] In a fourth aspect, a computer-readable storage medium storing a computer program is provided, wherein the computer program is executed by a processor to implement the steps in the method.

[0025] Compared with the prior art, the present application has the following beneficial effects:

[0026] The scalable planar grid point division adopted by the present application can adjust the positioning accuracy, and the planar grid positioning also saves the complex three-dimensional voxel reconstruction link, directly fuses the global positioning result from the two-dimensional detection result, and greatly saves the computing resources. BRIEF DESCRIPTION OF DRAWINGS

[0027] Other features, objects and advantages of the present application will become more apparent from the following detailed description of non-limiting embodiments, read in conjunction with the accompanying drawings:

[0028] Figure 1 It is a schematic diagram of the overall flow of the present application;

[0029] Figure 2 It is a schematic diagram of the configuration of multiple cameras and a calibration object;

[0030] Figure 3 It is a planar subdivision grid and a projected grid in the camera;

[0031] Figure 4 It is a planar coarse grid and a projected grid in the camera;

[0032] Figure 5 It is the position of the global coordinate origin after calibration and the result of single audience plane detection. DETAILED DESCRIPTION

[0033] The application will be described in detail below with specific examples. The following examples will help those skilled in the art to further understand the application, but do not limit the application in any form. It should be noted that those skilled in the art can make several changes and improvements without departing from the concept of the application. These are within the scope of protection of the application.

[0034] The embodiment of the application provides a 2.5D global positioning method for an exhibition audience, which comprises the following steps: according to the shape and size of a venue, a multi-baseline camera system is arranged in the exhibition hall by grouping according to specific requirements, 3-4 cameras are grouped, and there is an overlapping position between the camera groups, which is used to determine the relative position of the cameras in the groups; after calibration of the multi-group multi-baseline camera, the position of each camera in a global coordinate system is obtained, including translation and rotation relative to the global coordinate system (world coordinate system); in each group of cameras, a 2D pedestrian detection classification algorithm (yolov5 or SSD) is used to detect the position of the audience in each camera, and the lower boundary coordinate value of the boundary box is obtained. Figure 1 As shown in the figure, the method specifically comprises the following steps:

[0035] Step S1: place the zero Z-axis plane of the global coordinate system in the positioning domain, divide the entire scene into global grid points, and project the global grid points into each camera to obtain image grid points of the video image.

[0036] Specifically, the zero Z-axis plane of the global coordinate system is in the positioning domain, assuming that the entire scene is 10m*8m, and the grid points (Gx i, Gy j ) are divided according to 100mm (subdivision factor, which can adjust the positioning accuracy according to the computing power), i∈[0, 99], j∈[0, 79], the global grid points are projected into each camera to obtain image grid points (Ix i, Iy j ).

[0037] Step S2: obtain the nearest neighbor point according to the image grid point.

[0038] Specifically, in each group of cameras, a 2D pedestrian detection classification algorithm (yolov5 or SSD) is used to detect the boundary box position of the audience in each camera, and the lower boundary left and right point coordinate values (bx 3, by3), (bx 4, by4) of the boundary box in the video image are obtained, and the nearest neighbor point (Ix bm, Iy bm ) is found in the image grid point. The nearest neighbor point (Ix bm, Iy bm ) is used as the observation value of the human position in the camera, which is equivalent to the midpoint of the lower boundary, and the subscript m is the camera label.

[0039] Step S3: Take the mean of the neighboring points obtained by multiple cameras as the global position of the target.

[0040] Specifically, for each target in each frame image, take the mean of the position points (Ix bm, Iy bm ) calculated by m cameras as the global position of the target, and when occluded, m is less than the number of cameras in the group. For multiple targets, use the nearest neighbor method to perform target association first, and then take the mean after association. In the nearest neighbor method in this embodiment, for example, camera one detects 5 points and camera two detects 4 points. The distances between the five points in camera one and the four points in camera two are calculated, and the nearest point is found as the matching association point.

[0041] For the detection missing problem (decoding problem or occlusion) in continuous video images, the nearest neighbor matching method is used for inter-frame observation data association. M people position observation values (Xp, Yp) are calculated in the previous frame image, and N people position observation values (Xp+1, Yp+1) are calculated in the next frame image. If N < M, the current missing data in the next frame is predicted using the unmatched points in the previous frame. In addition to position matching, the color histogram of the target detected in the previous and next frames can also be used as a matching feature for nearest neighbor matching.

[0042] Embodiment 2 is a preferred embodiment of Embodiment 1. Those skilled in the art can understand the 2.5D global positioning method for exhibition visitors provided by the present application as a specific implementation of a 2.5D global positioning system for exhibition visitors, that is, the 2.5D global positioning system for exhibition visitors can be realized by executing the steps of the 2.5D global positioning method for exhibition visitors.

[0043] Next, the present application will be described in more detail.

[0044] Referring to Figure 1 , the 2.5D global positioning method for exhibition visitors of the present embodiment includes the following contents:

[0045] (1) The cameras calibrated by grouping detect the position of the visitors (anchor points) in the image using a 2D human body detection classification method (Yolov5, SSD, etc.), mark the position of the detection box, and perform nearest neighbor association with the projection points of the grid to calculate the grid point closest to the intersection line between the bounding box and the ground as the observation value.

[0046] (2) The number of observation values of multiple visitors in each camera may not be consistent due to occlusion, so the global state of each frame is estimated by fusing the observation values of multiple cameras through position association.

[0047] (3) In the time series video detection, it is possible that the Mth camera detects the Nth person in the previous frame, and does not detect in the next frame, resulting in missing observations, at this time, the observation position and the color histogram nearest neighbor matching method can be used for data association to ensure the smoothness of state estimation.

[0048] As shown in Figure 2 The 2.5D global positioning method for exhibition visitors of the embodiment comprises a first group of cameras 1, 2, 3 and 4, a calibration object A in the common view of 1, 2, 3 and 4, a second group of cameras 5, 6, 7 and 8, a calibration object C in the common view of the second group, a third group of cameras 9, 10, 11 and 12, a calibration object E in the common view of the third group of cameras, a calibration object B in the common view of cameras 3, 4, 5 and 6, used for associating and converting the global positioning coordinates of the second group and the first group, and a calibration object D in the common view of 7, 8, 9 and 10, used for associating the global positioning coordinates of the second group and the third group. In the application, the calibration objects B and D are vertically placed with other calibration objects, which can be used as a constraint in coordinate conversion and the propagation error of inter-group positioning.

[0049] Referring to Figures 3-5 As shown in the figure, according to different positioning accuracy, the scene is divided into grids of different sizes, and the grid points are projected into the camera and matched with the position of the person calculated by the detection algorithm. This method can flexibly adjust the positioning accuracy.

[0050] The embodiment of the application provides a 2.5D global positioning method, system, device and medium for exhibition visitors, which can adjust the positioning accuracy by the size of grid division, avoid the algorithm complexity caused by three-dimensional reconstruction, and bring convenience to the implementation of positioning indoor visitors.

[0051] Those skilled in the art know that, in addition to implementing the system provided by the application and each device, module and unit thereof in a pure computer readable program code manner, the system provided by the application and each device, module and unit thereof can also be implemented in the form of logic gates, switches, application specific integrated circuits, programmable logic controllers and embedded microcontrollers by logically programming the method steps to achieve the same functions. Therefore, the system provided by the application and each device, module and unit thereof can be considered as a hardware component, and the devices, modules and units included therein for implementing various functions can also be considered as structures within the hardware component. The devices, modules and units for implementing various functions can also be considered as both software modules implementing methods and structures within hardware components.

[0052] The specific embodiments of the present application are described above. It needs to be understood that the present application is not limited to the specific embodiments described above, and various changes or modifications can be made by those skilled in the art within the scope of the claims, which does not affect the essential content of the present application. The embodiments of the present application and the features in the embodiments can be combined with each other at will without conflict.

Claims

1. A 2.5D global positioning method for exhibition hall visitors, characterized in that, include: Step S1: Place the zero Z-axis plane of the global coordinate system in the positioning domain, divide the entire scene into global grid points, and project the global grid points onto the image grid points of each camera that obtains the video image; Step S2: Obtain the nearest neighbor points based on the image grid points; Step S2 includes: in each group of cameras, using a two-dimensional pedestrian detection and classification algorithm to measure the bounding box position of the audience in each camera, and obtaining the coordinate values ​​(bx) of the left and right points of the lower boundary of the bounding box in the video image. 3, by3), (bx 4, by4), and find the nearest neighbor (Ix) in the image grid points. bm, Iy bm ), which is equivalent to the midpoint of the lower boundary, and the subscript m is the camera label; Step S3: Take the average value of the neighboring points obtained by multiple cameras as the global position of the target; The three-dimensional voxel reconstruction step is omitted.

2. The 2.5D global positioning method for exhibition hall visitors according to claim 1, characterized in that, Step S3 includes: for each target in each frame of video image, taking m cameras to acquire neighboring points (Ix). bm, Iy bm The mean of the values ​​is used as the global position of the target. When occlusion occurs, m is less than the number of cameras in the group. For multiple targets, the most dynamic nearest neighbor method is used to associate the targets first, and then the mean is taken.

3. The 2.5D global positioning method for exhibition hall visitors according to claim 1, characterized in that, The method further includes: if a missing element is detected in consecutive video images, then inter-frame data association is performed using a nearest neighbor matching method.

4. A 2.5D global positioning system for exhibition hall visitors, characterized in that, include: Module M1: Places the zero Z-axis plane of the global coordinate system in the positioning domain, divides the entire scene into global grid points, and projects the global grid points onto the image grid points of each camera that obtains the video image; Module M2: Obtains the nearest neighbor point based on the image grid points; The module M2 includes: in each group of cameras, using a two-dimensional pedestrian detection and classification algorithm to measure the bounding box position of the audience in each camera, and obtaining the coordinate values ​​(bx) of the left and right points of the lower boundary of the bounding box in the video image. 3, by3), (bx 4, by4), and find the nearest neighbor (Ix) in the image grid points. bm, Iy bm ), which is equivalent to the midpoint of the lower boundary, and the subscript m is the camera label; Module M3: Takes the average value of the neighboring points obtained from multiple cameras as the global position of the target; The three-dimensional voxel reconstruction step is omitted.

5. The 2.5D global positioning system for exhibition hall visitors according to claim 4, characterized in that, The module M3 includes: for each target in each frame of video image, taking m cameras to acquire neighboring points (Ix). bm, Iy bm The mean of the values ​​is used as the global position of the target. When occlusion occurs, m is less than the number of cameras in the group. For multiple targets, the nearest neighbor method is used to associate the targets first, and then the mean is taken.

6. The 2.5D global positioning system for exhibition hall visitors according to claim 5, characterized in that, The system further includes: if a missing element is detected in consecutive video images, then inter-frame data association is performed using the nearest neighbor method.

7. A device, characterized in that, The device includes: One or more processors; Storage device for storing one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors perform the steps of the method as described in any one of claims 1 to 3.

8. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 3.

Images