Three-dimensional spatial positioning method, system and storage medium

Abstract

The application discloses a three-dimensional space positioning method and system and a storage medium, and is applied to the technical field of space positioning, and can realize efficient three-dimensional space positioning and improve positioning accuracy. The method comprises the following steps: a plurality of mark points are arranged in a preset positioning space; a mark point is arranged at the center of a camera picture; real coordinate data of the mark points are acquired and are converted into coordinate data in a world coordinate system, so that first world coordinate system data are obtained; the coordinate origin of the world coordinate system is the mark point arranged at the center of the camera picture; a mapping relationship between a pixel coordinate system and the world coordinate system is constructed according to the first world coordinate system data; camera coordinate system data of a target detection point and a rotation angle are calculated according to the mapping relationship; the rotation angle is a camera rotation angle for adjusting the target detection point to the center of the camera picture; the target detection point is adjusted to the center of the camera picture according to the camera coordinate system data and the rotation angle; and second world coordinate system data of a target pixel point in the current camera picture is obtained according to the mapping relationship.

Description

Technical Field

[0001] This invention relates to the field of spatial positioning technology, and in particular to a three-dimensional spatial positioning method, system and storage medium. Background Technology

[0002] Video surveillance acquires video images of monitored targets, records and reviews these images, and performs corresponding actions manually or automatically based on the video image information to achieve monitoring, control, security, and intelligent management of the monitored targets. It has been widely used in many public places such as military, customs, public security, and fire departments. Video surveillance is also a major component of intelligent monitoring. How to obtain accurate 3D information and precise positioning from video images is an indispensable part of intelligent monitoring. However, commonly used methods for locating 3D information in the real world using 2D information include binocular camera positioning and depth camera positioning. Binocular camera positioning requires two cameras and is greatly affected by pixel count, resulting in limited measurement accuracy. Depth cameras measure the distance between objects and the camera using infrared structured light or Time-of-Flight (ToF) principles. This method suffers from many problems, such as narrow measurement range, high noise, small field of view, susceptibility to sunlight interference, and inability to measure transmissive materials, making it difficult to use outdoors. Summary of the Invention

[0003] To address at least one of the aforementioned technical problems, this invention proposes a three-dimensional spatial positioning method, system, and storage medium, which can achieve relatively efficient three-dimensional spatial positioning and effectively improve positioning accuracy.

[0004] On one hand, embodiments of the present invention provide a three-dimensional spatial positioning method, including the following steps:

[0005] Several marker points are set in a preset positioning space; one of these marker points is set at the center of the camera image.

[0006] Obtain the actual coordinate data of the aforementioned marker points;

[0007] The real-world coordinate data is converted into coordinate data in the world coordinate system to obtain first world coordinate system data; wherein, the origin of the world coordinate system is the marker point set at the center of the camera image;

[0008] A mapping relationship between the pixel coordinate system and the world coordinate system is constructed based on the first world coordinate system data;

[0009] The camera coordinate system data and rotation angle of the target detection point are calculated based on the mapping relationship; wherein, the rotation angle is the camera rotation angle that adjusts the target detection point to the center of the camera image;

[0010] The target detection point is adjusted to the center of the camera image based on the camera coordinate system data and the rotation angle.

[0011] The second world coordinate system data of the target pixel in the current camera frame is calculated based on the mapping relationship.

[0012] A three-dimensional spatial positioning method according to an embodiment of the present invention has at least the following beneficial effects: First, several marker points are set in a preset positioning space, and one marker point is set at the center of the camera's image. Then, the real-world coordinate data of the arranged marker points are acquired, and the real-world coordinate data of each marker point is converted into coordinate data in a world coordinate system with the marker point at the center of the camera's image as the origin, obtaining the corresponding first world coordinate system data. Next, this embodiment constructs a mapping relationship between the pixel coordinate system and the world coordinate system based on the first world coordinate system data, so as to calculate the camera coordinate system data of the target detection point and the camera rotation angle required to adjust the target detection point to the center of the camera's image, i.e., the rotation angle, through the mapping relationship. Then, this embodiment adjusts the target detection point to the center of the camera's image using the calculated camera coordinate system data and the rotation angle, thereby adjusting the detection area of ​​the three-dimensional spatial positioning. This eliminates the need to recalibrate the mapping relationship between the camera coordinate system and the world coordinate system and effectively expands the detection field of view. Then, the second world coordinate system data of the target pixel in the current camera image is calculated based on the mapping relationship, thereby achieving three-dimensional spatial positioning of the target pixel. This embodiment effectively expands the field of view for 3D spatial positioning by adjusting the target detection point to the center of the camera's image. Furthermore, it eliminates the need for recalibrating the mapping relationship after the camera's attitude is adjusted, significantly improving the efficiency of 3D spatial positioning and achieving relatively high-efficiency 3D spatial positioning. Simultaneously, this embodiment effectively improves the accuracy of 3D spatial positioning by constructing a more precise mapping relationship between the camera coordinate system and the world coordinate system.

[0013] According to some embodiments of the present invention, before performing the step of setting a plurality of marker points in a preset positioning space, the method further includes:

[0014] The camera intrinsic parameters are calibrated using a calibration board; wherein the camera intrinsic parameters include the camera's intrinsic parameter matrix and camera distortion.

[0015] According to some embodiments of the present invention, the mapping relationship includes a linear relationship;

[0016] The step of constructing the mapping relationship between the pixel coordinate system and the world coordinate system based on the first world coordinate system data includes:

[0017] The linear relationship between the pixel coordinate system and the world coordinate system is calculated using the PNP algorithm based on the first world coordinate system data.

[0018] According to some embodiments of the present invention, calculating the second world coordinate system data of the target pixel in the current camera frame based on the mapping relationship includes:

[0019] Calculate the depth value of the target pixel based on the mapping relationship;

[0020] Calculate the world pseudo-coordinate data corresponding to the pixel coordinate data based on the depth value and the mapping relationship;

[0021] The second world coordinate system data is calculated based on the world pseudo-coordinate data.

[0022] According to some embodiments of the present invention, the step of calculating the world pseudo-coordinate data corresponding to the pixel coordinate data based on the depth value and the mapping relationship includes:

[0023] Substitute the depth value into the mapping relationship to calculate the translation vector between the camera coordinate system and the world coordinate system;

[0024] Substitute the translation vector into the mapping relationship to solve for the coefficient matrix in the mapping relationship;

[0025] The corresponding pseudo-inverse matrix is ​​obtained from the coefficient matrix;

[0026] The world pseudo-coordinate data is calculated based on the pseudo-inverse matrix and the depth value.

[0027] According to some embodiments of the present invention, the field coordinate data includes latitude and longitude coordinate data and altitude coordinate data;

[0028] The process of obtaining the actual coordinate data of the plurality of marker points includes:

[0029] The latitude and longitude coordinates and altitude coordinates of each of the marked points were measured in the field.

[0030] According to some embodiments of the present invention, the plurality of marker points includes a first marker point and a second marker point; wherein, the first marker point is one of the plurality of marker points, and the second marker point is one of the other marker points besides the first marker point;

[0031] The step of setting several marker points in the preset positioning space includes:

[0032] Set the first marker point at the center of the camera image;

[0033] The second marker is randomly set in the preset positioning space; wherein the number of the second markers in the same plane is less than four.

[0034] On the other hand, embodiments of the present invention also provide a three-dimensional spatial positioning system, including:

[0035] The marking module is used to set a number of marking points in a preset positioning space; wherein, one of the marking points is set at the center of the camera image;

[0036] The acquisition module is used to acquire the actual coordinate data of the plurality of marker points;

[0037] The conversion module is used to convert the real-world coordinate data into coordinate data in the world coordinate system to obtain first world coordinate system data; wherein, the origin of the world coordinate system is the marker point set at the center of the camera image;

[0038] The mapping module is used to construct a mapping relationship between the pixel coordinate system and the world coordinate system based on the first world coordinate system data;

[0039] The first calculation module is used to calculate the camera coordinate system data and rotation angle of the target detection point according to the mapping relationship; wherein, the rotation angle is the camera rotation angle for adjusting the target detection point to the center of the camera image;

[0040] An adjustment module is used to adjust the target detection point to the center of the camera image based on the camera coordinate system data and the rotation angle;

[0041] The second calculation module is used to calculate the second world coordinate system data of the target pixel in the current camera image based on the mapping relationship.

[0042] On the other hand, embodiments of the present invention also provide a three-dimensional spatial positioning system, including:

[0043] At least one processor;

[0044] At least one memory for storing at least one program;

[0045] When the at least one program is executed by the at least one processor, the at least one processor implements the three-dimensional spatial positioning method as described in the above embodiments.

[0046] On the other hand, embodiments of the present invention also provide a computer storage medium storing a processor-executable program, which, when executed by the processor, is used to implement the three-dimensional spatial positioning method as described in the above embodiments. Attached Figure Description

[0047] Figure 1 This is a flowchart of the three-dimensional spatial positioning method provided in the embodiments of the present invention;

[0048] Figure 2This is a block diagram illustrating the principle of a three-dimensional spatial positioning system provided in an embodiment of the present invention.

[0049] Figure 3 This is a three-dimensional schematic diagram of the camera imaging principle provided in an embodiment of the present invention;

[0050] Figure 4 This is a schematic diagram of several marker point settings provided in an embodiment of the present invention;

[0051] Figure 5 This is a schematic diagram illustrating the principle of camera rotation angle calculation provided in an embodiment of the present invention;

[0052] Figure 6 This is a schematic diagram illustrating the principle of calculating the depth value of a target pixel provided in an embodiment of the present invention;

[0053] Figure 7 This is a schematic diagram illustrating the principle of world pseudo-coordinate data to world coordinate data conversion provided in this embodiment of the invention. Detailed Implementation

[0054] The embodiments described in this application should not be considered as limitations on this application. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0055] In the following description, references are made to “some embodiments,” which describe a subset of all possible embodiments. However, it is understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0056] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0057] Video surveillance involves acquiring video image information of the monitored target, monitoring, recording, and reviewing these images, and then responding manually or automatically based on the video image information to achieve monitoring, control, security, and intelligent management of the monitored target. Surveillance technology has gone through many different stages, with image monitoring technology being the core of video surveillance. Simultaneously, video surveillance is also a major component of intelligent surveillance. Intelligent surveillance systems employ image processing, pattern recognition, and computer vision technologies. By adding intelligent video analysis modules to the surveillance system, and leveraging the powerful data processing capabilities of computers, alarms or other actions are issued quickly and optimally, thereby achieving pre-event warnings, in-event processing, and timely post-event evidence collection. Obtaining accurate and effective three-dimensional information and precise positioning from video images is an indispensable part of intelligent surveillance. However, accurately locating the three-dimensional information of the displayed world from the two-dimensional information of video images has always been a challenging task. Among related technologies, there are two main methods commonly used for locating three-dimensional data from two-dimensional information: one is to use a binocular camera to calculate the depth information of an object under known baseline length and camera intrinsic parameters. However, this method requires two cameras, and the ranging accuracy is greatly affected by the pixel count, resulting in limited measurement accuracy when the target is far away. Another method is to use a depth camera for 3D positioning. Due to limitations in sensor accuracy and measurement range, depth cameras also suffer from numerous problems, including narrow measurement range, high noise, small field of view, susceptibility to sunlight interference, and inability to measure transmissive materials.

[0058] One embodiment of the present invention provides a three-dimensional spatial positioning method, system, and storage medium, which can achieve relatively efficient three-dimensional spatial positioning and effectively improve positioning accuracy. (See also...) Figure 1 The method in this embodiment of the invention includes, but is not limited to, steps S110, S120, S130, S140, S150, S160 and S170.

[0059] Specifically, the application process of the method in this embodiment of the invention includes, but is not limited to, the following steps:

[0060] S110: Set several marker points in the preset positioning space. Among them, one marker point is set in the center of the camera frame.

[0061] S120: Obtain the actual coordinate data of several marker points.

[0062] S130: Convert the real-world coordinate data into coordinate data in the world coordinate system to obtain first-world coordinate system data. The origin of the world coordinate system is a marker point set at the center of the camera image.

[0063] S140: Construct the mapping relationship between the pixel coordinate system and the world coordinate system based on the first world coordinate system data.

[0064] S150: Calculate the camera coordinate system data and rotation angle of the target detection point based on the mapping relationship. The rotation angle is the camera rotation angle used to adjust the target detection point to the center of the camera image.

[0065] S160: Adjust the target detection point to the center of the camera image based on the camera coordinate system data and rotation angle.

[0066] S170: Calculate the second world coordinate system data of the target pixel in the current camera frame based on the mapping relationship.

[0067] In the operation of this specific embodiment, several marker points are first set in a preset positioning space. The preset positioning space is the area where three-dimensional spatial positioning is required, such as areas like highways, rivers, and railways. Simultaneously, one of the marker points set in the preset positioning space needs to be placed at the center of the camera's image frame, i.e., the center of the camera's view, to facilitate the calibration of the mapping relationship between the pixel coordinate system and the world coordinate system. For example, refer to... Figure 4Twelve marker points are set in a preset positioning space, one of which is located at the center of the camera frame. Further, this embodiment acquires the actual coordinate data of each marker point, i.e., the ground coordinate data. Then, this embodiment converts the ground coordinate data into coordinate data in the world coordinate system to obtain first world coordinate system data. Specifically, the origin of the world coordinate system constructed in this embodiment is the marker point at the center of the camera frame. This embodiment converts the ground coordinate data of each marker point into a three-dimensional Cartesian coordinate system, i.e., coordinate data in the world coordinate system, to obtain first world coordinate system data. Further, this embodiment constructs a mapping relationship between the pixel coordinate system and the world coordinate system based on the first world coordinate system data. This embodiment uses some 2D-3D matching algorithms to solve the mapping relationship between the pixel coordinate system and the world coordinate system based on the coordinate data of each marker point in the world coordinate system, to obtain the transformation relationship between the pixel coordinate system and the world coordinate system. For example, 2D-3D matching algorithms include the Linear Transformation Algorithm (DLT), the Nonlinear Optimization Algorithm (BA), and the Perspective-n-Point (PNP) algorithm, etc. This embodiment effectively improves the positioning accuracy of 3D spatial positioning by constructing a high-precision mapping relationship between the pixel coordinate system and the world coordinate system. Further, this embodiment calculates the target detection point to obtain camera coordinate system data and rotation angle based on the mapping relationship between the pixel coordinate system and the world coordinate system. Specifically, the rotation angle is the camera rotation angle required to adjust the target detection point to the center position of the camera image. This embodiment calculates the coordinates of any target detection point in the camera coordinate system based on the constructed mapping relationship between the pixel coordinate system and the world coordinate system, and calculates the corresponding rotation angle, so as to adjust the target detection point to the center position of the image by controlling the rotation and image scaling through the camera control system. (Refer to...) Figure 5 In the figure, O w Z is the origin of the world coordinate system. c O c Y c O c and X c O c The resulting coordinate system is the camera coordinate system, O c Let be the origin of the camera coordinate system. For example, when the coordinates of the target detection point P in the camera coordinate system are (x... p ,y p ,z p If the target detection point P rotates along the X-axis, then the rotation angle of the target detection point P is... The rotation angle of the target detection point P in the Y-axis direction is: Then, in this embodiment, the target detection point is adjusted to the center of the camera frame based on the camera coordinate system data and rotation angle. This embodiment adjusts the target detection point to the center of the camera frame using the camera control system based on the calculated camera coordinate system data and rotation angle, thus eliminating the need to recalibrate the mapping relationship between the world coordinate system and the camera coordinate system, effectively improving the efficiency of 3D spatial positioning. Simultaneously, by adjusting the target detection point to the center of the camera frame, the detection area for 3D spatial positioning is adjusted, effectively expanding the field of view of the 3D positioning system and alleviating the problem of narrow measurement range of depth cameras. Furthermore, this embodiment does not require two cameras to form a binocular imaging system; instead, it uses a monocular camera to achieve 3D spatial positioning. Further, this embodiment calculates the second world coordinate system data of the target pixel based on the mapping relationship, thereby achieving more efficient 3D spatial positioning of the target pixel and effectively improving positioning accuracy. It is easy to understand that in practical applications, according to the needs of the three-dimensional spatial positioning task, the corresponding target detection points can be set in the real three-dimensional space. After the target detection points are adjusted to the center of the camera image, any pixel in the current camera image can be used as the target pixel according to the actual positioning needs. Then, the coordinate data of the target pixel in the world coordinate system, i.e., the second world coordinate system data, is calculated through the mapping relationship to realize the three-dimensional spatial positioning of any pixel in the current camera image.

[0068] In some embodiments of the present invention, before performing the step of setting a plurality of marker points in a preset positioning space, the three-dimensional spatial positioning method provided in this embodiment also includes, but is not limited to:

[0069] The camera's intrinsic parameters are calibrated using a calibration board. These intrinsic parameters include the camera's intrinsic parameter matrix and camera distortion.

[0070] In this specific embodiment, camera intrinsic parameters need to be calibrated before 3D spatial positioning. Specifically, camera intrinsic parameters include the camera's intrinsic parameter matrix and camera distortion. Since the degree of distortion varies for each camera lens, multi-camera distortion correction is required through camera calibration. The camera intrinsic parameters mainly include focal length, principal point position, and pixel size ratio to the real environment. This embodiment uses a calibration board to calibrate the camera's intrinsic parameters and distortion parameters to obtain a more accurate intrinsic parameter matrix and camera distortion parameters, thereby improving the fitting degree of the mapping relationship between the constructed pixel coordinate system and the world coordinate system and reducing the error in 3D spatial positioning.

[0071] In some embodiments of the present invention, the mapping relationship includes a linear relationship. Accordingly, the mapping relationship between the pixel coordinate system and the world coordinate system is constructed based on the first world coordinate system data, including but not limited to:

[0072] The linear relationship between the pixel coordinate system and the world coordinate system is calculated using the PNP algorithm based on first-world coordinate system data.

[0073] In this specific embodiment, the mapping relationship between the pixel coordinate system and the world coordinate system includes a linear relationship. Furthermore, this embodiment uses the PNP algorithm to solve for the linear relationship between the pixel coordinate system and the world coordinate system based on the first world coordinate system data. Specifically, refer to... Figure 3 Z w O w Y w O w and X w O w The coordinate system formed is the world coordinate system, O w Z is the origin of the world coordinate system. c O c Y c O c and X c O c The resulting coordinate system is the camera coordinate system, O c Let xoy be the origin of the camera coordinate system, xoy be the image coordinate system, and uv be the pixel coordinate system. In this embodiment, the linear relationship between the pixel coordinate system and the world coordinate system is calculated using the PNP (Perspective-n-Point) algorithm based on the first world coordinate system data, as shown in equation (1):

[0074]

[0075] Where, dx represents the width of each pixel in the pixel coordinate system, dy represents the height of each pixel in the pixel coordinate system, u0 represents the x-coordinate of the origin of the image coordinate system in the pixel coordinate system, v0 represents the y-coordinate of the origin of the image coordinate system in the pixel coordinate system, f is the camera focal length, R is the camera rotation matrix, T is the translation vector, (x w ,y w ,z w ,1) represents the three-dimensional homogeneous coordinates of the marker point in the world coordinate system, (x w ,y w ,z w (u, v, 1) represents the first-world coordinate system data, (u, v, 1) represents the homogeneous pixel coordinates of the marker point, and z represents the first-world coordinate system data. c This is the scaling factor.

[0076] This embodiment uses the PNP algorithm to solve for the linear relationship between camera pixels and the world coordinate system based on the first world coordinate system data of several marker points in the world coordinate system and the camera intrinsic parameters, thereby realizing the transformation from the world coordinate system to the pixel coordinate system.

[0077] In some embodiments of the present invention, the second world coordinate system data of the target pixel in the current camera frame is calculated according to the mapping relationship, including but not limited to:

[0078] The depth value of the target pixel is calculated based on the mapping relationship.

[0079] The world pseudo-coordinate data corresponding to the pixel coordinate data is calculated based on the depth value and the mapping relationship.

[0080] The second world coordinate system data was calculated based on the world pseudo-coordinate data.

[0081] In this specific embodiment, the depth value of the target pixel is first calculated based on the mapping relationship. Then, the world pseudo-coordinate data corresponding to the pixel coordinate data is calculated based on the depth value and the mapping relationship. Finally, the second world coordinate system data is calculated based on the world pseudo-coordinate data. Specifically, refer to... Figure 6 This is a schematic diagram illustrating the calculation of the depth value of the target pixel. Where O... c O w O' represents the depth value at the origin of the world coordinate system. c In the world coordinate system X w O w Y w The perpendicular point on the surface. P is the position of the target pixel in the world coordinate system, and P' is the perpendicular point of point P on the line O. c O w The perpendicular point on, P w This represents the position of the target pixel in the pixel coordinate system. Additionally, ∠PO... c O w Let ∠α and ∠O be the bases for these two points. w O c Let O` be denoted as ∠β, ∠PO c O` is denoted as ∠γ, where ∠β is the camera's rotation angle along the Y-axis (or X-axis). In this embodiment, the depth value (deep = O) at the origin of the world coordinate system is first calculated based on the mapping relationship between the pixel coordinate system and the world coordinate system. c O w Then, in this embodiment, measurements are taken at the origin O in the world coordinate system. w The ratio V of the actual distance to the plane parallel to the camera plane and the pixel distance in both the horizontal and vertical directions. x and V y For example, in actual measurement, the ratio of the actual distance to the pixel distance can be calculated in both the horizontal and vertical directions from any point in the world coordinate system with the same depth value as the origin. Next, this embodiment calculates the ratio based on the depth value passing through the origin O in the world coordinate system. w V is the ratio of the world distance to the plane parallel to the camera plane to the pixel distance in the pixel coordinate system in the horizontal direction.x and the ratio V in the vertical direction y Calculate P w O w The actual distance is shown in equations (2) and (3) below:

[0082] P w O w | x =V x ×du (2)

[0083] P w O w | y =V y ×dy (3)

[0084] Where, du is the horizontal pixel distance between the target pixel and the center point of the camera image, and dy is the vertical pixel distance between the target pixel and the center point of the camera image.

[0085] Furthermore, this embodiment is based on P w O w And O c O w Calculate the size of angle α. Meanwhile, in this embodiment, the line segment O is calculated based on the depth value (deep) at the origin of the world coordinate system and the angle β. c The length of O'. Where O... c O` = deep × cosβ. It's easy to understand that the angle γ is equal to the sum of angle α and angle β. Furthermore, this embodiment is based on line segment O... c The length of O' and the angle γ are used to calculate line segment PO. c The length of. Among them, Next, this embodiment is based on line segment PO c Calculate the length of line segment P`O c The length of P'O c =PO c ×cosα. Therefore, according to line segment P`O c Length and O c O w The length of P`O is calculated. w The distance. Specifically, P'O w =P`O c -O c O w Then, in this embodiment, the total depth change Δd = P`O is calculated based on the depth changes in the horizontal direction and the vertical direction. w | x +P`O w | yTherefore, the depth value d of the target pixel is calculated based on the depth value deep at the origin of the world coordinate system and the total depth change Δd. p =deep+Δd.

[0086] Furthermore, in this embodiment, the world pseudo-coordinate data corresponding to the pixel coordinate data is calculated based on the depth value and mapping relationship. The world pseudo-coordinate data is an intermediate result of converting pixel coordinate data into world coordinate data. Then, in this embodiment, the second world coordinate system data is calculated from the world pseudo-coordinate data. Specifically, refer to... Figure 7 Due to the world pseudo-coordinate point Not in world coordinate system X w O w Y w Surface (Y w =0), therefore it is necessary to put The point moves along the light path to the X coordinate of the world coordinate system. w O w Y w On the surface, i.e., solving for the straight line X of the world coordinate system w O w Y w Surface (Y w The intersection of (=0). This embodiment solves for the intersection of the points. The direction vector is the optical center, which is the origin O of the camera coordinate system. c When the time comes optical path direction The straight line and the X in the world coordinate system w O w Y w Surface (Y w The intersection of (=0) is used to move the world pseudo-coordinate point along the light path. To the X coordinate system w O w Y w Surface (Y w Point P on the plane of the world coordinate system (=0) is used to convert world pseudo-coordinate data to world coordinate data. This embodiment achieves the conversion between pixel coordinate system and world coordinate system by constructing a specified plane conversion algorithm from camera to world coordinate system, which effectively improves the accuracy of 3D spatial positioning.

[0087] In some embodiments of the present invention, world pseudo-coordinate data corresponding to pixel coordinate data is calculated based on depth values ​​and mapping relationships, including but not limited to:

[0088] Substitute the depth value into the mapping relationship to calculate the translation vector between the camera coordinate system and the world coordinate system.

[0089] Substitute the translation vector into the mapping relation to solve for the coefficient matrix in the mapping relation.

[0090] The corresponding pseudo-inverse matrix is ​​obtained from the coefficient matrix.

[0091] World pseudo-coordinate data is calculated based on the pseudo-inverse matrix and depth values.

[0092] In this specific embodiment, the depth value of the target pixel is first substituted into the mapping relationship to calculate the translation vector between the camera coordinate system and the world coordinate system. For example, after calculating the linear relationship between the pixel coordinate system and the world coordinate system using the PNP algorithm based on the first world coordinate system data as shown in equation (1) above, this embodiment substitutes the depth value d of the target pixel... p Substituting into this linear relationship, we can obtain the corresponding translation vector T. p Then, in this embodiment, based on the obtained translation vector T... p Substituting the mapping relationship between the pixel coordinate system and the world coordinate system, the coefficient matrix in the mapping relationship is solved. For example, in this embodiment, the translation vector T is used... p Substituting the values ​​into the linear relationship between the pixel coordinate system and the world coordinate system yields the corresponding coefficient matrix. Then, in this embodiment, the corresponding pseudo-inverse matrix W is obtained by transforming the coefficient matrix. -1 Furthermore, in this embodiment, the world pseudo-coordinate data corresponding to the pixel coordinate point is calculated based on the pseudo-inverse matrix and the depth value. Specifically, in this embodiment, the pixel point is multiplied by the pseudo-inverse matrix W. -1 Thus, the corresponding world pseudo-coordinate data is obtained, and the calculation process is shown in the following formula (4):

[0093]

[0094] Where, in the formula The coordinate data is the world pseudo-coordinate point, and (u,v,1) is the pixel homogeneous coordinate of the marker point.

[0095] In some embodiments of the present invention, the field coordinate data includes latitude and longitude coordinate data and altitude coordinate data. Accordingly, the field coordinate data of several marker points are obtained, including but not limited to:

[0096] The latitude, longitude, and altitude coordinates of each marker point were measured on-site.

[0097] In this specific embodiment, after setting several marker points in a preset positioning space, the latitude, longitude, and altitude coordinates of these marker points are acquired. Specifically, when setting up the corresponding marker points in the preset positioning space, the latitude, longitude, and altitude coordinates of each marker point are measured in the field. For example, this embodiment uses a GPS positioning module to locate and detect the positions of each marker point, thereby obtaining the latitude, longitude, and altitude coordinates of each marker point with relatively high accuracy, providing more accurate data support for subsequently constructing the mapping relationship between the pixel coordinate system and the world coordinate system.

[0098] It should be noted that in some embodiments of the present invention, the latitude and longitude coordinates and altitude coordinates of the actual coordinate data are converted into coordinate data in the world coordinate system to obtain first world coordinate system data. Specifically, in this embodiment, a certain marker point is defined as the origin of the world coordinate system, and the distance from each marker point to the origin along the longitude or latitude direction is calculated. This distance can be arc length or chord length. When the latitude and longitude of the origin of the world coordinate system are (er1, nr1, h1), and the latitude and longitude of the marker point to be calculated are (er2, nr2, h2), and the Earth's radius is R, the arc length in the longitude direction is directly calculated as dx = (er2 - er1) × (R × cos(nr2)), and the arc length in the latitude direction is calculated as dy = (nr2 - nr1) × R. Thus, the world coordinates of the marker point are calculated as (dx, dy, h2), realizing the conversion of the actual coordinate data into coordinate data in the world coordinate system.

[0099] In some embodiments of the present invention, the plurality of marker points includes a first marker point and a second marker point. The first marker point is one of the plurality of marker points, and the second marker point is one of the other marker points besides the first marker point. Accordingly, a plurality of marker points are set in a preset positioning space, including but not limited to:

[0100] Set the first marker point in the center of the camera frame.

[0101] The second marker points are randomly set in a preset positioning space. The number of second marker points in the same plane is less than four.

[0102] In this specific embodiment, several marker points arranged within a preset positioning space are divided into first marker points and second marker points. The first marker point is any one of the several marker points, located at the center of the camera frame. All marker points except the first marker point located at the center of the camera frame are second marker points. In this embodiment, the second marker points are randomly placed within the preset positioning space. The number of second marker points on the same plane should be less than four. For example, this embodiment sets fifteen marker points within the preset positioning space, with one marker point located at the center of the camera frame as the first marker point. Then, the remaining fourteen marker points are randomly placed within the preset positioning space. Furthermore, the number of marker points distributed on the same plane is less than four to avoid too many marker points on the same plane affecting the accuracy of three-dimensional spatial positioning.

[0103] An embodiment of the present invention also provides a three-dimensional spatial positioning system, comprising:

[0104] The marking module is used to set several marker points in a preset positioning space. One marker point is set at the center of the camera image.

[0105] The acquisition module is used to obtain the actual coordinate data of several marker points.

[0106] The conversion module is used to convert field coordinate data into coordinate data in the world coordinate system, resulting in first-world coordinate system data. The origin of the world coordinate system is a marker point set at the center of the camera image.

[0107] The mapping module is used to construct the mapping relationship between the pixel coordinate system and the world coordinate system based on the first world coordinate system data.

[0108] The first calculation module is used to calculate the camera coordinate system data and rotation angle of the target detection point according to the mapping relationship. The rotation angle is the camera rotation angle used to adjust the target detection point to the center of the camera image.

[0109] The adjustment module is used to adjust the target detection point to the center of the camera image based on camera coordinate system data and rotation angle.

[0110] The second calculation module is used to calculate the second world coordinate system data of the target pixel points in the current camera image based on the mapping relationship.

[0111] Reference Figure 2 An embodiment of the present invention also provides a three-dimensional spatial positioning system, comprising:

[0112] At least one processor 210.

[0113] At least one memory 220 is used to store at least one program.

[0114] When at least one program is executed by at least one processor 210, the at least one processor 210 implements the three-dimensional spatial positioning method as described in the above embodiments.

[0115] An embodiment of the present invention also provides a computer-readable storage medium storing computer-executable instructions that are executed by one or more control processors, for example, performing the steps described in the above embodiments.

[0116] It will be understood by those skilled in the art that all or some of the steps and systems in the methods disclosed above can be implemented as software, firmware, hardware, and suitable combinations thereof. Some or all of the physical components can be implemented as software executed by a processor, such as a central processing unit, digital signal processor, or microprocessor, or as hardware, or as an integrated circuit, such as an application-specific integrated circuit. Such software can be distributed on a computer-readable medium, which can include computer storage media (or non-transitory media) and communication media (or transient media). As is known to those skilled in the art, the term computer storage media includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data). Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, digital versatile disc (DVD) or other optical disc storage, magnetic cartridges, magnetic tape, disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and is accessible to a computer. Furthermore, as is known to those skilled in the art, communication media typically contain computer-readable instructions, data structures, program modules, or other data in modulated data signals such as carrier waves or other transmission mechanisms, and may include any information delivery medium.

[0117] The above is a detailed description of the preferred embodiments of the present invention. However, the present invention is not limited to the above embodiments. Those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of the present invention. All such equivalent modifications or substitutions are included within the scope defined by the claims of the present invention.

Claims

1. A three-dimensional spatial positioning method, characterized in that, Includes the following steps: Several marker points are set in a preset positioning space; one of these marker points is set at the center of the camera image. Obtain the actual coordinate data of the aforementioned marker points; The real-world coordinate data is converted into coordinate data in the world coordinate system to obtain first world coordinate system data; wherein, the origin of the world coordinate system is the marker point set at the center of the camera image; A mapping relationship between the pixel coordinate system and the world coordinate system is constructed based on the first world coordinate system data; The camera coordinate system data and rotation angle of the target detection point are calculated based on the mapping relationship; wherein, the rotation angle is the camera rotation angle that adjusts the target detection point to the center of the camera image; The target detection point is adjusted to the center of the camera image based on the camera coordinate system data and the rotation angle. The second world coordinate system data of the target pixel in the current camera frame is calculated based on the mapping relationship.

2. The three-dimensional spatial positioning method according to claim 1, characterized in that, Before performing the step of setting several marker points in the preset positioning space, the method further includes: The camera intrinsic parameters are calibrated using a calibration board; wherein the camera intrinsic parameters include the camera's intrinsic parameter matrix and camera distortion.

3. The three-dimensional spatial positioning method according to claim 1, characterized in that, The mapping relationship includes linear relationships; The step of constructing the mapping relationship between the pixel coordinate system and the world coordinate system based on the first world coordinate system data includes: The linear relationship between the pixel coordinate system and the world coordinate system is calculated using the PNP algorithm based on the first world coordinate system data.

4. The three-dimensional spatial positioning method according to claim 1, characterized in that, The step of calculating the second world coordinate system data of the target pixel in the current camera frame based on the mapping relationship includes: Calculate the depth value of the target pixel based on the mapping relationship; Calculate the world pseudo-coordinate data corresponding to the pixel coordinate data based on the depth value and the mapping relationship; The second world coordinate system data is calculated based on the world pseudo-coordinate data.

5. The three-dimensional spatial positioning method according to claim 4, characterized in that, The step of calculating the world pseudo-coordinate data corresponding to the pixel coordinate data based on the depth value and the mapping relationship includes: Substitute the depth value into the mapping relationship to calculate the translation vector between the camera coordinate system and the world coordinate system; Substitute the translation vector into the mapping relationship to solve for the coefficient matrix in the mapping relationship; The corresponding pseudo-inverse matrix is ​​obtained from the coefficient matrix; The world pseudo-coordinate data is calculated based on the pseudo-inverse matrix and the depth value.

6. The three-dimensional spatial positioning method according to claim 1, characterized in that, The actual coordinate data includes latitude and longitude coordinate data and altitude coordinate data; The process of obtaining the actual coordinate data of the plurality of marker points includes: The latitude and longitude coordinates and altitude coordinates of each of the marked points were measured in the field.

7. The three-dimensional spatial positioning method according to claim 1, characterized in that, The plurality of marker points includes a first marker point and a second marker point; wherein, the first marker point is one of the plurality of marker points, and the second marker point is the other marker point among the plurality of marker points besides the first marker point; The step of setting several marker points in the preset positioning space includes: Set the first marker point at the center of the camera image; The second marker is randomly set in the preset positioning space; wherein the number of the second markers in the same plane is less than four.

8. A three-dimensional spatial positioning system, characterized in that, include: The marking module is used to set a number of marking points in a preset positioning space; wherein, one of the marking points is set at the center of the camera image; The acquisition module is used to acquire the actual coordinate data of the plurality of marker points; The conversion module is used to convert the real-world coordinate data into coordinate data in the world coordinate system to obtain first world coordinate system data; wherein, the origin of the world coordinate system is the marker point set at the center of the camera image; The mapping module is used to construct a mapping relationship between the pixel coordinate system and the world coordinate system based on the first world coordinate system data; The first calculation module is used to calculate the camera coordinate system data and rotation angle of the target detection point according to the mapping relationship; wherein, the rotation angle is the camera rotation angle for adjusting the target detection point to the center of the camera image; An adjustment module is used to adjust the target detection point to the center of the camera image based on the camera coordinate system data and the rotation angle; The second calculation module is used to calculate the second world coordinate system data of the target pixel in the current camera image based on the mapping relationship.

9. A three-dimensional spatial positioning system, characterized in that, include: At least one processor; At least one memory for storing at least one program; When the at least one program is executed by the at least one processor, the at least one processor implements the three-dimensional spatial positioning method as described in any one of claims 1 to 7.

10. A computer storage medium storing a processor-executable program, characterized in that, The processor-executable program, when executed by the processor, is used to implement the three-dimensional spatial positioning method as described in any one of claims 1 to 7.

Images