A method and system for relative pose positioning of a drone

Abstract

The application discloses a kind of unmanned aerial vehicle relative pose positioning method and system, including obtaining unmanned aerial vehicle two-dimensional code marker board image, and the two-dimensional code recognition of unmanned aerial vehicle two-dimensional code marker board image, output multiple two-dimensional code encoding information;Using preset pose conversion formula according to the two-dimensional code length distance data corresponding to each two-dimensional code encoding information, the relative pose of each two-dimensional code encoding information corresponding unmanned aerial vehicle camera is calculated;According to the random sampling consistent algorithm and each unmanned aerial vehicle camera relative pose, determine the fusion unmanned aerial vehicle camera relative pose;The fusion unmanned aerial vehicle camera relative pose is substituted into preset pose conversion formula, and the center relative pose of target unmanned aerial vehicle is calculated;Solved the existing unmanned aerial vehicle relative pose positioning technology is mostly sensitive to the identification distance of unmanned aerial vehicle, leading to the technical problem that the accuracy of unmanned aerial vehicle relative pose is lower.

Description

Technical Field

[0001] This invention relates to the field of UAV visual positioning technology, and in particular to a method and system for UAV relative pose positioning. Background Technology

[0002] In recent years, the capabilities of unmanned aerial vehicle (UAV) systems have been continuously improving, and they have begun to show their potential in military and civilian fields such as urban counter-terrorism reconnaissance, disaster monitoring, logistics distribution, and traffic management, with particularly broad application prospects. UAV positioning is a necessary step for UAVs to perform dynamic missions and is also an important manifestation of their autonomous capabilities. Currently, indoor positioning technology is becoming increasingly mature, with solutions such as UWB positioning, Wi-Fi positioning, Bluetooth positioning, QR code positioning, and SLAM positioning emerging.

[0003] In the field of positioning technology, the advantages of QR codes are being increasingly explored and recognized, and they are being used in the development of positioning and auxiliary positioning modules for intelligent devices such as robots and drones. They are gradually becoming one of the important positioning methods in robot inspection and intelligent logistics scenarios. Especially in situations where repetitive features make it easy to lose positioning by relying solely on sensors such as lasers and vision (e.g., in data center server rack scenarios), QR codes can significantly reduce the possibility of positioning loss by providing posterior information for robot odometers, for example.

[0004] Most existing drone relative pose positioning technologies are based on the principle of camera imaging to calculate the drone's pose relative to the unmanned vehicle platform. However, this process is quite sensitive to the drone's recognition distance. When the drone is close to the unmanned vehicle platform, the QR code mark cannot appear completely in the field of view. When the drone is far from the unmanned vehicle platform, the area occupied by the QR code mark in the field of view is too small, resulting in low accuracy of the drone's relative pose. Summary of the Invention

[0005] This invention provides a method and system for relative pose positioning of unmanned aerial vehicles (UAVs), which addresses the technical problem that most existing UAV relative pose positioning technologies are highly sensitive to the recognition distance of the UAV, resulting in low accuracy of the UAV's relative pose.

[0006] The first aspect of this invention provides a method for relative pose positioning of a UAV, comprising:

[0007] Acquire an image of a drone QR code marker board, perform QR code recognition on the image, and output multiple QR code encoding information;

[0008] The relative pose of the drone camera corresponding to each QR code encoding information is calculated using a preset pose transformation calculation formula based on the QR code length and distance data corresponding to each QR code encoding information.

[0009] The relative pose of the fused drone cameras is determined based on the random sampling consensus algorithm and the relative poses of each drone camera.

[0010] The relative pose of the fused drone camera is substituted into a preset pose conversion formula to calculate the relative pose of the target drone center.

[0011] Optionally, the step of performing QR code recognition on the UAV QR code marking board image and outputting multiple QR code encoding information includes:

[0012] The UAV QR code marker image is preprocessed to generate a denoised QR code marker image;

[0013] The denoised QR code label image is segmented using the maximum inter-class variance method to output a black and white QR code label image.

[0014] A contour recognition algorithm is used to extract the graphic contours from the black and white QR code marking board image, and multiple graphic contours are output.

[0015] A polygon approximation algorithm is used to filter the outlines of each graphic and output the QR code candidate boxes corresponding to each graphic outline.

[0016] Perform perspective transformation on each of the QR code candidate boxes to generate a corrected QR code candidate box corresponding to each of the QR code candidate boxes;

[0017] The candidate boxes for each correction QR code are divided into grids, and multiple pixel grids corresponding to each candidate box for correction QR code are output.

[0018] A pre-set QR code dictionary is used to generate QR code encoding information corresponding to each of the correction QR code candidate boxes based on the number of pixels in each of the pixel grids.

[0019] Optionally, the preset pose transformation calculation formula includes a pose transformation matrix formula group and a relative pose calculation formula; the QR code length and distance data includes the QR code length, the QR code horizontal distance, and the QR code vertical distance; the step of calculating the relative pose of the drone camera corresponding to each QR code encoding information based on the QR code length and distance data corresponding to each QR code encoding information using the preset pose transformation calculation formula includes:

[0020] Based on the QR code length corresponding to each QR code encoding information, determine the corner coordinate data corresponding to each QR code encoding information;

[0021] Based on the corner coordinate data and the pose transformation matrix formula set, determine the pose transformation matrix corresponding to each QR code encoding information;

[0022] The relative pose calculation formula is used to calculate the relative pose of the drone camera corresponding to each QR code encoding information based on the pose transformation matrix, the horizontal distance of each QR code, and the vertical distance of each QR code.

[0023] Optionally, the fused drone camera relative pose includes a fused rotation matrix and a fused translation vector; the step of determining the fused drone camera relative pose based on the random sampling consensus algorithm and the relative poses of each drone camera includes:

[0024] According to a preset sampling ratio, select multiple first intermediate drone camera relative poses from the relative poses of each drone camera.

[0025] The mean value of the translation vector of the first drone camera in the relative pose of each of the first intermediate drone cameras is calculated, and the mean value of the first translation vector is output.

[0026] According to the preset sampling ratio, select multiple second intermediate drone camera relative poses from the relative poses of each drone camera;

[0027] The mean value of the translation vector of the second drone camera in the relative pose of each of the second intermediate drone cameras is calculated, and the mean value of the second translation vector is output.

[0028] Perform a difference calculation on the mean values ​​of the first translation vector and the second translation vector, output the mean error, and compare it with a preset error threshold.

[0029] If the mean error is less than the preset error threshold, then the mean of the second translation vector is used as the fused translation vector.

[0030] Euler angle fusion is performed on the rotation matrix of the second drone camera in the relative pose of each of the second intermediate drone cameras, and the rotation matrix is ​​fused.

[0031] Optionally, the step of performing Euler angle fusion on the rotation matrices of the second UAV cameras in the relative poses of each of the second intermediate UAV cameras, and fusing the rotation matrices, includes:

[0032] The Euler angles corresponding to each rotation matrix of the second UAV camera are calculated using a preset Euler angle calculation formula based on the matrix elements in each rotation matrix of the second UAV camera.

[0033] Calculate the mean of each Euler angle and output the mean Euler angle.

[0034] The fusion rotation matrix is ​​determined based on the Euler angle mean and the preset fusion rotation matrix transformation formula set.

[0035] Optionally, the corner coordinate data includes the coordinates of a first corner, a second corner, a third corner, and a fourth corner. The first corner coordinates include the x-coordinate, y-coordinate, and vertices of the first corner; the second corner coordinates include the x-coordinate, y-coordinate, and vertices of the second corner; the third corner coordinates include the x-coordinate, y-coordinate, and vertices of the third corner; and the fourth corner coordinates include the x-coordinate, y-coordinate, and vertices of the fourth corner. The pose transformation matrix formula group is specifically as follows:

[0036]

[0037] Where s is the pixel depth value; u is the x-coordinate of the pixel in the drone QR code marker image in the image coordinate system; v is the y-coordinate of the pixel in the drone QR code marker image in the image coordinate system; t1 is the first matrix element in the pose transformation matrix; x m Here are the x-coordinates of the corner points, including the x-coordinates of the first, second, third, and fourth corner points; y m The ordinates of the corner points are: z = z + ... m t1 represents the vertical coordinates of the corner points, including the vertical coordinates of the first, second, third, and fourth corner points; t2 represents the second element of the pose transformation matrix; t3 represents the third element of the pose transformation matrix; t4 represents the fourth element of the pose transformation matrix; t5 represents the fifth element of the pose transformation matrix; t6 represents the sixth element of the pose transformation matrix; t7 represents the seventh element of the pose transformation matrix; t8 represents the eighth element of the pose transformation matrix; t9 represents the ninth element of the pose transformation matrix; t 10 t is the tenth element in the pose transformation matrix; 11 t is the eleventh element in the pose transformation matrix; 12 This is the twelfth element in the pose transformation matrix.

[0038] The relative pose calculation formula is as follows:

[0039]

[0040] Among them, R w t represents the drone camera rotation matrix in the relative pose of the drone camera; w d is the translation vector of the drone camera in the relative pose of the drone camera; x d represents the horizontal distance of the QR code. y R represents the vertical distance of the QR code. mt is the pose rotation matrix in the pose transformation matrix; m is the pose translation vector in the pose transformation matrix.

[0041] A second aspect of the present invention provides a relative pose positioning system for an unmanned aerial vehicle (UAV), comprising:

[0042] The acquisition module is used to acquire images of drone QR code marking boards, perform QR code recognition on the drone QR code marking board images, and output multiple QR code encoding information;

[0043] A module is used to calculate the relative pose of the drone camera corresponding to each QR code encoding information based on the QR code length and distance data corresponding to each QR code encoding information using a preset pose transformation calculation formula.

[0044] According to the module, it is used to determine the fused relative pose of the drone cameras based on the random sampling consensus algorithm and the relative pose of each drone camera;

[0045] The calculation module is used to substitute the relative pose of the fused UAV camera into a preset pose transformation formula to calculate the relative pose of the target UAV center.

[0046] A third aspect of the present invention provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor performs the steps of the UAV relative pose positioning method as described in any of the preceding claims.

[0047] The fourth aspect of the present invention provides a computer-readable storage medium having a computer program / instructions stored thereon, wherein the computer program / instructions, when executed by a processor, implement the steps of the UAV relative pose positioning method as described in any of the preceding claims.

[0048] The fifth aspect of the present invention provides a computer program product, comprising a computer program / instructions, which, when executed by a processor, implement the steps of the UAV relative pose positioning method as described in any of the preceding claims.

[0049] As can be seen from the above technical solutions, the present invention has the following advantages:

[0050] The above-mentioned technical solution of the present invention provides a method for relative pose localization of a UAV. First, an image of a UAV QR code marker board is acquired, and QR code recognition is performed on the image to output multiple QR code encoding information. Next, a preset pose transformation calculation formula is used to calculate the relative pose of the UAV camera corresponding to each QR code encoding information based on the QR code length and distance data. The fused relative pose of the UAV camera is determined based on the random sampling consensus algorithm and the relative poses of each UAV camera. Finally, the fused relative pose of the UAV camera is substituted into the preset pose transformation formula to calculate the relative pose of the target UAV center. Based on the above scheme, by combining the multiple QR code encoding information obtained from QR code recognition of the UAV QR code marker board image with the relative poses substituted into the preset pose transformation formula, the preset pose transformation calculation formula, and the random sampling consensus algorithm, the relative pose of the target UAV center is determined. This process integrates the pose information of multiple QR codes, reduces the sensitivity of the UAV's recognition distance, and further improves the accuracy of the UAV's relative pose. Attached Figure Description

[0051] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0052] Figure 1 A flowchart illustrating the steps of a UAV relative pose positioning method provided in an embodiment of the present invention;

[0053] Figure 2 A schematic diagram of a UAV QR code marking board provided in an embodiment of the present invention;

[0054] Figure 3 A flowchart illustrating the steps of another UAV relative pose positioning method provided in an embodiment of the present invention;

[0055] Figure 4 A schematic diagram of the UAV coordinate system, image coordinate system, and ArUco code coordinate system provided in the embodiments of the present invention;

[0056] Figure 5 This is a structural block diagram of a UAV relative pose positioning system provided in an embodiment of the present invention. Detailed Implementation

[0057] This invention provides a method and system for relative pose positioning of unmanned aerial vehicles (UAVs), which addresses the technical problem that most existing UAV relative pose positioning technologies are highly sensitive to the recognition distance of the UAV, resulting in low accuracy of the UAV's relative pose.

[0058] To make the objectives, features, and advantages of this invention more apparent and understandable, the technical solutions of the embodiments of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the embodiments described below are only some embodiments of this invention, and not all embodiments. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0059] Terminology Explanation:

[0060] Pose estimation: In computer vision and robotics, it refers to the process of determining the position (translation) and orientation (rotation) of an object using sensor data or image information. It is typically implemented using mathematical models and algorithms, such as the PnP algorithm.

[0061] Relative pose: refers to the position and orientation of an object relative to other objects or coordinate systems. In this patent, it refers to the position and orientation of the UAV relative to the center of the ArUco (Augmented Reality University of Cordoba) marker board.

[0062] ArUco codes: A QR code labeling system used in computer vision and augmented reality. ArUco codes consist of black and white blocks and have unique identifiers for location and recognition.

[0063] Camera imaging principle: describes how a camera projects a three-dimensional scene onto a two-dimensional plane. This includes aspects such as perspective projection, distortion correction, and camera intrinsic and extrinsic parameters.

[0064] PnP algorithm: Perspective-n-Point algorithm, used to estimate the position and pose of a camera from pairs of points in an image. It solves for the camera pose based on known 3D points and their projections into the image.

[0065] Image segmentation algorithms: Algorithms that divide an image into different regions or objects. Commonly used image segmentation algorithms include thresholding, edge detection, and region growing.

[0066] Subpixel-level corner detection: The detected corners in the image are refined at the subpixel level to improve the accuracy and precision of detection.

[0067] Camera-to-drone rotation and translation matrices: These matrices describe the attitude and positional relationship between the camera and the drone. They are typically used to transform points in the camera coordinate system to the drone coordinate system.

[0068] The relationship between pixel coordinates and platform 3D coordinates: This describes the mapping between pixel coordinates in an image and object coordinates in the real world. The transformation from pixel to actual coordinates is achieved through a combination of camera intrinsic and extrinsic parameter matrices.

[0069] Please see Figure 1 , Figure 1 A flowchart illustrating the steps of a UAV relative pose positioning method provided in an embodiment of the present invention.

[0070] This invention provides a method for relative pose positioning of a UAV, comprising:

[0071] Step 101: Obtain the image of the drone QR code marking board, and perform QR code recognition on the drone QR code marking board image to output multiple QR code encoding information.

[0072] The QR code encoding information is the ID value of the ArUco code (QR code).

[0073] It should be noted that you should refer to [link / reference]. Figure 2 The image of the drone QR code marking board is a color image captured in real time by the drone's camera on the ArUco code marking board affixed to the unmanned vehicle platform. The drone camera is a Pixelplus roll-up camera with an image resolution of 800 pixels × 600 pixels. The ArUco code marking board consists of 61 square ArUco QR codes, including 4 large QR codes and 57 small QR codes. The side length of the large QR codes is 17.6 cm, and the side length of the small QR codes is 4.4 cm.

[0074] Further, after obtaining the UAV QR code label image, the acquired UAV QR code label image is grayscaled and denoised to generate a denoised QR code label image. Then, the OTU threshold segmentation algorithm (maximum inter-class variance method) is used to segment the denoised QR code label image to obtain a black and white QR code label image. A contour recognition algorithm is used to extract the graphic contours of the black and white QR code label image, outputting multiple graphic contours. Then, a polygon approximation algorithm is used to filter each graphic contour, that is, to find the polygons in the image and filter the identified polygons, thereby outputting the QR code candidate boxes corresponding to each graphic contour. Perspective transformation is performed on each QR code candidate box to generate the corresponding corrected QR code candidate boxes. Each corrected QR code candidate box is divided into grids, and multiple pixel grids corresponding to each corrected QR code candidate box are output. Finally, based on the number of pixels in each pixel grid, the initial encoding information corresponding to each corrected QR code candidate box is obtained and matched with a preset QR code dictionary to determine the QR code encoding information corresponding to each corrected QR code candidate box.

[0075] In this embodiment, an image of a drone QR code marker board is acquired, and QR code recognition is performed on the image to output multiple QR code encoding information.

[0076] Step 102: Using the preset pose transformation calculation formula, calculate the relative pose of the drone camera corresponding to each QR code encoding information based on the QR code length and distance data.

[0077] The preset pose transformation calculation formulas include the pose transformation matrix formula group and the relative pose calculation formulas.

[0078] The QR code length and distance data include the QR code length, the horizontal distance, and the vertical distance. The QR code length is the side length of the QR code. The horizontal distance is the horizontal distance from the center of the recognized QR code to the center of the ArUco code marking board. The vertical distance is the vertical distance from the center of the recognized QR code to the center of the ArUco code marking board.

[0079] It should be noted that after obtaining the QR code encoding information for each QR code according to the above steps, we can know the QR code length, horizontal distance, and vertical distance for each QR code. Based on the QR code length *l* corresponding to each QR code encoding information, we determine the corner coordinate data corresponding to each QR code encoding information. Since each QR code has four corner points, the corner coordinate data for each QR code includes the coordinates of the first corner point. Second corner point coordinates Coordinates of the third corner point and the coordinates of the fourth corner point The coordinates of the first corner point include the x-coordinate, y-coordinate, and vertices of the first corner point; the coordinates of the second corner point include the x-coordinate, y-coordinate, and vertices of the second corner point; the coordinates of the third corner point include the x-coordinate, y-coordinate, and vertices of the third corner point; and the coordinates of the fourth corner point include the x-coordinate, y-coordinate, and vertices of the fourth corner point.

[0080] Furthermore, based on the coordinate data of each corner point and the formula set of pose transformation matrices, the pose transformation matrix corresponding to each QR code encoding information is determined. Specifically, the four corner points of each QR code are substituted into the formula set of pose transformation matrices, thus generating 12 equations. Solving these equations yields the pose transformation matrix corresponding to each QR code. The pose transformation matrix is ​​the pose transformation matrix from the pixel plane center to the QR code center, and it includes the pose rotation matrix R. m and pose translation vector t m The pose transformation matrix formula set is as follows:

[0081]

[0082] Where s is the pixel depth value; u is the x-coordinate of the pixel in the drone QR code marker image in the image coordinate system; v is the y-coordinate of the pixel in the drone QR code marker image in the image coordinate system; t1 is the first matrix element in the pose transformation matrix; x m Here are the x-coordinates of the corner points, including the x-coordinates of the first, second, third, and fourth corner points; y m The ordinates of the corner points are: z = z + ... m t1 represents the vertical coordinates of the corner points, including the vertical coordinates of the first, second, third, and fourth corner points; t2 represents the second element of the pose transformation matrix; t3 represents the third element of the pose transformation matrix; t4 represents the fourth element of the pose transformation matrix; t5 represents the fifth element of the pose transformation matrix; t6 represents the sixth element of the pose transformation matrix; t7 represents the seventh element of the pose transformation matrix; t8 represents the eighth element of the pose transformation matrix; t9 represents the ninth element of the pose transformation matrix; t 10 t is the tenth element in the pose transformation matrix; 11 t is the eleventh element in the pose transformation matrix; 12 This is the twelfth element in the pose transformation matrix.

[0083] Furthermore, using the relative pose calculation formula, based on each pose transformation matrix, the horizontal distance of each QR code, and the vertical distance of each QR code, the relative pose of the UAV camera corresponding to each QR code encoding information is calculated. The relative pose of the UAV camera is the relative pose between the UAV camera and the center of the ArUco code marking board, which includes the UAV camera rotation matrix R. w and the drone camera translation vector t w The formula for relative pose calculation is as follows:

[0084]

[0085] Among them, R w t is the rotation matrix in the relative pose of the drone camera; w d is the translation vector in the relative pose of the drone camera; x d represents the horizontal distance of the QR code. y R represents the vertical distance of the QR code. m t is the rotation matrix in the pose transformation matrix; m is the translation vector in the pose transformation matrix.

[0086] In this embodiment, a preset pose transformation calculation formula is used to calculate the relative pose of the drone camera corresponding to each QR code encoding information based on the QR code length and distance data.

[0087] Step 103: Determine the relative pose of the fused drone cameras based on the random sampling consensus algorithm and the relative pose of each drone camera.

[0088] The fusion of relative pose of drone cameras includes the fusion of rotation matrix and the fusion of translation vector.

[0089] It should be noted that, according to a preset sampling ratio, multiple first intermediate drone camera relative poses are randomly selected from all drone camera relative poses; the average value of the first drone camera translation vector in each first intermediate drone camera relative pose is calculated, and the average value of the first translation vector is output; then, according to a preset sampling ratio, multiple second intermediate drone camera relative poses are randomly selected from all drone camera relative poses; the average value of the second drone camera translation vector in each second intermediate drone camera relative pose is calculated, and the average value of the second translation vector is output; the difference between the average value of the first translation vector and the average value of the second translation vector is calculated, and the average error is output and compared with a preset error threshold; if the average error is less than the preset error threshold, the average value of the second translation vector is used as the fused translation vector; Euler angle fusion is performed on the rotation matrix of the second drone camera in each second intermediate drone camera relative pose, and the rotation matrix is ​​fused; wherein, the preset sampling ratio and the preset error threshold can be set as needed, for example, the preset sampling ratio can be set to 0.7, but the present invention is not limited thereto.

[0090] In this embodiment, the relative pose of the fused drone cameras is determined based on the random sampling consensus algorithm and the relative pose of each drone camera.

[0091] Step 104: Substitute the relative pose of the fused drone camera into the preset pose transformation formula to calculate the relative pose of the target drone center.

[0092] It should be noted that this is based on the pre-measured pose transformation relationship from the camera to the center of the drone, i.e., the preset rotation matrix R. bc and the preset translation vector t bc The relative pose of the target drone center is calculated using a preset pose transformation formula based on the relative pose of the fused drone camera. The relative pose of the target drone center is the pose of the drone center relative to the center of the ArUco code marker board. The preset pose transformation formula is as follows:

[0093]

[0094] Among them, R bw t is the target rotation matrix in the relative pose of the target UAV center; bw R is the target translation vector in the relative pose of the target UAV center; bc For the preset rotation matrix; t bc R is the preset translation vector; f For the fusion rotation matrix; t f This is the fused translation vector.

[0095] In this embodiment, the relative pose of the fused drone camera is substituted into a preset pose conversion formula to calculate the relative pose of the target drone center.

[0096] In this embodiment of the invention, a method for relative pose localization of a UAV is provided. First, an image of a UAV QR code marker board is acquired, and QR code recognition is performed on the image to output multiple QR code encoding information. Next, a preset pose transformation calculation formula is used to calculate the relative pose of the UAV camera corresponding to each QR code encoding information based on the QR code length and distance data. Then, a fused relative pose of the UAV camera is determined based on a random sampling consensus algorithm and the relative poses of each UAV camera. Finally, the fused relative pose of the UAV camera is substituted into a preset pose transformation formula to calculate the relative pose of the target UAV center. Based on the above scheme, by combining the multiple QR code encoding information obtained from QR code recognition of the UAV QR code marker board image with the relative poses substituted into the preset pose transformation formula, the preset pose transformation calculation formula, and the random sampling consensus algorithm, the relative pose of the target UAV center is determined. This process integrates the pose information of multiple QR codes, reduces the sensitivity of the UAV's recognition distance, and further improves the accuracy of the UAV's relative pose.

[0097] Please see Figure 3 , Figure 3 A flowchart illustrating the steps of another UAV relative pose positioning method provided in an embodiment of the present invention.

[0098] This invention provides a method for relative pose positioning of a UAV, comprising:

[0099] Step 301: Obtain the image of the drone QR code marking board, and perform QR code recognition on the drone QR code marking board image to output multiple QR code encoding information.

[0100] Further, step 301 may include the following sub-steps:

[0101] S11. Preprocess the drone QR code label image to generate a denoised QR code label image;

[0102] It should be noted that the preprocessing of the drone QR code label image is divided into two stages: grayscale conversion and denoising of the acquired color image (drone QR code label image). Specifically, the drone QR code label image is first grayscale converted to obtain a grayscale QR code label image, and then the grayscale QR code label image is denoised to generate a denoised QR code label image.

[0103] S12. Use the maximum inter-class variance method to segment the denoised QR code label image and output a black and white QR code label image.

[0104] S13. Use a contour recognition algorithm to extract the graphic contours from the black and white QR code marking board image and output multiple graphic contours;

[0105] S14. Use a polygon approximation algorithm to filter the contours of each graphic and output the QR code candidate boxes corresponding to each graphic contour.

[0106] It should be noted that after obtaining multiple graphic contours, polygons in the image are found through a polygon approximation algorithm, and the identified polygons are filtered to output the QR code candidate boxes corresponding to each graphic contour. The ArUco code is composed of a square frame, and the square frame is still a convex quadrilateral when projected onto the pixel plane. Based on this property, the candidate box of each ArUco code, i.e., the QR code candidate box, can be obtained.

[0107] S15. Perform perspective transformation on each QR code candidate box to generate the corresponding corrected QR code candidate box.

[0108] S16. Divide each candidate frame of the correction QR code into a grid and output the multiple pixel grids corresponding to each candidate frame of the correction QR code.

[0109] S17. Using a pre-set QR code dictionary, generate QR code encoding information corresponding to each correction QR code candidate box based on the number of pixels in each pixel grid.

[0110] It should be noted that after obtaining each QR code candidate box, the position of each QR code can be known. A perspective transformation is performed on each QR code candidate box, and then a grid is divided. The number of black and white pixels in each grid is counted. If there are more black pixels, it is 0, and if there are fewer black pixels, it is 1. The initial encoding information of each corrected QR code candidate box is matched with the preset ArUco dictionary to finally determine the ID value of each recognized ArUco code, that is, the QR code encoding information.

[0111] In this embodiment, an image of a drone QR code marker board is acquired, and QR code recognition is performed on the image to output multiple QR code encoding information.

[0112] Step 302: Based on the QR code length corresponding to each QR code encoding information, determine the corner coordinate data corresponding to each QR code encoding information.

[0113] Corner coordinate data includes the coordinates of the first corner point. Second corner point coordinates Coordinates of the third corner point and the coordinates of the fourth corner point The coordinates of the first corner point include the x-coordinate, y-coordinate, and vertices of the first corner point; the coordinates of the second corner point include the x-coordinate, y-coordinate, and vertices of the second corner point; the coordinates of the third corner point include the x-coordinate, y-coordinate, and vertices of the third corner point; and the coordinates of the fourth corner point include the x-coordinate, y-coordinate, and vertices of the fourth corner point.

[0114] In this embodiment, the corner coordinate data corresponding to each QR code encoding information is determined based on the QR code length corresponding to each QR code encoding information.

[0115] Step 303: Determine the pose transformation matrix corresponding to each QR code encoding information based on the coordinate data of each corner point and the pose transformation matrix formula group.

[0116] It should be noted that you should refer to [link / reference]. Figure 4 The origin of the image coordinate system is defined as the center of the pixel plane. c The origin of the ArUco code coordinate system is the center o of the ArUco code square frame. m The origin of the ArUco plate coordinate system is the geometric center of the plate. w The origin of the UAV's coordinate system is the UAV's centroid o. u The transformation relationship from the image coordinate system to the ArUco code coordinate system is constructed using a camera projection model, specifically as follows:

[0117]

[0118] Where s is the pixel depth value; u is the x-coordinate of the pixel in the drone QR code marker image in the image coordinate system; v is the y-coordinate of the pixel in the drone QR code marker image in the image coordinate system; K is the camera intrinsic parameter; x m Let y be the x-coordinate of a point in space in the ArUco code coordinate system; m z is the ordinate of a point in space in the ArUco code coordinate system; m Let be the vertical coordinate of a point in space in the ArUco code coordinate system.

[0119] Furthermore, by combining the intrinsic parameter matrix and pose transformation matrix in the above transformation formula, we can obtain:

[0120]

[0121] Where s is the pixel depth value; u is the x-coordinate of the pixel in the drone QR code marker image in the image coordinate system; v is the y-coordinate of the pixel in the drone QR code marker image in the image coordinate system; x m Let y be the x-coordinate of a point in space in the ArUco code coordinate system; mz is the ordinate of a point in space in the ArUco code coordinate system; m t1 is the vertical coordinate of a point in space in the ArUco code coordinate system; t2 is the second element of the pose transformation matrix; t3 is the third element of the pose transformation matrix; t4 is the fourth element of the pose transformation matrix; t5 is the fifth element of the pose transformation matrix; t6 is the sixth element of the pose transformation matrix; t7 is the seventh element of the pose transformation matrix; t8 is the eighth element of the pose transformation matrix; t9 is the ninth element of the pose transformation matrix; t 10 t is the tenth element in the pose transformation matrix; 11 t is the eleventh element in the pose transformation matrix; 12 This is the twelfth element in the pose transformation matrix.

[0122] Furthermore, based on the above formulas and their integration, a set of pose transformation matrix formulas is obtained. Each QR code has four corner points. Based on the QR code encoding information of each QR code, the length of each QR code is 'l', and the coordinates of the four corner points of each QR code in the ArUco code coordinate system are the coordinates of the first corner point. Second corner point coordinates Coordinates of the third corner point and the coordinates of the fourth corner point Substituting the corner coordinates of each QR code into the pose transformation matrix formula, that is, replacing the coordinates of a point in space in the ArUco code coordinate system with the corner coordinates, 12 equations can be listed, and the pose transformation matrix R from the pixel plane center to the code center can be obtained. m and t m That is, to calculate the pose transformation matrix corresponding to each QR code.

[0123] In this embodiment, the pose transformation matrix corresponding to each QR code encoding information is determined based on the coordinate data of each corner point and the pose transformation matrix formula set.

[0124] Step 304: Using the relative pose calculation formula, calculate the relative pose of the drone camera corresponding to each QR code encoding information based on each pose transformation matrix, the horizontal distance of each QR code, and the vertical distance of each QR code.

[0125] It should be noted that by using the relative pose calculation formula, based on the recognized QR code encoding information (ArUco code ID value), the horizontal and vertical distances from the center of the recognized ArUco code to the center of the ArUco code marking board, as well as the pose transformation matrix corresponding to each QR code, can be determined, thus obtaining the relative pose R of the drone camera corresponding to each QR code. w and t w .

[0126] In this embodiment, the relative pose calculation formula is used to calculate the relative pose of the drone camera corresponding to each QR code encoding information based on each pose transformation matrix, the horizontal distance of each QR code, and the vertical distance of each QR code.

[0127] Step 305: Determine the relative pose of the fused drone cameras based on the random sampling consensus algorithm and the relative pose of each drone camera.

[0128] The fusion of relative pose of drone cameras includes the fusion of rotation matrix and the fusion of translation vector.

[0129] Furthermore, step 305 may include the following sub-steps:

[0130] S51. Select multiple first intermediate drone camera relative poses from the relative poses of each drone camera according to a preset sampling ratio.

[0131] S52. Calculate the mean value of the translation vector of the first UAV camera in the relative pose of each first intermediate UAV camera, and output the mean value of the first translation vector.

[0132] S53. Select multiple second intermediate drone camera relative poses from the relative poses of each drone camera according to a preset sampling ratio.

[0133] S54. Calculate the mean value of the translation vector of the second drone camera in the relative pose of each second intermediate drone camera, and output the mean value of the second translation vector.

[0134] S55. Perform a difference calculation on the mean of the first translation vector and the mean of the second translation vector, output the mean error and compare it with a preset error threshold.

[0135] S56. If the mean error is less than the preset error threshold, then the mean of the second translation vector is used as the fused translation vector.

[0136] It should be noted that a certain proportion of the relative poses of all drone cameras are randomly selected as the first intermediate drone camera relative poses. The mean of the first drone camera translation vector among all the first intermediate drone camera relative poses is calculated, and the mean of the first translation vector is output. The formula for calculating the mean of the translation vector is as follows:

[0137]

[0138] Among them, t avg The mean of the translation vectors, which includes the mean of the first translation vector and the mean of the second translation vector; t wiLet be the translation vector of the i-th drone camera, which includes the translation vectors of the first and second drone cameras; n is the number of QR codes.

[0139] Furthermore, a certain proportion of the relative poses of all drone cameras are randomly selected as the second intermediate drone camera relative poses. The mean of the translation vectors of the second drone cameras in all the relative poses of the second intermediate drone cameras is calculated, and the mean of the second translation vectors is output. The difference between the mean of the second translation vectors and the mean of the first translation vectors is calculated to obtain the mean error e. This mean error e is compared with a preset error threshold. If the mean error is less than the preset error threshold, the mean of the second translation vectors is used as the fused translation vector t. f If the mean error is greater than or equal to a preset error threshold, the mean of the second translation vector is used as the new mean of the first translation vector, and execution jumps to S53 until the mean error is less than the preset error threshold. The mean of the second translation vector when the mean error is less than the preset error threshold is then used as the fused translation vector t. f .

[0140] S57. Perform Euler angle fusion on the rotation matrix of the second drone camera in the relative pose of each second intermediate drone camera, and fuse the rotation matrix.

[0141] Furthermore, S57 may include the following sub-steps:

[0142] S571. Using a preset Euler angle calculation formula, calculate the Euler angles corresponding to the rotation matrix of each second UAV camera based on the matrix elements in the rotation matrix of each second UAV camera.

[0143] Euler angles include the first Euler angle α i Second Euler angle β i and the third Euler angle γ i .

[0144] It should be noted that, in order to obtain the fused rotation matrix, this invention adopts the most common Euler angle rotation method, z-x'-z", that is, a point in space is first rotated around the z-axis, then around the x-axis, and finally around the z-axis to obtain the target point. Under this definition of Euler angle rotation, a preset Euler angle calculation formula is used to calculate the Euler angle α corresponding to each second UAV camera rotation matrix based on the matrix elements in each second UAV camera rotation matrix. i β i γ i The preset Euler angle calculation formula is as follows:

[0145]

[0146] Where, α iLet β be the first Euler angle of the rotation matrix of the i-th second UAV camera; i γ is the second Euler angle of the rotation matrix of the i-th second UAV camera; i R is the third Euler angle of the rotation matrix of the i-th second UAV camera; i13 R is the matrix element in the first row and third column of the rotation matrix of the i-th second UAV camera; i23 R is the matrix element in the second row and third column of the rotation matrix of the i-th second UAV camera; i33 R is the matrix element in the third row and third column of the rotation matrix of the i-th second UAV camera; i31 For; R i32 Let be the matrix element in the third row and second column of the rotation matrix of the i-th second drone camera.

[0147] S572. Perform the mean calculation on each Euler angle and output the mean Euler angle.

[0148] It should be noted that the first Euler angle α corresponding to each second UAV camera rotation matrix is... i Second Euler angle β i and the third Euler angle γ i The mean of Euler angles is obtained by performing a mean operation, which includes the first, second, and third Euler angle mean. The calculation process for the Euler angle mean is as follows:

[0149]

[0150]

[0151]

[0152] Where, α f The mean of the first Euler angles; β f The mean of the second Euler angles; γ f The mean of the third Euler angle; α i Let β be the first Euler angle of the rotation matrix of the i-th second UAV camera; i γ is the second Euler angle of the rotation matrix of the i-th second UAV camera; i Let be the third Euler angle of the i-th second drone camera rotation matrix; n is the number of second drone camera rotation matrices.

[0153] S573. Determine the fusion rotation matrix based on the Euler angle mean and the preset fusion rotation matrix transformation formula set.

[0154] It should be noted that substituting the mean values ​​of the first, second, and third Euler angles into the preset fusion rotation matrix transformation formula set yields the fusion rotation matrix. The preset fusion rotation matrix transformation formula set is as follows:

[0155]

[0156]

[0157]

[0158] R f =R z1 R x R z2 ;

[0159] Among them, R f For the fusion rotation matrix; R z1 R is the first vertical axis rotation matrix; x R is the rotation matrix along the horizontal axis; z2 α is the rotation matrix for the second vertical axis; f The mean of the first Euler angles; β f The mean of the second Euler angles; γ f It is the mean of the third Euler angle.

[0160] In this embodiment, the relative pose of the fused drone cameras is determined based on the random sampling consensus algorithm and the relative pose of each drone camera.

[0161] Step 306: Substitute the relative pose of the fused UAV camera into the preset pose transformation formula to calculate the relative pose of the target UAV center.

[0162] It should be noted that the pose transformation relationship R from the pre-measured camera to the center of the drone is... bc and t bc Based on the preset pose transformation formula, the obtained fusion result (fusion rotation matrix and fusion translation vector) can be transformed into the pose R of the UAV center relative to the board center. bw and t bw .

[0163] In this embodiment, the relative pose of the fused drone camera is substituted into a preset pose conversion formula to calculate the relative pose of the target drone center.

[0164] For comparison of technical effects, existing technologies can be referenced. Current vision-based QR code tags are mainly used for relative pose (position and attitude) estimation of UAVs. These QR code tags consist of black and white squares or other geometric shapes, possessing relatively obvious visual features. Currently, commonly used QR code tags include ArUco, AprilTag, and ARTag (Augmented Reality Tag). Among them, ArUco codes have a faster detection speed and higher detection accuracy, and are often used to solve relative pose. In relative pose calculation, the UAV acquires a QR code image on the unmanned vehicle platform through its onboard camera, uses visual algorithms to extract the corner feature information of the QR code, and then solves a system of equations based on the relationship between the image coordinate system and the platform's three-dimensional coordinates to calculate the UAV's pose relative to the platform. Currently, the pose estimation scheme most similar to this invention is based on point projection. This method is based on the camera imaging principle, that is, a point in three-dimensional space is projected onto a pixel plane by the camera. This method first obtains the pixel coordinates of QR code feature points in the image through image processing. Based on the known QR code size information, the corresponding three-dimensional spatial coordinates of these points are calculated. A system of equations is then established using the pixel-spatial coordinate correspondence of these points to solve for the pose of the UAV relative to the unmanned vehicle platform. Additionally, there is a line projection-based scheme. This scheme utilizes the projection relationships of multiple straight lines in space onto the image plane, along with known line slopes, to solve for the geometric transformation between the spatial coordinate system and the pixel coordinate system, thereby obtaining the relative pose. While the line projection-based scheme can make more efficient use of geometric information, the markers are prone to falling out of the field of view in close-range detection, so it is less commonly used.

[0165] However, the point projection-based schemes mentioned above are quite sensitive to recognition distance. When the drone is close to the unmanned vehicle platform, the QR code marker cannot appear completely within the field of view, making it impossible to calculate the pose. When the drone is far from the platform, the area occupied by the QR code marker within the field of view is too small, making recognition difficult. Furthermore, this scheme suffers from significant pose estimation errors when the marker is occluded or in reflective, rainy, or foggy weather. The line projection-based scheme, limited by specific spatial linear geometry, also fails to meet the requirements for close-range pose estimation.

[0166] To address the aforementioned problems, this invention proposes a method for relative pose localization of unmanned aerial vehicles (UAVs). ArUco markers are affixed to an unmanned vehicle platform, and a camera is activated to capture images in real time. Candidate bounding boxes are detected in the captured images to segment the regions containing ArUco codes. All identified ArUco codes are then numbered. ArUco codes consist of multiple black and white squares, and their unique numbers can be determined based on the number and arrangement of these squares. Corner point correction is then performed. The second step detects pixel-level corner points. This step constructs a local window centered on the corner point location for further refinement, obtaining sub-pixel-level corner points. Pose estimation is then performed based on these corner points. The PnP algorithm is used for solving this problem. The pose of the camera relative to each ArUco code obtained in step five is transformed to a localization center. Based on the numbering information detected in step three and the known distance from the code center to the board center, the code pose can be unified to the center of the ArUco marker board. The localization results are fused. The pose results obtained in step six are iterated using the RANSAC algorithm until a result with minimal error is selected. The pose transformation matrix of the camera and the UAV center, along with the result obtained in step six, is used to solve for the UAV's pose relative to the ArUco marker board center. Addressing the shortcomings of existing solutions, a QR code array and a relative pose estimation algorithm are designed to enable the UAV to accurately estimate its relative pose to the unmanned vehicle platform in various application scenarios. Specific shapes and arrangements improve the accuracy and robustness of ArUco code recognition. This layout design allows ArUco codes to be quickly and accurately recognized in different scenarios, providing a reliable foundation for information recognition and localization, and enabling rapid recognition of ArUco codes with complex layouts. This algorithm combines image processing and pattern recognition technologies, exhibiting strong adaptability and robustness. By optimizing the algorithm, accurate recognition of ArUco codes under different angles and lighting conditions is achieved, providing reliable technical support for information extraction and positioning. It also exhibits good adaptability to various application scenarios and can be widely used in fields such as industrial production, logistics management, and intelligent transportation. Through customized ArUco code layout and optimized recognition algorithms, the system meets the information recognition and positioning needs of different scenarios, providing a reliable solution for information management and positioning in various application scenarios.

[0167] In summary, current relative positioning methods suffer from insufficient accuracy, poor robustness, or high cost, which affect the performance and reliability of air-to-ground cooperative systems. This invention proposes a UAV relative pose positioning method that solves the problem of UAV acquisition and relative pose estimation in air-to-ground cooperative systems. It can quickly and accurately identify ArUco codes in images and use the RANSAC algorithm to estimate the UAV's pose. Compared to traditional methods, this invention offers faster recognition speed and higher positioning accuracy. Furthermore, it considers application scenarios under different environmental conditions and designs a more adaptable ArUco code layout and recognition algorithm. Even under conditions of changing lighting, occlusion, or interference, it can still stably identify ArUco codes and achieve accurate position positioning. By placing multiple ArUco codes and establishing their positional relationships, this invention can integrate multiple positional information and improve system robustness. Even if one ArUco code fails or is occluded, the information from other ArUco codes can still be used to accurately locate the UAV. This invention also considers application requirements under different distances and environmental conditions, designing ArUco code layouts of different sizes and implementing corresponding recognition algorithms. This enables UAVs to effectively identify and locate ArUco codes at different distances, improving the adaptability and stability of the entire air-to-ground cooperative system. Therefore, compared with existing technologies, this invention has higher recognition speed, stronger robustness, greater location information integration capabilities, and better environmental adaptability, enabling more effective precise positioning and navigation of the air-to-ground cooperative system.

[0168] In this embodiment of the invention, a method for relative pose localization of a UAV is provided. First, an image of a UAV QR code marker board is acquired, and QR code recognition is performed on the image to output multiple QR code encoding information. Next, a preset pose transformation calculation formula is used to calculate the relative pose of the UAV camera corresponding to each QR code encoding information based on the QR code length and distance data. Then, a fused relative pose of the UAV camera is determined based on a random sampling consensus algorithm and the relative poses of each UAV camera. Finally, the fused relative pose of the UAV camera is substituted into a preset pose transformation formula to calculate the relative pose of the target UAV center. Based on the above scheme, by combining the multiple QR code encoding information obtained from QR code recognition of the UAV QR code marker board image with the relative poses substituted into the preset pose transformation formula, the preset pose transformation calculation formula, and the random sampling consensus algorithm, the relative pose of the target UAV center is determined. This process integrates the pose information of multiple QR codes, reduces the sensitivity of the UAV's recognition distance, and further improves the accuracy of the UAV's relative pose.

[0169] Please see Figure 5 , Figure 5 This is a structural block diagram of a UAV relative pose positioning system provided in an embodiment of the present invention.

[0170] The present invention provides a relative pose positioning system for unmanned aerial vehicles, comprising:

[0171] The acquisition module 501 is used to acquire the image of the drone QR code marking board, perform QR code recognition on the image of the drone QR code marking board, and output multiple QR code encoding information;

[0172] Module 502 is used to calculate the relative pose of the drone camera corresponding to each QR code encoding information based on the QR code length and distance data corresponding to each QR code encoding information using a preset pose transformation calculation formula.

[0173] According to module 503, it is used to determine the relative pose of the fused drone cameras based on the random sampling consensus algorithm and the relative pose of each drone camera;

[0174] The calculation module 504 is used to substitute the relative pose of the fused UAV camera into a preset pose transformation formula to calculate the relative pose of the target UAV center.

[0175] Furthermore, module 501 includes:

[0176] The preprocessing submodule is used to preprocess the drone QR code label image to generate a denoised QR code label image;

[0177] The segmentation submodule is used to segment the denoised QR code label image using the maximum inter-class variance method and output a black and white QR code label image.

[0178] The extraction submodule is used to extract graphic contours from the black and white QR code marker image using a contour recognition algorithm and output multiple graphic contours.

[0179] The filtering submodule is used to filter the contours of each graphic using a polygon approximation algorithm and output the QR code candidate boxes corresponding to each graphic contour.

[0180] The transformation submodule is used to perform perspective transformation on each QR code candidate box to generate the corresponding corrected QR code candidate box.

[0181] The partitioning submodule is used to divide each candidate frame of the correction QR code into a grid and output the multiple pixel grids corresponding to each candidate frame of the correction QR code.

[0182] The statistics submodule is used to generate QR code encoding information corresponding to each correction QR code candidate box based on the number of pixels in each pixel grid according to the pre-set QR code dictionary.

[0183] Furthermore, the preset pose transformation calculation formula includes a pose transformation matrix formula group and a relative pose calculation formula; the QR code length and distance data includes the QR code length, the horizontal distance of the QR code, and the vertical distance of the QR code; module 502 is used, including:

[0184] The corner coordinate submodule is used to determine the corner coordinate data corresponding to each QR code encoding information based on the QR code length corresponding to each QR code encoding information.

[0185] The pose transformation submodule is used to determine the pose transformation matrix corresponding to each QR code encoding information based on the coordinate data of each corner point and the pose transformation matrix formula group.

[0186] The relative pose submodule is used to calculate the relative pose of the drone camera corresponding to each QR code encoding information by using the relative pose calculation formula based on each pose transformation matrix, the horizontal distance of each QR code, and the vertical distance of each QR code.

[0187] Furthermore, the fused drone camera relative pose includes a fused rotation matrix and a fused translation vector, which, according to module 503, include:

[0188] The first submodule is used to select multiple first intermediate drone camera relative poses from the relative poses of each drone camera according to a preset sampling ratio.

[0189] The second submodule is used to perform mean calculation on the translation vector of the first drone camera in the relative pose of each first intermediate drone camera, and output the mean of the first translation vector.

[0190] The third submodule is used to select multiple second intermediate drone camera relative poses from the relative poses of each drone camera according to a preset sampling ratio.

[0191] The fourth submodule is used to calculate the mean of the translation vectors of the second drone cameras in the relative poses of each second intermediate drone camera, and output the mean of the second translation vectors.

[0192] The fifth submodule is used to perform a difference calculation on the mean of the first translation vector and the mean of the second translation vector, output the mean error and compare it with a preset error threshold.

[0193] The sixth submodule is used to take the mean of the second translation vector as the fused translation vector if the mean error is less than a preset error threshold.

[0194] The seventh submodule is used to perform Euler angle fusion on the rotation matrix of the second drone camera in the relative pose of each second intermediate drone camera, and fuse the rotation matrix.

[0195] Furthermore, the seventh submodule includes:

[0196] Euler angle unit, used to calculate the Euler angles corresponding to each second UAV camera rotation matrix based on the matrix elements in each second UAV camera rotation matrix using a preset Euler angle calculation formula;

[0197] Euler angle mean unit, used to perform mean calculation on each Euler angle and output the mean Euler angle;

[0198] The fusion matrix unit is used to determine the fusion rotation matrix based on the Euler angle mean and a preset set of fusion rotation matrix transformation formulas.

[0199] Optionally, the corner coordinate data includes the coordinates of a first corner, a second corner, a third corner, and a fourth corner. The coordinates of the first corner include the x-coordinate, y-coordinate, and vertices of the first corner; the coordinates of the second corner include the x-coordinate, y-coordinate, and vertices of the second corner; the coordinates of the third corner include the x-coordinate, y-coordinate, and vertices of the third corner; and the coordinates of the fourth corner include the x-coordinate, y-coordinate, and vertices of the fourth corner. The pose transformation matrix formula set is as follows:

[0200]

[0201] Where s is the pixel depth value; u is the x-coordinate of the pixel in the drone QR code marker image in the image coordinate system; v is the y-coordinate of the pixel in the drone QR code marker image in the image coordinate system; t1 is the first matrix element in the pose transformation matrix; x m Here are the x-coordinates of the corner points, including the x-coordinates of the first, second, third, and fourth corner points; y m The ordinates of the corner points are: z = z + ... m t1 represents the vertical coordinates of the corner points, including the vertical coordinates of the first, second, third, and fourth corner points; t2 represents the second element of the pose transformation matrix; t3 represents the third element of the pose transformation matrix; t4 represents the fourth element of the pose transformation matrix; t5 represents the fifth element of the pose transformation matrix; t6 represents the sixth element of the pose transformation matrix; t7 represents the seventh element of the pose transformation matrix; t8 represents the eighth element of the pose transformation matrix; t9 represents the ninth element of the pose transformation matrix; t 10 t is the tenth element in the pose transformation matrix; 11 t is the eleventh element in the pose transformation matrix; 12 This is the twelfth element in the pose transformation matrix.

[0202] The formula for relative pose calculation is as follows:

[0203]

[0204] Among them, R w t represents the drone camera rotation matrix in the relative pose of the drone camera; w d is the translation vector of the drone camera in the relative pose of the drone camera; x d represents the horizontal distance of the QR code. y R represents the vertical distance of the QR code. m t is the pose rotation matrix in the pose transformation matrix; m is the pose translation vector in the pose transformation matrix.

[0205] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, modules, sub-modules, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0206] This invention also provides an electronic device, including a memory and a processor, wherein the memory stores a computer program; when the computer program is executed by the processor, the processor performs the steps of the UAV relative pose positioning method as described in any of the above embodiments.

[0207] This invention also provides a computer-readable storage medium storing a computer program / instructions thereon, which, when executed by a processor, implements the steps of the UAV relative pose positioning method as described in any of the above embodiments.

[0208] This invention also provides a computer program product, including a computer program / instructions, which, when executed by a processor, implement the steps of the UAV relative pose positioning method as described in any of the above embodiments.

[0209] In the several embodiments provided in this application, it should be understood that the disclosed systems and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0210] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0211] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for relative pose positioning of a UAV, characterized in that, include: The drone QR code labeling board image is acquired, and QR code recognition is performed on the drone QR code labeling board image to output multiple QR code encoding information. The drone QR code labeling board image includes multiple large QR codes and multiple small QR codes. The relative pose of the drone camera corresponding to each QR code encoding information is calculated using a preset pose transformation calculation formula based on the QR code length and distance data corresponding to each QR code encoding information. The relative pose of the fused drone cameras is determined based on the random sampling consensus algorithm and the relative poses of each drone camera. Substitute the relative pose of the fused drone camera into the preset pose conversion formula to calculate the relative pose of the target drone center. The preset pose transformation calculation formula includes a pose transformation matrix formula group and a relative pose calculation formula; the QR code length and distance data includes the QR code length, the QR code horizontal distance, and the QR code vertical distance; the step of calculating the relative pose of the drone camera corresponding to each QR code encoding information based on the QR code length and distance data corresponding to each QR code encoding information using the preset pose transformation calculation formula includes: Based on the QR code length corresponding to each QR code encoding information, determine the corner coordinate data corresponding to each QR code encoding information; Based on the corner coordinate data and the pose transformation matrix formula set, determine the pose transformation matrix corresponding to each QR code encoding information; The relative pose calculation formula is used to calculate the relative pose of the drone camera corresponding to each QR code encoding information based on each pose transformation matrix, each horizontal distance of each QR code and each vertical distance of each QR code; Wherein, the QR code length is the side length of the QR code, the horizontal distance of the QR code is the horizontal distance from the center of the recognized QR code to the center of the QR code marking board, and the vertical distance of the QR code is the vertical distance from the center of the recognized QR code to the center of the QR code marking board. The fused drone camera relative pose includes a fused rotation matrix and a fused translation vector; the step of determining the fused drone camera relative pose based on the random sampling consensus algorithm and the relative poses of each drone camera includes: According to a preset sampling ratio, select multiple first intermediate drone camera relative poses from the relative poses of each drone camera. The mean value of the translation vector of the first drone camera in the relative pose of each of the first intermediate drone cameras is calculated, and the mean value of the first translation vector is output. According to the preset sampling ratio, select multiple second intermediate drone camera relative poses from the relative poses of each drone camera; The mean value of the translation vector of the second drone camera in the relative pose of each of the second intermediate drone cameras is calculated, and the mean value of the second translation vector is output. Perform a difference calculation on the mean values ​​of the first translation vector and the second translation vector, output the mean error, and compare it with a preset error threshold. If the mean error is less than the preset error threshold, then the mean of the second translation vector is used as the fused translation vector. Perform Euler angle fusion on the rotation matrix of the second UAV camera in the relative pose of each of the second intermediate UAV cameras to determine the fused rotation matrix; The step of performing Euler angle fusion on the rotation matrices of the second UAV cameras in the relative poses of each of the second intermediate UAV cameras to determine the fused rotation matrix includes: The Euler angles corresponding to each rotation matrix of the second UAV camera are calculated using a preset Euler angle calculation formula based on the matrix elements in each rotation matrix of the second UAV camera. Calculate the mean of each Euler angle and output the mean Euler angle. The fusion rotation matrix is ​​determined based on the Euler angle mean and the preset fusion rotation matrix transformation formula set; The corner coordinate data includes the coordinates of a first corner point, a second corner point, a third corner point, and a fourth corner point. The first corner point coordinates include the x-coordinate, y-coordinate, and vertical coordinate of the first corner point; the second corner point coordinates include the x-coordinate, y-coordinate, and vertical coordinate of the second corner point; the third corner point coordinates include the x-coordinate, y-coordinate, and vertical coordinate of the third corner point; and the fourth corner point coordinates include the x-coordinate, y-coordinate, and vertical coordinate of the fourth corner point. The pose transformation matrix formula group is specifically as follows: ； Where s is the pixel depth value; u is the horizontal coordinate of the pixel in the drone QR code marking board image in the image coordinate system; v is the vertical coordinate of the pixel in the drone QR code marking board image in the image coordinate system; t1 is the first matrix element in the pose transformation matrix. The x-coordinates of the corner points include the x-coordinates of the first, second, third, and fourth corner points. The ordinates of the corner points are: the ordinates of the first corner point, the ordinates of the second corner point, the ordinates of the third corner point, and the ordinates of the fourth corner point. t1 represents the vertical coordinates of the corner points, including the vertical coordinates of the first, second, third, and fourth corner points; t2 represents the second element of the pose transformation matrix; t3 represents the third element of the pose transformation matrix; t4 represents the fourth element of the pose transformation matrix; t5 represents the fifth element of the pose transformation matrix; t6 represents the sixth element of the pose transformation matrix; t7 represents the seventh element of the pose transformation matrix; t8 represents the eighth element of the pose transformation matrix; t9 represents the ninth element of the pose transformation matrix; t 10 t is the tenth element in the pose transformation matrix; 11 t is the eleventh element in the pose transformation matrix; 12 This is the twelfth element in the pose transformation matrix; The relative pose calculation formula is as follows: ； in, Let be the drone camera rotation matrix in the relative pose of the drone camera; Let be the drone camera translation vector in the relative pose of the drone camera; The horizontal distance of the QR code; Vertical distance of the QR code; This is the pose rotation matrix in the pose transformation matrix; is the pose translation vector in the pose transformation matrix.

2. The UAV relative pose positioning method according to claim 1, characterized in that, The step of performing QR code recognition on the drone QR code marking board image and outputting multiple QR code encoding information includes: The UAV QR code marker image is preprocessed to generate a denoised QR code marker image; The denoised QR code label image is segmented using the maximum inter-class variance method to output a black and white QR code label image. A contour recognition algorithm is used to extract the graphic contours from the black and white QR code marking board image, and multiple graphic contours are output. A polygon approximation algorithm is used to filter the outlines of each graphic and output the QR code candidate boxes corresponding to each graphic outline. Perform perspective transformation on each of the QR code candidate boxes to generate a corrected QR code candidate box corresponding to each of the QR code candidate boxes; The candidate boxes for each correction QR code are divided into grids, and multiple pixel grids corresponding to each candidate box for correction QR code are output. A pre-set QR code dictionary is used to generate QR code encoding information corresponding to each of the correction QR code candidate boxes based on the number of pixels in each of the pixel grids.

3. A UAV relative pose positioning system, applied to the UAV relative pose positioning method of claim 1, characterized in that, include: The acquisition module is used to acquire images of drone QR code marking boards, perform QR code recognition on the drone QR code marking board images, and output multiple QR code encoding information; A module is used to calculate the relative pose of the drone camera corresponding to each QR code encoding information based on the QR code length and distance data corresponding to each QR code encoding information using a preset pose transformation calculation formula. According to the module, it is used to determine the fused relative pose of the drone cameras based on the random sampling consensus algorithm and the relative pose of each drone camera; The calculation module is used to substitute the relative pose of the fused UAV camera into a preset pose transformation formula to calculate the relative pose of the target UAV center.

4. An electronic device, characterized in that, The system includes a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor performs the steps of the UAV relative pose positioning method as described in any one of claims 1-2.

5. A computer-readable storage medium having a computer program / instructions stored thereon, characterized in that, When the computer program / instructions are executed by the processor, they implement the steps of the UAV relative pose positioning method as described in any one of claims 1-2.

6. A computer program product comprising a computer program / instructions, characterized in that, When the computer program / instructions are executed by the processor, they implement the steps of the UAV relative pose positioning method as described in any one of claims 1-2.

Images