A method and device for relative positioning of a UAV without GNSS, and a medium

Abstract

The application discloses a kind of unmanned plane relative positioning method without GNSS, device and medium, it is related to unmanned plane autonomous navigation and intelligent environment sensing technical field.The present application discards the traditional PnP pose estimation framework and the dependence on the physical size of mark, instead, the relative geometric center position of visual mark in image is extracted using deep learning, combined with the real height reference provided by millimeter wave radar and the real-time attitude information of unmanned plane, a scale adaptive, analytical, non-iterative relative position solution model is constructed, to realize the high-precision estimation of the three-dimensional relative position of unmanned plane relative to the center of landing mark.

Description

Technical Field

[0001] This invention relates to the field of autonomous navigation and intelligent environmental perception technology for unmanned aerial vehicles (UAVs), specifically to a method, device, and medium for relative positioning of UAVs without GNSS. Background Technology

[0002] The statements in this section are provided only as background information in connection with this disclosure and may not constitute prior art.

[0003] With the widespread application of rotary-wing drones in logistics delivery, power line inspection, emergency rescue, and unmanned platform operations, higher demands are being placed on their autonomous flight and precise landing capabilities in complex environments. Especially in environments where GNSS (Global Navigation Satellite System) signals are limited or completely denied (such as dense urban areas, tunnels, indoor spaces, and areas with strong electromagnetic interference), traditional technical approaches relying on GNSS-RTK or differential positioning are no longer viable, necessitating the development of autonomous relative navigation methods that do not rely on external infrastructure.

[0004] Visual-assisted positioning (VOL) has become the mainstream technology for achieving autonomous landing of unmanned aerial vehicles (UAVs) due to its low cost, rich information, and flexible deployment. Existing methods are mainly divided into two categories: one is visual / visual-inertial odometry (VO / VIO) based on natural features, and the other is relative pose estimation based on pre-set artificial visual markers (such as AprilTag, ArUco, concentric circles, crosshairs, etc.). Among these, methods based on pre-set markers are widely used in tasks such as automated airports and mobile platform landings due to their high recognition rate, strong anti-interference ability, and clear spatial reference.

[0005] A typical technical solution employs the Perspective-n-Point (PnP) algorithm, which solves for the six-DOF pose of the camera relative to the marker by matching the known 3D coordinates of the marker corner with its 2D projection in the image. However, this type of method has significant limitations: 1. High dependence on prior information of the markers: The physical size and corner layout of the markers must be accurately calibrated in advance, which limits the versatility of the system; 2. Sensitive to feature extraction: Corner detection is easily affected by image noise, lighting changes, and motion blur, which can lead to pose jumps or loss of lock. 3. Scale uncertainty: At long distances or with small markers, the image scale changes are slight, and PnP solution is highly sensitive to errors, especially in the height direction. 4. High computational complexity: The nonlinear optimization process is difficult to implement in real time on resource-constrained embedded flight control platforms.

[0006] Although some studies have attempted localization using monocular SLAM or visual odometry (VO) and visual inertial odometry (VIO) based on natural features, scale drift issues still exist, making it difficult to meet the reliability requirements of engineering applications.

[0007] More importantly, existing technologies generally fail to effectively integrate the prior geometric structure of visual markers with the high-precision ranging capabilities of millimeter-wave radar to construct a lightweight, analytical relative positioning mechanism that does not require prior knowledge of marker size and relies solely on image relative geometry and external height references. Therefore, developing a relative positioning method that is independent of marker physical parameters and possesses high precision, robustness, and real-time performance has become a key technological challenge for improving the adaptability and practicality of UAV autonomous landing systems. Summary of the Invention

[0008] The purpose of this invention is to address the problems existing in the prior art by providing a method, device, and medium for UAV relative positioning without GNSS. This method abandons the traditional PnP pose estimation framework and its dependence on the physical size of the marker, and instead utilizes deep learning to extract the relative geometric center position of the visual marker in the image. Combined with the real altitude reference provided by millimeter-wave radar and the real-time attitude information of the UAV, a scale-adaptive, analytical, and non-iterative relative position calculation model is constructed to achieve the three-dimensional relative position of the UAV relative to the center of the landing marker. High-precision estimation of ).

[0009] The technical solution of the present invention is as follows: A method for GNSS-free relative positioning of unmanned aerial vehicles (UAVs) includes: Step S1: System calibration; complete the calibration of the intrinsic parameter matrix of the downward-looking camera, distortion coefficients, and the installation relationship between the downward-looking camera, millimeter-wave radar, and attitude sensor; Step S2: Data Synchronization Acquisition; During the descent of the UAV, simultaneously acquire the downward view image, the distance measured by the millimeter-wave radar, and the real-time attitude angle of the UAV; Step S3: Centroid extraction; A deep learning target detection network is used to detect the visual landing markers in the downward view image and output their geometric center coordinates in the image coordinate system; Step S4: Attitude compensation; Based on the current real-time attitude angle of the UAV, perform projection correction on the detected geometric center coordinates to eliminate imaging offset caused by the tilt of the aircraft. Step S5: Analytical calculation of relative position; calculate relative height based on vertical distance measured by millimeter-wave radar. By combining the camera intrinsic parameter matrix and the centroid offset after image correction, the horizontal relative position is analytically calculated based on the perspective projection model. Step S6: Output and guidance; The calculated horizontal relative position is sent to the flight control system for autonomous landing control.

[0010] Further, step S1 includes: The intrinsic parameter matrix of the downward-looking camera is calculated based on the calibration board. With distortion coefficient Calibration; Using the UAV coordinate system as a reference, calibrate the mounting matrix on the camera coordinate system and the UAV coordinate system. ; Using the UAV coordinate system as a reference, calibrate the rotation matrix between the attitude sensor and the UAV coordinate system. ; Using the camera coordinate system as a reference, calibrate the translation matrix between the millimeter-wave radar and the downward-looking camera. .

[0011] Furthermore, the data obtained in step S2 includes: The real-time attitude angles of the drone include the roll angle. Pitch angle Yaw angle ; Distance measured by millimeter-wave radar ; The image below.

[0012] Furthermore, the real-time attitude angle of the UAV is obtained in the following way: Based on the roll angle output by the attitude sensor Pitch angle Heading angle And the rotation matrix from the UAV coordinate system to the attitude sensor system. Following the rotation order of ZYX, the attitude matrix of the UAV in the navigation coordinate system is obtained. ; According to the attitude matrix And the real-time attitude angles of the UAV were solved in reverse order according to ZYX. .

[0013] Further, step S3 includes: Based on the distortion model and the calibrated intrinsic parameter matrix With distortion coefficient Distortion correction is performed on the acquired bottom view image, assuming the original image pixel coordinates The corrected image pixel coordinates are ; A deep learning object detection network is used to identify and locate the landing markers in the distortion-corrected bottom view image, and output the bounding box or pixel-level mask of the markers. Based on the detection results of the deep learning object detection network, the geometric center is calculated, and the pixel coordinates of the geometric center in the pixel coordinate system are extracted. This serves as the visual observation benchmark for subsequent relative position calculations.

[0014] Further, step S4 includes: pixel coordinates of the geometric center in the pixel coordinate system Inverse compensation is performed to correct it to an equivalent projected position in the horizontal flight state of the UAV; the corrected coordinates after attitude compensation are: .

[0015] Further, step S5 includes: Step S51: Calculate the corrected geometric center Corresponding depth information ; Step S52: Solve the relative position based on the analytical expression of relative geometry.

[0016] Further, step S51 includes: Distance measured by millimeter-wave radar And the translation matrix between the calibrated millimeter-wave radar and the downward-looking camera. The vertical distance from the optical center of the downward-looking camera to the landing plane was calculated. Based on the reasonable assumption that the landing surface is nearly horizontal, and considering that the geometric center has been compensated for by projection onto the frontal plane, the depth corresponding to the corrected geometric center is consistent with the vertical distance corresponding to the optical center of the downward-looking camera. ; Step S52 includes: relative height ; Based on the perspective projection model and the principle of similar triangles, the relative position of the landing mark in the camera coordinate system is directly calculated analytically using the internal parameter matrix of the downward-looking camera and the corrected center offset. Based on the installation matrix in the camera coordinate system and the UAV coordinate system and yaw angle Obtain the coordinates of the UAV relative to the landing marker in the navigation coordinate system.

[0017] The present invention also proposes a GNSS-free relative positioning device for unmanned aerial vehicles (UAVs), comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the GNSS-free relative positioning method for UAVs as described above.

[0018] The present invention also proposes a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the above-described method for GNSS-free relative positioning of an unmanned aerial vehicle.

[0019] Compared with existing technologies, the advantages of this invention are: 1. No prior information required: It does not rely on the physical size, type or encoding information of visual identifiers, and can be adapted to a variety of common graphic identifiers, significantly improving the system's versatility and deployment flexibility.

[0020] 2. High precision and accurate scale guarantee: The millimeter-wave radar provides the true height, and the camera intrinsic parameters are combined to realize scale recovery, ensuring that the horizontal positioning has the true physical scale, and the positioning accuracy can reach the centimeter level.

[0021] 3. Strong robustness: Based on overall contour and center detection, it has a strong ability to adapt to interference such as local occlusion, lighting changes, and image blurring, avoiding positioning interruption caused by corner mismatch.

[0022] 4. Low computational complexity: It adopts analytical direct calculation, without the need for nonlinear optimization, and the algorithm is lightweight, enabling real-time operation at high frame rates (about 30Hz).

[0023] 5. Strong resistance to attitude disturbances: The attitude compensation mechanism effectively suppresses the impact of roll / pitch on horizontal positioning, improving the stability and safety of the landing process.

[0024] 6. High engineering practicality: The system has a simple structure and is easy to integrate into existing rotary-wing UAV platforms. It is suitable for various autonomous landing scenarios such as automatic airports, mobile vehicles, and ship decks. Attached Figure Description

[0025] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments recorded in the embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings.

[0026] Figure 1 A flowchart of a GNSS-free relative positioning method for unmanned aerial vehicles (UAVs); Figure 2 This is a schematic diagram of the camera coordinate system; Figure 3 This is a schematic diagram of the pixel coordinate system; Figure 4 This is a schematic diagram of the carrier coordinate system; Figure 5 The coordinate system is a geocentric coordinate system and a navigation coordinate system. Figure 6This is a diagram illustrating the drone and its landing markers. Figure 7 A schematic diagram showing the installation of each sensor. Detailed Implementation

[0027] It should be noted that relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0028] The features and performance of the present invention will be further described in detail below with reference to embodiments.

[0029] Example 1 This embodiment addresses the following problems existing in the prior art: 1. Strong dependence on sign size: Existing visual positioning methods generally require the precise physical size of the sign to be known. Once the size is unknown or the calibration error is large, it will lead to systematic deviation of the positioning results, which limits the general deployment capability of the system.

[0030] 2. High sensitivity to feature points: The PnP method based on corner matching is sensitive to image quality and is prone to failure in low light, blurry, occluded and other scenarios, and has poor robustness.

[0031] 3. Insufficient vertical accuracy: Pure vision methods have weak observability in the Z-axis direction and are prone to drift; even with IMU fusion, it is still difficult to meet the centimeter-level vertical accuracy requirements during the landing phase.

[0032] 4. Loose fusion of multi-source information: Vision, radar and attitude data are mostly processed in a loosely coupled manner, failing to build a unified geometric analytical model, resulting in low information utilization and limited fusion gain.

[0033] 5. Significant impact of attitude disturbance: When the UAV rolls or pitches, the projection of the marker in the image is distorted. If no effective compensation is made, it will introduce horizontal positioning error.

[0034] 6. High algorithm complexity: PnP or nonlinear optimization algorithms have high computational overhead.

[0035] This paper proposes a GNSS-free relative positioning method for unmanned aerial vehicles (UAVs). This method analyzes the relative geometric features of pre-defined ground markers collected by a downward-looking visual sensor, integrates precise vertical distance measurements from millimeter-wave radar with real-time attitude information output by the flight control system, and constructs a scale-adaptive analytical positioning model without requiring prior knowledge of the markers' physical size or type. This enables high-precision, low-latency 3D relative position estimation of the UAV relative to the landing point. This invention is applicable to GNSS-denied scenarios such as urban canyons, indoor spaces, and areas with electromagnetic interference, exhibiting high robustness, strong versatility, and good engineering deployability.

[0036] First, the spatial coordinate system involved in this embodiment will be described as follows.

[0037] (1) Camera coordinate system The camera coordinate system (c-frame for short) is a spatial coordinate system with the optical center (the center of the lens) as the origin Oc, the Xc axis (passing through the optical center and parallel to the image plane, along the direction of increasing width) as the axis, the Yc axis (passing through the optical center and parallel to the direction of increasing height) as the axis, and the Zc axis (passing through the optical center and extending outward along the camera's line of sight). According to the imaging model of a pinhole camera, any point P in the camera coordinate system can be represented as Pc(Xc, Yc, Zc), such as... Figure 2 As shown. In Figure 2 In the diagram, O'-X'Y'Z' is the image coordinate system, and o-xyz is the coordinate system established using the normalized plane.

[0038] (2) Pixel coordinate system In image processing, the pixel coordinate system refers to a system where the top-left corner of the image is the origin O, the direction of increasing image width is the positive u-axis, and the direction of increasing image height is the positive v-axis. The position of any point in space projected onto the pixel coordinate system as a point p can be represented as (u, v), with one unit of length representing one pixel. Figure 3 As shown.

[0039] (3) Carrier coordinate system (in this embodiment, it refers to the UAV coordinate system) The carrier coordinate system (abbreviated as b-system) has two forms of representation. One refers to a coordinate system with the carrier's center of mass as the origin Ob, the Xb-axis as the direction of the carrier's head passing through the center of mass and in the positive direction, the Yb-axis as the direction of the carrier's right wing passing through the center of mass and in the positive direction, and the Zb-axis as the direction of the carrier's right wing passing through the center of mass and in the positive direction conforming to the right-hand rule. This is simply referred to as the "front-right-lower" coordinate system. Figure 4 As shown in (a); another type is the "right front upper" coordinate system, with the right wing of the carrier passing through its center of mass and in the positive direction as the Xb axis, the direction of the carrier's nose passing through its center of mass and in the positive direction as the Yb axis, and the direction of the carrier passing through its center of mass and in the positive direction conforming to the right-hand rule as the Zb axis, vertically upward, as shown in (a). Figure 4As shown in (b). The angle of rotation about the coordinate axis in the direction of the carrier's head is called the roll angle, the angle of rotation about the axis in the direction of the right wing of the carrier is called the pitch angle, and the angle of rotation about the Zb axis of the carrier's coordinate system is called the yaw angle.

[0040] (4) Navigation coordinate system The Earth-centered, Earth-fixed coordinate system (e-system) is a spatial coordinate system with the Earth's center of mass as the origin, the Xe axis defined by the intersection of the equator and the prime meridian (passing through the Earth's center and positively aligned with the prime meridian), the Ze axis defined by the North Pole (passing through the Earth's center and positively aligned with the prime meridian), and the Ye axis determined by the right-hand rule. Figure 5 As shown. In the Earth-centered Earth-fixed coordinate system, the position of any point is represented by longitude λ and latitude φ. The origin of longitude is the Prime Meridian (0°), representing the angle from the Prime Meridian to the meridian where the observation point is located, ranging from -180° to +180°, increasing eastward to +180° and decreasing westward to -180°. The origin of latitude is the equator (0°), representing the angle from the equator to the latitude circle where the observation point is located, ranging from -90° to +90°, increasing towards the North Pole to +90° and decreasing towards the South Pole to -90°. The e-system is widely used in various navigation coordinate systems, such as satellite navigation coordinate systems and inertial navigation coordinate systems.

[0041] The navigation coordinate system (referred to as the n-system) of a UAV is established at the location of the observation point within the geocentric-geocentric coordinate system. It has two representations: one is the "North-East-Ground" coordinate system, which refers to establishing the Xn axis with the observation point's location as the origin On, the Yn axis with the geographic north point as the positive direction, and the Zn axis with the direction perpendicular to the Earth's center as the positive direction. For example... Figure 5 As shown in (a); another is the "Northeast-Sky" coordinate system, which establishes the Xn axis with the geographic east point as the positive direction, the Yn axis with the geographic north point as the positive direction, and the Zn axis with the vertical upward direction as the positive direction, as shown in (a). Figure 5 As shown in (b).

[0042] Secondly, the coordinate transformations between the various spatial coordinate systems involved in this embodiment are explained as follows.

[0043] (1) Normalized coordinate system → Image coordinate system

[0044] in, For pixel coordinates, For normalized planar coordinates, This is the intrinsic parameter matrix of the downward-looking camera; (2) Navigation coordinate system → Carrier coordinate system When the navigation coordinate system is established with the northeast coordinate system and the vehicle coordinate system (attitude sensor) is established with the lower right coordinate system, the coordinate transformation relationship is as follows:

[0045] in: The coordinates are in the carrier coordinate system; Coordinates in the navigation coordinate system; The attitude matrix is ​​represented as follows:

[0046] When the navigation coordinate system is established using the northeast-central coordinate system and the carrier coordinate system is established using the right-front-upper coordinate system, The coordinates are in the carrier coordinate system. These are the coordinates in the navigation coordinate system.

[0047] (3) Carrier coordinate system → Camera coordinate system

[0048] in: The coordinates of the camera are given below.

[0049] Please see Figure 1 The following is a detailed description of the UAV relative positioning method without GNSS proposed in this embodiment, which specifically includes the following steps: Step S1: System calibration; complete the calibration of the intrinsic parameter matrix of the downward-looking camera, distortion coefficients, and the installation relationship between the downward-looking camera, millimeter-wave radar, and attitude sensor; Step S2: Data Synchronization Acquisition; During the descent of the UAV, simultaneously acquire the downward view image, the distance measured by the millimeter-wave radar, and the real-time attitude angle of the UAV; Step S3: Centroid Extraction; A deep learning target detection network is used to detect visual landing markers (such as circles, crosses, squares, rings, etc.) in the bottom view image and output their geometric center (centroid) coordinates in the image coordinate system; Step S4: Attitude compensation; Based on the current real-time attitude angle of the UAV, perform projection correction on the detected geometric center coordinates to eliminate imaging offset caused by the tilt of the aircraft. Step S5: Analytical calculation of relative position; calculate relative height based on vertical distance measured by millimeter-wave radar. By combining the camera intrinsic parameter matrix and the centroid offset after image correction, the horizontal relative position is analytically calculated based on the perspective projection model. Step S6: Output and Guidance; Output the calculated horizontal relative position ( The signal is sent to the flight control system for autonomous landing control.

[0050] In this embodiment, it should be noted that the schematic diagram of the relative positions of the drone and the landing mark is as follows: Figure 6 As shown, the landing marker is located in the navigation coordinate system of northeast.

[0051] The installation methods of each sensor are as follows: Figure 7 As shown, the attitude sensor is mounted above the drone platform. Figure 7 As shown in (a), the downward-looking camera and millimeter-wave radar are mounted below the UAV platform. Figure 7 As shown in (b).

[0052] The intrinsic parameter matrix of the downward-looking camera is calculated based on the calibration board. With distortion coefficient Calibration; Using the UAV coordinate system as a reference, calibrate the mounting matrix on the camera coordinate system and the UAV coordinate system. ; Using the UAV coordinate system as a reference, calibrate the rotation matrix between the attitude sensor and the UAV coordinate system. ; Using the camera coordinate system as a reference, calibrate the translation matrix between the millimeter-wave radar and the downward-looking camera. ; in, , The radial distortion coefficient is... The tangential distortion coefficient; It is a 3×3 rotation matrix. It is a translation matrix.

[0053] In this embodiment, specifically, the data obtained in step S2 includes: The real-time attitude angles of the drone include the roll angle. Pitch angle Yaw angle ; Distance measured by millimeter-wave radar ; The image below.

[0054] In this embodiment, the real-time attitude angle of the UAV is obtained (i.e., the attitude of the UAV is calculated) in the following way: Based on the roll angle output by the attitude sensor Pitch angle Heading angle And the rotation matrix from the UAV coordinate system to the attitude sensor system. Following the rotation order of ZYX, the attitude matrix of the UAV in the navigation coordinate system is obtained. :

[0055] in: The rotation matrix from the navigation coordinate system to the attitude sensor's coordinate system is expressed as follows:

[0056] According to the attitude matrix And the real-time attitude angles of the UAV were solved in reverse order according to ZYX. .

[0057] In this embodiment, specifically, step S3 includes: Based on the distortion model and the calibrated intrinsic parameter matrix With distortion coefficient Distortion correction is performed on the acquired bottom view image, assuming the original image pixel coordinates The corrected image pixel coordinates are ; A deep learning object detection network is used to identify and locate the landing markers in the distortion-corrected bottom view image, and output the bounding box or pixel-level mask of the markers. Based on the detection results of the deep learning object detection network, the geometric center (centroid) is calculated, and the pixel coordinates of the geometric center in the pixel coordinate system are extracted. This serves as the visual observation benchmark for subsequent relative position calculations.

[0058] In this embodiment, it should be noted that, assuming the landing platform is flat or near-horizontal and the height of the UAV above the landing platform is accurately measured by millimeter-wave radar, the projected position of pixels in the image captured by the downward-looking camera is mainly affected by the current attitude angle of the UAV. When the UAV has a roll angle... With pitch angle When the camera's optical axis deviates from the vertical direction, the entire image undergoes a systematic translation, meaning that the projected positions of all ground targets are uniformly shifted.

[0059] like Figure 7 The installation method, and when the navigation coordinate system is established with the northeast coordinate system, and the UAV coordinate system is established with the lower right coordinate system, this offset is expressed as along the image coordinate system. Axis (column direction) and The equivalent pixel displacement along the axis (row direction), its theoretical offset can be expressed as:

[0060] in, This refers to the camera's internal parameters.

[0061] To eliminate this effect, the pixel coordinates of the geometric center in the pixel coordinate system need to be adjusted. Inverse compensation is performed to correct it to an equivalent projected position in the horizontal flight state of the UAV; the corrected coordinates after attitude compensation are: :

[0062] In this embodiment, specifically, step S5 includes: Step S51: Calculate the corrected geometric center Corresponding depth information ; Step S52: Solve the relative position based on the analytical expression of relative geometry.

[0063] In this embodiment, specifically, step S51 includes: Distance measured by millimeter-wave radar And the translation matrix between the calibrated millimeter-wave radar and the downward-looking camera. The vertical distance from the optical center of the downward-looking camera to the landing plane was calculated. :

[0064] Based on the reasonable assumption that the landing surface is nearly horizontal, and considering that the geometric center has been compensated for by projection onto the frontal plane, the depth corresponding to the corrected geometric center is consistent with the vertical distance corresponding to the optical center of the downward-looking camera. .

[0065] In this embodiment, specifically, step S52 includes: relative height ; Based on the perspective projection model and the principle of similar triangles, the relative position of the landing mark in the camera coordinate system is directly calculated analytically using the intrinsic parameter matrix of the downward-looking camera and the corrected center offset.

[0066] in: It is a three-dimensional column vector; Based on the installation matrix in the camera coordinate system and the UAV coordinate system and yaw angle Obtain the coordinates of the UAV relative to the landing marker in the navigation coordinate system (northeast):

[0067] This embodiment also proposes a GNSS-free relative positioning device for unmanned aerial vehicles (UAVs), comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the steps of the aforementioned GNSS-free relative positioning method for UAVs. Preferably, the computer program can be run on a terminal device, such as a personal computer.

[0068] This embodiment also proposes a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the above-described UAV GNSS-free relative positioning method. However, the computer-readable storage medium of the present invention is not limited thereto. In this document, the readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0069] A readable medium can be a readable signal medium or a readable storage medium. A readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0070] The computer-readable storage medium may include data signals propagated in baseband or as part of a carrier wave, carrying readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. The readable storage medium may also be any readable medium other than a readable storage medium, capable of transmitting, propagating, or transmitting programs for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the readable storage medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, RF, etc., or any suitable combination thereof.

[0071] Program code for performing the operations of this invention can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java and C++, and conventional procedural programming languages ​​such as C or similar languages. The program code can execute entirely on the user's computing device, partially on the user's device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0072] In this embodiment, it should be noted that the core innovation of the UAV GNSS-free relative positioning method proposed in this invention lies in: 1. The concept of "relative geometric modeling" is proposed: instead of relying on the absolute size of the marker, the relative geometric properties of its center offset in the image coordinate system are used as the basis for positioning. 2. Introduce radar altitude as a scale anchor point: Use the distance between the UAV and the ground measured by millimeter-wave radar as the depth input in the perspective projection model to establish the mapping relationship between pixel coordinates and real space offset in reverse, so as to achieve scale recovery without prior size. 3. Attitude compensation enhances robustness: combining the roll angle output by the IMU or flight controller ( Pitch angle ( ) and yaw angle ( The image undergoes inverse attitude transformation to correct projection distortion caused by the drone's tilt, thereby improving horizontal positioning accuracy. 4. Direct analytical calculation: The position is calculated using a closed-form formula based on the principle of similar triangles within the camera, avoiding complex optimization and ensuring real-time performance.

[0073] In summary, this invention employs a target detection network from deep learning to achieve robust recognition and centroid localization of various types of visual landing markers (such as circles, crosses, squares, and rings). The network model directly outputs the geometric center (centroid) of the marker in the image coordinate system, without relying on prior knowledge of the marker's physical dimensions or type, or manually calibrated parameters, effectively overcoming the strong dependence of traditional methods on marker design specifications. This method significantly enhances the system's adaptability to different marker styles and improves its versatility and engineering deployment flexibility in complex environments.

[0074] The embodiments described above merely illustrate specific implementation methods of this application, and while the descriptions are detailed and specific, they should not be construed as limiting the scope of protection of this application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the technical solution of this application, and these modifications and improvements all fall within the scope of protection of this application.

[0075] This background section is provided to generally present the context of the invention. The work of the currently named inventors, the work to the extent described in this background section, and aspects of this section that did not constitute prior art at the time of application are neither expressly nor impliedly acknowledged as prior art to the invention.

Claims

1. A method for relative positioning of unmanned aerial vehicles (UAVs) without GNSS, characterized in that, include: Step S1: System calibration; Complete the calibration of the intrinsic parameter matrix of the downward-looking camera, distortion coefficients, and the installation relationship between the downward-looking camera, millimeter-wave radar, and attitude sensor; Step S2: Data Synchronization Acquisition; During the descent of the UAV, simultaneously acquire the downward view image, the distance measured by the millimeter-wave radar, and the real-time attitude angle of the UAV; Step S3: Centroid extraction; A deep learning target detection network is used to detect the visual landing markers in the downward view image and output their geometric center coordinates in the image coordinate system; Step S4: Attitude compensation; Based on the current real-time attitude angle of the UAV, perform projection correction on the detected geometric center coordinates to eliminate imaging offset caused by the tilt of the aircraft. Step S5: Analytical calculation of relative position; calculate relative height based on vertical distance measured by millimeter-wave radar. By combining the camera intrinsic parameter matrix and the centroid offset after image correction, the horizontal relative position is analytically calculated based on the perspective projection model. Step S6: Output and guidance; The calculated horizontal relative position is sent to the flight control system for autonomous landing control.

2. The method for GNSS-free relative positioning of an unmanned aerial vehicle (UAV) according to claim 1, characterized in that, Step S1 includes: The intrinsic parameter matrix of the downward-looking camera is calculated based on the calibration board. With distortion coefficient Calibration; Using the UAV coordinate system as a reference, calibrate the mounting matrix on the camera coordinate system and the UAV coordinate system. ; Using the UAV coordinate system as a reference, calibrate the rotation matrix between the attitude sensor and the UAV coordinate system. ; Using the camera coordinate system as a reference, calibrate the translation matrix between the millimeter-wave radar and the downward-looking camera. .

3. The method for GNSS-free relative positioning of an unmanned aerial vehicle (UAV) according to claim 2, characterized in that, The data obtained in step S2 includes: The real-time attitude angles of the drone include the roll angle. Pitch angle Yaw angle ; Distance measured by millimeter-wave radar ; The image below.

4. The UAV GNSS-free relative positioning method according to claim 3, characterized in that, The real-time attitude angle of the UAV is obtained in the following way: Based on the roll angle output by the attitude sensor Pitch angle Heading angle And the rotation matrix from the UAV coordinate system to the attitude sensor system. Following the rotation order of ZYX, the attitude matrix of the UAV in the navigation coordinate system is obtained. ; According to the attitude matrix And the real-time attitude angles of the UAV were solved in reverse order according to ZYX. .

5. A method for relative positioning of a UAV without GNSS according to claim 4, characterized in that, Step S3 includes: Based on the distortion model and the calibrated intrinsic parameter matrix With distortion coefficient Distortion correction is performed on the acquired bottom view image, assuming the original image pixel coordinates The corrected image pixel coordinates are ; A deep learning object detection network is used to identify and locate the landing markers in the distortion-corrected bottom view image, and output the bounding box or pixel-level mask of the markers. Based on the detection results of the deep learning object detection network, the geometric center is calculated, and the pixel coordinates of the geometric center in the pixel coordinate system are extracted. This serves as the visual observation benchmark for subsequent relative position calculations.

6. A method for relative positioning of a UAV without GNSS according to claim 5, characterized in that, Step S4 includes: pixel coordinates of the geometric center in the pixel coordinate system Inverse compensation is performed to correct it to an equivalent projected position in the horizontal flight state of the UAV; the corrected coordinates after attitude compensation are: .

7. A method for GNSS-free relative positioning of an unmanned aerial vehicle (UAV) according to claim 6, characterized in that, Step S5 includes: Step S51: Calculate the corrected geometric center Corresponding depth information ; Step S52: Solve the relative position based on the analytical expression of relative geometry.

8. A method for relative positioning of a UAV without GNSS according to claim 7, characterized in that, Step S51 includes: Distance measured by millimeter-wave radar And the translation matrix between the calibrated millimeter-wave radar and the downward-looking camera. The vertical distance from the optical center of the downward-looking camera to the landing plane was calculated. Based on the reasonable assumption that the landing surface is nearly horizontal, and considering that the geometric center has been compensated for by projection onto the frontal plane, the depth corresponding to the corrected geometric center is consistent with the vertical distance corresponding to the optical center of the downward-looking camera. ; Step S52 includes: relative height ; Based on the perspective projection model and the principle of similar triangles, the relative position of the landing mark in the camera coordinate system is directly calculated analytically using the internal parameter matrix of the downward-looking camera and the corrected center offset. Based on the installation matrix in the camera coordinate system and the UAV coordinate system and yaw angle Obtain the coordinates of the UAV relative to the landing marker in the navigation coordinate system.

9. A GNSS-free relative positioning device for unmanned aerial vehicles (UAVs), comprising: The memory, the processor, and the computer program stored in the memory and executable on the processor are characterized in that, when the processor executes the computer program, it implements the steps of a UAV GNSS-free relative positioning method as described in any one of claims 1-8.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of a GNSS-free relative positioning method for unmanned aerial vehicles as described in any one of claims 1-8.

Images