Drone landing system, drone landing control method, and drone

Abstract

An embodiment of the present application provides a drone landing system, a landing control method for a drone, and a drone. The system includes: a drone, a hangar monocular vision sensor, and a hangar radar. The drone includes a drone monocular vision sensor and a processor, and a drone state machine runs on the processor. The hangar monocular vision sensor and the hangar radar are installed on the hangar; a hangar QR code is set on the hangar, and a landing point is set on the hangar QR code; the drone monocular vision sensor is used to determine a first pose of the drone monocular vision sensor relative to the landing point; the hangar monocular vision sensor is used to determine a second pose of a marker on the drone relative to the landing point; the hangar radar is used to determine the position of the drone; the drone state machine is used to generate a control instruction according to the first pose, the drone position, and the second pose, and control the drone to land at the landing point according to the control instruction. Through the present application, the problem that the drone cannot land accurately is solved.

Description

Technical Field

[0001] The embodiments of the present application relate to the technical field of unmanned aerial vehicles, and in particular, to an unmanned aerial vehicle landing system, a landing control method for an unmanned aerial vehicle, and an unmanned aerial vehicle. Background Art

[0002] Currently, the precise landing of an unmanned aerial vehicle on an unmanned aerial vehicle hangar is usually achieved by the following method: detecting the two-dimensional code on the hangar based on the monocular vision sensor of the unmanned aerial vehicle to achieve precise positioning and guiding the unmanned aerial vehicle to descend. However, the lower vision system of the unmanned aerial vehicle is greatly affected by environmental light, and there is a situation where it cannot be precisely recognized in environments such as fog and heavy rain. Summary of the Invention

[0003] The embodiments of the present application provide an unmanned aerial vehicle landing system, a landing control method for an unmanned aerial vehicle, and an unmanned aerial vehicle, so as to at least solve the technical problem that the unmanned aerial vehicle cannot land precisely in the related art.

[0004] According to one aspect of the embodiments of the present application, an unmanned aerial vehicle landing system is provided, including: an unmanned aerial vehicle, a hangar monocular vision sensor, and a hangar radar. The unmanned aerial vehicle includes an unmanned aerial vehicle monocular vision sensor and a processor, and an unmanned aerial vehicle state machine runs on the processor. The hangar monocular vision sensor and the hangar radar are installed on the hangar. Wherein, the unmanned aerial vehicle monocular vision sensor is configured to, when the unmanned aerial vehicle descends to a preset height range, capture a hangar two-dimensional code arranged on the hangar to obtain a first monocular image, wherein a landing point is arranged on the hangar two-dimensional code, and the landing point refers to the landing position of the unmanned aerial vehicle on the hangar; determine a first pose of the unmanned aerial vehicle monocular vision sensor relative to the landing point according to the first monocular image, and transmit the first pose to the unmanned aerial vehicle state machine; the hangar monocular vision sensor is configured to capture an object within a preset viewing angle to obtain a second monocular image, wherein the second monocular image includes the hangar two-dimensional code; determine a second pose of a marker on the unmanned aerial vehicle relative to the landing point according to the second monocular image, and transmit the second pose to the unmanned aerial vehicle state machine; the hangar radar is configured to scan an object within a preset scattering cross-section multiple times to obtain multiple sets of object point cloud data; track the unmanned aerial vehicle based on the multiple sets of object point cloud data to obtain the position of the unmanned aerial vehicle; transmit the position of the unmanned aerial vehicle to the unmanned aerial vehicle state machine; the unmanned aerial vehicle state machine is configured to generate a control instruction according to the first pose, the position of the unmanned aerial vehicle, and the second pose, and control the unmanned aerial vehicle to land at the landing point according to the control instruction.

[0005] According to another aspect of the embodiments of the present application, there is also provided a landing control method for a drone, including: when the drone descends to a preset height range, capturing a hangar QR code arranged on a hangar by a monocular vision sensor on the drone to obtain a first monocular image, wherein the hangar QR code is provided with information of a landing point, and the landing point refers to the landing position of the drone on the hangar; determining a first pose of the monocular vision sensor on the drone relative to the landing point according to the first monocular image, and transmitting the first pose to a drone state machine; obtaining a second pose transmitted by a monocular vision sensor on the hangar, wherein the second pose is a pose of a marker on the drone relative to the landing point determined according to a second monocular image obtained by the monocular vision sensor on the hangar capturing an object within a preset viewing angle; obtaining the position of the drone transmitted by a hangar radar, wherein the position of the drone is obtained by tracking the drone based on object point cloud data obtained by the hangar radar scanning an object within a preset scattering cross section; generating a control instruction by the drone state machine according to the first pose, the position of the drone, and the second pose, and controlling the drone to land at the landing point according to the control instruction.

[0006] According to still another aspect of the embodiments of the present application, there is also provided a drone, including: a monocular vision sensor on the drone and a processor, with a drone state machine running on the processor, wherein the monocular vision sensor on the drone is configured to, when the drone descends to a preset height range, capture a hangar QR code arranged on a hangar to obtain a first monocular image, wherein the hangar QR code is provided with information of a landing point, and the landing point refers to the landing position of the drone on the hangar; determining a first pose of the monocular vision sensor on the drone relative to the landing point according to the first monocular image, and transmitting the first pose to the drone state machine; the drone state machine is configured to obtain a second pose transmitted by a monocular vision sensor on the hangar, wherein the second pose is a pose of a marker on the drone relative to the landing point determined according to a second monocular image obtained by the monocular vision sensor on the hangar capturing an object within a preset viewing angle; obtaining the position of the drone transmitted by a hangar radar, wherein the position of the drone is obtained by tracking the drone based on object point cloud data obtained by the hangar radar scanning an object within a preset scattering cross section; generating a control instruction according to the first pose, the position of the drone, and the second pose, and controlling the drone to land at the landing point according to the control instruction.

[0007] According to another aspect of the embodiments of the present application, there is also provided a computer-readable storage medium storing a computer program, wherein the computer program is configured to execute the steps in any one of the above method embodiments when running.

[0008] According to another aspect of the embodiments of the present application, there is provided a computer program product or a computer program, which includes computer instructions stored in a computer-readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the steps in any one of the above method embodiments.

[0009] According to another aspect of the embodiments of the present application, there is also provided an electronic device including a memory and a processor, wherein a computer program is stored in the memory, and the processor is configured to execute the steps in any one of the above method embodiments through the computer program.

[0010] Through the present application, the hangar monocular vision sensor satisfies image acquisition within a preset viewing angle. By identifying the UAV markers, it calculates the second pose of the markers relative to the landing point. The selection of the markers improves the accuracy and reliability of image recognition. Especially under low-light or strong reflection conditions, the hangar monocular vision can be used as a supplement to provide additional visual positioning information and enhance the adaptability of the system. Further, by scanning the preset scattering cross-section multiple times, collecting the UAV echo signals, and generating multiple sets of object point cloud data, the tracking and positioning of the UAV are realized. During the descent of the UAV, when the UAV monocular vision sensor or the hangar monocular vision sensor cannot accurately detect the hangar QR code due to environmental factors, the hangar radar can provide the stable position information of the UAV to ensure the continuity and robustness of the landing process. The UAV state machine module comprehensively analyzes the first pose, the UAV position, and the second pose information to generate control instructions to guide the UAV to accurately land at the hangar landing point. The state machine fuses multi-source sensor data for decision-making. When the information of a certain sensor is missing or unreliable, it can automatically switch to the information of other sensors to ensure the precise control and safe landing of the UAV in a complex environment. In summary, through the collaborative work of multiple sensors on the UAV and the hangar side, combined with the intelligent decision-making of the UAV state machine module, the limitations of a single sensor in a specific environment are effectively overcome. During the landing of the UAV, even in the face of light changes or bad weather, the complementary advantages of different sensors can be used to provide continuous and accurate position information, generate appropriate control instructions, and finally achieve the precise and stable landing of the UAV, significantly improving the operation ability and safety of the UAV in a complex environment. Description of the Drawings

[0011] Figure 1Schematic diagram of an application scenario of a drone landing system according to an embodiment of the present application;

[0012] Figure 2 Schematic diagram of an optional hangar QR code according to an embodiment of the present application;

[0013] Figure 3 Schematic diagram of an optional coordinate system of a hangar QR code according to an embodiment of the present application;

[0014] Figure 4 Schematic diagram of a processing flow of an optional drone landing system according to an embodiment of the present application;

[0015] Figure 5 Flowchart of an optional method for determining the false inspection probability according to an embodiment of the present application;

[0016] Figure 6 Flowchart of an optional method for controlling the landing of a drone according to an embodiment of the present application;

[0017] Figure 7 Block diagram of a computer system of an optional electronic device according to an embodiment of the present application. Detailed implementation manners

[0018] In order to enable those skilled in the art to better understand the solutions of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.

[0019] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and do not necessarily need to describe a specific order or sequence. It should be understood that such data can be interchanged under appropriate circumstances so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "comprising" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device that includes a series of steps or units does not necessarily need to be limited to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0020] According to one aspect of the embodiments of the present application, a drone landing system is provided. Optionally, in this embodiment, the above-mentioned drone landing system can be but is not limited to being applied to, for exampleFigure 1 In the hardware environment shown, which includes a drone 102, a hangar monocular vision sensor 104, and a hangar radar 106. Among them, the drone 102 includes a drone monocular vision sensor 1021 and a processor 1022. A drone state machine runs on the processor 1022. The hangar monocular vision sensor 104 and the hangar radar 106 are installed on the hangar 108. A hangar QR code 110 is also set on the hangar 108. A landing point is set on the hangar QR code 110. The landing point refers to the landing position of the drone 102 on the hangar 108.

[0021] The drone monocular vision sensor is used to capture the hangar QR code set on the hangar when the drone descends to a preset height range, obtaining a first monocular image; according to the first monocular image, determine the first pose of the drone monocular vision sensor relative to the landing point, and transmit the first pose to the drone state machine. Among them, the drone monocular vision sensor is installed on the drone at a specified angle (such as a horizontal FOV of 80 degrees and a pitch FOV of 70 degrees), and is used to capture the hangar QR code below the drone at a specified frame rate (such as 25Hz). The hangar QR code refers to the QR code set on the hangar for assisting the drone in positioning. The hangar QR code is used for assisting in positioning and attitude estimation. A landing point is set on the hangar QR code. The landing point refers to the landing position of the drone on the hangar, and this landing position is also the starting position for the drone's next takeoff; the landing point can be set at any position on the hangar QR code, and the specific position of the landing point on the hangar QR code is not limited here. The preset height range refers to the height range within which the drone can clearly capture the hangar QR code. The first monocular image obtained by the drone monocular vision sensor when capturing the area below the drone when the drone descends to the preset height range includes various objects in the area below the drone, such as houses, trees, roads, and the hangar QR code below the drone. The drone monocular vision sensor can effectively identify the hangar QR code in the first monocular image through recognition, and based on the recognized hangar QR code, determine the position and attitude of the drone monocular vision sensor relative to the landing point, that is, the first pose, and transmit the first pose to the drone state machine. The drone state machine runs on the processor of the drone as a software decision-making layer. Its function is to fuse and process various sensor information, analyze sensor data and make control decisions, and guide the adjustment of the drone's flight attitude to achieve precise and safe landing.

[0022] The hangar monocular vision sensor is used to capture an object within a preset viewing angle to obtain a second monocular image, where the second monocular image includes a hangar QR code; based on the second monocular image, determine the second pose of the marker on the drone relative to the landing point, and transmit the second pose to the drone state machine. In the current drone landing technology, in adverse weather conditions such as low light, strong reflection, heavy fog, and heavy rain, the accuracy and robustness of QR code recognition will significantly decrease based on the drone monocular vision sensor landing method, which directly affects the autonomous landing ability and safety of the drone. Therefore, in the embodiments of this application, through the collaborative action of the enhanced vision and radar system at the hangar end, the precise landing performance of the drone in various environments is improved. Among them, the hangar monocular vision sensor is set on the hangar so that the hangar monocular vision sensor can capture an object within a preset viewing angle at a certain frame rate (such as 25Hz), and the hangar monocular vision sensor can always capture the complete hangar QR code within the preset viewing angle, and can capture the clear contour of the drone. The preset viewing angle is a viewing angle range determined according to the horizontal viewing angle and the pitch angle. For example, the preset viewing angle can be a viewing angle range with a horizontal FOV (field of view) of 100 degrees and a pitch FOV (field of view) of 100 degrees. The second monocular image captured by the hangar monocular vision sensor within the preset viewing angle contains hangar environment information, especially the image data of the hangar QR code. To further assist in positioning and pose estimation, the second monocular image also includes specific markers on the drone. The relative position between the hangar QR code and the drone landing point has been preset in advance. By identifying the positional relationship between the hangar QR code and the marker on the drone, the pose change of the marker on the drone relative to the landing point, that is, the second pose, can be calculated. The marker is a physical reference for the drone state machine to calculate the precise position and pose of the drone relative to the landing point. For example, the drone propeller, the front of the drone (including binoculars, Tof, front logo), the back of the drone (drone battery, battery knob), etc. In the embodiments of this application, the marker recognition technology is combined with the hangar QR code detection to provide more comprehensive position information for the drone state machine to achieve highly robust precise landing. In summary, the hangar monocular vision sensor provides key visual positioning information for the drone state machine by capturing the second monocular image, which contains the hangar QR code and the drone marker, to assist the drone in achieving accurate landing attitude control in complex environments.

[0023] The hangar radar is used to scan objects within a preset radar cross section multiple times to obtain multiple sets of object point cloud data; track the UAV based on the multiple sets of object point cloud data to obtain the UAV position; and transmit the UAV position to the UAV state machine. To solve the problem that the accuracy and robustness of identifying QR codes are poor for the landing method of the UAV's monocular vision sensor under harsh weather conditions such as low light, strong reflection, heavy fog, and heavy rain, the embodiment of this application combines an integrated system of millimeter-wave radar and multiple vision sensors to improve the precise landing performance of the UAV in various environments. Among them, the hangar radar is a millimeter-wave radar device installed at the hangar end. When the UAV approaches landing, it continuously scans a preset radar cross section area to collect multiple sets of object point cloud data of the UAV, thereby realizing the positioning and tracking of the UAV under various environmental conditions. The hangar radar reduces the influence of adverse factors such as light and weather on the landing accuracy of the UAV through its strong environmental adaptability. The preset radar cross section refers to a virtual cross section area that is specially set with a specific size for capturing the radar scattering signal of the UAV when the hangar radar performs scanning. The object point cloud data refers to a set of three-dimensional point data collections containing information such as the target position, speed, and radar cross section collected after the hangar radar scans the target within the preset radar cross section multiple times. The UAV position refers to the real-time coordinate information of the UAV in space determined by the hangar radar through processing the object point cloud data using target tracking algorithms (such as the density-based clustering algorithm DBSCAN and Kalman filter), including the X, Y, and Z coordinates of the UAV relative to the hangar radar.

[0024] The UAV state machine is used to generate control commands based on the first pose, the UAV position, and the second pose, and control the UAV to land at the landing point according to the control commands. Among them, the UAV state machine is the core module of the precise landing digital control system in the UAV's processor. It comprehensively processes the first pose obtained by the UAV through the recognition of the hangar QR code by the monocular vision sensor, the UAV position tracked by the millimeter-wave radar at the hangar end, and the second pose obtained by the hangar monocular vision sensor through the recognition of the UAV marker. The UAV state machine uses this information, combines the on-board IMU (inertial measurement unit) data and the pre-set landing point coordinates, and generates control commands for adjusting the flight attitude and speed of the UAV through state estimation and control algorithms such as Kalman filter and the knob-to-landing-point pose calculation formula, ensuring that the UAV can land safely and accurately within the hangar QR code even under harsh weather conditions.

[0025] Optionally, when the drone descends to a preset altitude range (e.g., below 100 meters), the drone's monocular vision sensor starts working, takes a picture of the hangar QR code on the hangar, obtains the first monocular image, uses an open-source QR code recognition algorithm such as Apriltag or ArUco to identify and locate the hangar QR code, calculates the first pose of the drone's monocular vision sensor relative to the landing point, and transmits the calculated first pose to the drone state machine for subsequent landing control decisions. The hangar monocular vision sensor continuously monitors the area within the preset viewing angle, obtains the second monocular image containing the hangar QR code, applies a deep learning model to identify specific markers (such as battery knobs) on the drone, and extracts the corner coordinates based on the analysis of the marker contour. The PnP (Perspective-n-Point) algorithm is used to solve the corner coordinates of the marker, obtaining the second pose of the drone marker relative to the hangar monocular vision sensor, and synchronizing the second pose of the drone marker to the drone state machine for decision-making. The hangar radar continuously scans within the preset cross-sectional area, collects the drone echo signal, generates multiple sets of object point cloud data, uses a density-based clustering algorithm to cluster the object point cloud data, identifies and tracks the drone, analyzes the speed and direction of the drone through the Doppler effect, and estimates the drone's position; the drone position data is transmitted to the drone state machine in real time to provide radar detection information for precise landing. The drone state machine receives the first pose from the drone's monocular vision sensor, the drone position from the hangar radar, and the second pose from the hangar monocular vision sensor, analyzes the validity of the received information and environmental conditions (such as lighting, weather), determines the fusion strategy, preferentially uses the data source with higher accuracy, and based on the selected pose data, combines the drone IMU data and the pre-set landing point coordinates to generate control commands (such as speed and angular velocity), and executes the control commands through the drone's flight control system to adjust the drone's attitude and speed to precisely align with the landing point; during the landing process, continuously monitor the sensor data and adjust the control strategy as needed to ensure the safe landing of the drone.

[0026] Through this embodiment, the hangar monocular vision sensor satisfies image acquisition within a preset viewing angle. By identifying the UAV markers, the second pose of the markers relative to the landing point is calculated. The selection of markers improves the accuracy and reliability of image recognition. Especially under low-light or strong-reflection conditions, the hangar monocular vision can be used as a supplement to provide additional visual positioning information and enhance the adaptability of the system. Further, by scanning the preset scattering cross-section multiple times, the UAV echo signals are collected to generate multiple sets of object point cloud data, realizing the tracking and positioning of the UAV. During the descent of the UAV, when the UAV monocular vision sensor or the hangar monocular vision sensor cannot accurately detect the hangar QR code due to environmental factors, the hangar radar can provide the stable position information of the UAV to ensure the continuity and robustness of the landing process. The UAV state machine module comprehensively analyzes the first pose, the UAV position, and the second pose information to generate control commands to guide the UAV to accurately land at the hangar landing point. The state machine fuses multi-source sensor data for decision-making. When the information of a certain sensor is missing or unreliable, it can automatically switch to the information of other sensors to ensure the precise control and safe landing of the UAV in a complex environment. In summary, through the collaborative work of multi-sensors at the UAV and hangar ends, combined with the intelligent decision-making of the UAV state machine module, the limitations of a single sensor in a specific environment are effectively overcome. During the landing of the UAV, even in the face of light changes or bad weather, the complementary advantages of different sensors can be used to provide continuous and accurate position information, generate appropriate control commands, and finally achieve the precise and stable landing of the UAV, significantly improving the operation ability and safety of the UAV in a complex environment.

[0027] In an exemplary embodiment, the hangar QR code includes multiple QR codes, and the sizes of different QR codes among the multiple QR codes are different from each other. Among them, the size and relative position relationship of each QR code in the hangar QR code are jointly determined by the allowable size of the hangar, the maximum distance of the UAV's lower vision to detect the hangar QR code, and the minimum allowable number of pixels of the QR code in the lower vision image. The ideal calculation formula is shown in formula (1):

[0028] (1)

[0029] Wherein, represents the number of pixels of the QR code in the image, represents the true physical size of the QR code (square), represents the focal length, z represents the depth of the QR code from the UAV monocular vision sensor, and pd represents the physical size of each pixel. Example: Focal length : 2mm, z: 25m, : 7, pd: 1.2um, and is obtained as 0.18m.

[0030] The sizes of different two-dimensional codes are different to meet different measurement distances. For example, Figure 2 is a schematic diagram of an optional hangar two-dimensional code according to an embodiment of the present application. As Figure 2 shown, the hangar two-dimensional code includes five two-dimensional codes A, B, C, D, and E. The sizes of these five two-dimensional codes are all different. Denote the height of the two-dimensional code as H and the width as W. For example, H1 and W1 of code A can be set to 0.2 m, H3 and W3 of code B can be set to 0.18 m, H2 and W2 of code C can be set to 0.15 m, H4 and W4 of code D can be set to 0.7 m, and H5 and W5 of code E can be set to 0.08 m.

[0031] Figure 3 is a schematic diagram of an optional hangar two-dimensional code coordinate system according to an embodiment of the present application. The position of the landing point and the set hangar two-dimensional code coordinate system are shown in Figure 3 , where the coordinate system conforms to the right-hand rule, the origin of the coordinate system is located at the center of the smallest two-dimensional code E, and the position of the landing point relative to the center of the coordinate system is determined by the geometric dimensions of the hangar. For example, the position of the landing point is (-0.08, -0.01, 0).

[0032] In some embodiments, the UAV monocular vision sensor is further configured to perform two-dimensional code recognition on the first monocular image, and determine a target two-dimensional code from a group of recognized two-dimensional codes. The target two-dimensional code refers to the two-dimensional code with the smallest Hamming distance in a group of two-dimensional codes; calculate the third pose of the UAV monocular vision sensor relative to the target two-dimensional code, and compensate the third pose according to the position difference between the landing point and the center point of the hangar two-dimensional code to obtain the first pose.

[0033] Among them, the target two-dimensional code refers to the two-dimensional code with the smallest Hamming distance in a group of two-dimensional codes recognized by the UAV monocular vision sensor, which means that the target two-dimensional code has the mode closest to the ideal state and is most suitable for positioning. The Hamming distance is an index for measuring the difference between two strings of the same length and is applicable to comparing the similarity of two-dimensional patterns. The target two-dimensional code is based on the recognition result of the UAV monocular vision sensor in the first monocular image, ensuring that the UAV state machine can accurately identify a specific two-dimensional code for positioning from a group of possible two-dimensional codes, thereby calculating the accurate relative position between the UAV and the hangar landing point.

[0034] Optionally, when the drone descends to a preset height range, the drone's monocular vision sensor is activated, outputting a first monocular image with a size of 780*720 pixels, a frame rate of 25Hz, a horizontal FOV of 80 degrees, and a pitch FOV of 70 degrees. The drone's monocular vision sensor uses the deep learning model YOLOv7 based on CNN (Convolutional Neural Network) to preprocess the first monocular image and detect objects, identifying the hangar QR code. If the detection confidence output by the deep learning model YOLOv7 exceeds 0.85, the drone's monocular vision sensor uses QR code detection technology (such as the Apriltag (tag recognition) detection method) within the detection box (Box) given by the deep network to detect each QR code and its corners in the hangar QR code, obtaining a set of recognized QR codes; otherwise, the entire image is recognized. After recognizing a set of QR codes, the drone's monocular vision sensor determines the QR code with the smallest Hamming distance among the recognized set of QR codes as the target QR code. For example, taking the hangar QR code shown in Figure 2 as an example, the target QR code is denoted as target QR code N, where N is one of the five QR codes A, B, C, D, and E of the hangar QR code. The positions of the four corners of the target QR code N in the hangar QR code coordinate system are respectively denoted as (-LN / 2, -LN / 2, 0), (LN / 2, -LN / 2, 0), (LN / 2, LN / 2, 0), (-LN / 2, LN / 2, 0), where LN is the size of the target QR code N. The drone's monocular vision sensor uses the Pnp algorithm (Perspective-n-Point) to solve the third pose from the drone's monocular vision sensor to the target QR code N. Then, the drone's monocular vision sensor compensates according to the position difference between the landing point and the center point of the hangar QR code, adjusts the third pose, and finally obtains the first pose (including the rotation matrix and the translation vector ) of the drone's monocular vision sensor relative to the landing point, providing accurate landing position information for the drone.

[0035] In some embodiments, during the training process of the deep learning model YOLOv7, images of the hangar shell where the meteorological column is located are selected as training samples, and the hangar QR codes on the hangar shell are used as training labels for the deep model. Through multiple training samples, the trained network parameters are obtained, and the network is quantized and transplanted onto the network processing unit to obtain the trained deep learning model YOLOv7.

[0036] Through this embodiment, when the monocular vision sensor of the drone identifies a group of two-dimensional codes, by calculating the Hamming distance between different two-dimensional codes, the two-dimensional code with the smallest Hamming distance is selected as the target two-dimensional code. The two-dimensional code with the smallest Hamming distance means that its information is closest to the two-dimensional code information preset in the hangar, which is the most reliable recognition result under the current environmental conditions. This innovative selection mechanism improves the recognition accuracy of two-dimensional codes in complex environments and ensures that the drone can make a landing decision based on the two-dimensional code closest to the real hangar information; based on the target two-dimensional code, the monocular vision sensor of the drone calculates its third pose relative to the target two-dimensional code, that is, the precise pose information including rotation and translation. Then, according to the actual position difference between the landing point and the center point of the hangar two-dimensional code, the third pose is compensated accordingly to obtain a more accurate first pose. This pose compensation mechanism takes into account the fixed offset between the two-dimensional code and the landing point. Even when there is a slight deviation in the position detection of the two-dimensional code, through compensation adjustment, it can ensure that the drone obtains accurate landing point pose information, thus achieving a more precise landing; through the above mechanism, even in harsh environmental conditions, the drone can use the relatively clearest target two-dimensional code for precise positioning and calculate the compensated precise pose information. The combination of the above intelligent selection and precise calculation significantly improves the landing robustness of the drone in harsh environments and reduces the risk of landing failure caused by external environmental impacts.

[0037] In an exemplary embodiment, the hangar two-dimensional code includes multiple two-dimensional codes; the sizes of different two-dimensional codes among the multiple two-dimensional codes are different from each other; the control instruction includes a first instruction; the first instruction is used to control the drone to land at the landing point.

[0038] In some embodiments, the realization of the precise landing of the drone at the hangar mainly depends on the recognition of the two-dimensional code on the hangar by the vision sensor. However, in harsh environments such as low light, strong reflection, heavy fog or heavy rain, the accuracy and stability of visual recognition are greatly challenged. In particular, when the two-dimensional code detected by the drone does not actually exist but is a false detection caused by factors such as environmental noise, light change or occlusion effect, this false detection may cause a deviation in the landing position of the drone and even lead to safety problems. To solve the above problems, this embodiment introduces the concept of the false detection probability of the target two-dimensional code. After the monocular vision sensor of the drone identifies a group of two-dimensional codes, by analyzing the relative position relationship between the target two-dimensional code and other known two-dimensional codes, the possibility of false detection of the target two-dimensional code, that is, the false detection probability, is evaluated. The introduction of the false detection probability enables the drone state machine to adjust the landing strategy according to the reliability of the current detection result, improving the landing robustness and accuracy under complex environmental conditions.

[0039] In this embodiment, the monocular vision sensor of the unmanned aerial vehicle (UAV) is further configured to determine the false detection probability of the target two-dimensional code (QR code) and transmit the false detection probability to the UAV state machine. The target QR code refers to the QR code with the smallest Hamming distance among a group of QR codes obtained by the UAV monocular vision sensor through QR code recognition of the first monocular image. The false detection probability characterizes the possibility of false detection of the target QR code. The UAV state machine is further configured to generate a first instruction according to the false detection probability and the first pose when the UAV monocular vision sensor detects the hangar QR code, and control the UAV to land at the landing point according to the first instruction.

[0040] Among them, the target QR code has been explained in the above embodiment and will not be elaborated here.

[0041] The false detection probability is for the target QR code detected by the UAV monocular vision sensor to evaluate the possibility that it actually does not exist but is misrecognized. The false detection probability is a quantitative index for judging whether the target QR code really exists by comparing the current detection result with the relative position of the known hangar QR code, and by performing binarization, gradient clustering, and edge extraction on a specific image area, and then using variance for convex quadrilateral fitting.

[0042] When the UAV's downward vision detects the hangar QR code, the UAV state machine generates a first instruction based on the accurate pose information of the UAV relative to the hangar landing point. The first instruction includes the heading angular velocity and a vector (representing the speed of the UAV in the X, Y, and Z directions). The heading angular velocity represents the heading angular velocity instruction, which is used to adjust the heading angle of the UAV, that is, to control the rotation direction of the UAV to ensure its correct landing attitude. According to the following formula (2), based on the deviation between the hangar QR code and the UAV's downward vision detection result, as well as the current speed and direction of the UAV, the heading angular velocity is calculated. According to formula (2), it can be seen that when the UAV state machine processes the landing information, the false detection probability is used as a weight factor to participate in the information fusion process. When there is a deviation between the detected QR code position and the ideal landing position, the value of will be adjusted according to the size of the deviation. The greater the deviation, the greater the value of may be, to quickly correct the deviation; otherwise, it will be reduced or even set to zero to maintain a stable heading. The vector represents the speed instructions of the UAV in the X, Y, and Z directions, which are used to adjust the linear speed of the UAV, so as to control the forward, lateral, and vertical speeds of the UAV, enabling it to move along the correct trajectory towards the hangar landing point. The specific values are calculated by formula (3) based on the position deviation information provided by the millimeter-wave radar at the hangar end, the monocular vision at the hangar end, and the UAV's downward vision, ensuring that the UAV can quickly correct the position deviation during the landing process.

[0043] (2)

[0044] (3)

[0045] Among them, is the false detection probability detected by the UAV monocular vision sensor; is the heading angular velocity; is the heading angular velocity gain, set according to experience; (W) represents finding the heading angle of W; () represents finding the inverse function; represents the initial rotation matrix from the UAV monocular vision sensor to the landing point (that is, before takeoff, when the UAV is at the landing point, the rotation matrix from the UAV monocular vision sensor to the landing point), set according to experience; is a vector representing the velocities in the X, Y, and Z directions; represents the velocity gain, set according to experience; represents the initial translation vector from the UAV monocular vision sensor to the landing point (that is, before takeoff, when the UAV is at the landing point, the translation vector from the UAV monocular vision sensor to the landing point), set according to experience.

[0046] [[ID=NO=30]]The generation of the first instruction takes into account the false detection probability of the hangar QR code detected by the UAV monocular vision sensor and the first pose (i.e., the precise pose information of the UAV relative to the hangar landing point), as well as information such as the current state of the UAV (including speed, position, and attitude). By fusing this information, the UAV state machine can comprehensively consider the reliability of various sensors and environmental factors to generate the most appropriate landing instruction. For example, if the false detection probability is high, the UAV state machine may rely more on the information provided by the millimeter-wave radar and the monocular vision at the hangar end; otherwise, it trusts more in the data of the UAV's downward vision. Through this intelligent decision-making mechanism, it is ensured that the UAV can land precisely and safely at the hangar under various conditions.

[0047] Optionally, Figure 4 is a schematic diagram of the processing flow of an optional UAV landing system according to an embodiment of the present application. As Figure 4 shown, after determining the target QR code (which has been explained in the above embodiments and will not be elaborated here), the UAV monocular vision sensor determines the false detection probability of the target QR code and sends the false detection probability to the UAV state machine. The UAV state machine receives the false detection probability, the first pose, and other sensor information. In the case of detecting the hangar QR code, the UAV state machine evaluates the reliability of the current detection in combination with the false detection probability. According to the evaluation result and the first pose, by adjusting the heading angular velocity and speed instructions, the first instruction is generated, and according to the first instruction, the UAV state machine controls the aircraft to adjust its attitude and speed to achieve precise landing.

[0048] For example, using deep learning techniques such as convolutional neural networks (CNNs), analyze the images of the drone's monocular vision sensor to determine the false detection probability of the target QR code. The specific steps are as follows: Construct a dataset containing real and false QR code images under various environmental conditions (such as different lighting and weather conditions), and label whether the QR code in each image is a false detection. Use the CNN model to train the dataset. The model is designed to be able to identify the features of the QR code in the image and predict the probability of it being a true detection or a false detection. During the drone landing process, the images captured by the drone's monocular vision sensor are input into the trained CNN model in real time. The model outputs the probability that the currently detected QR code is a false detection, that is, the false detection probability.

[0049] For example, based on statistical principles, through the analysis of historical detection data, evaluate the false detection probability of the target QR code under the current detection conditions. The specific steps are as follows: Collect a large amount of historical data on the drone's QR code recognition under different environmental conditions, including successful and failed recognition cases. In each historical detection case, extract the environmental condition features (such as light intensity, weather conditions) and the detection result features (such as the contrast and clarity of the QR code image). Using statistical methods such as Bayesian classifiers, based on the above condition features and detection result features, establish a model for predicting the false detection probability when identifying specific QR code features under specific environmental conditions. The image features (such as contrast and clarity) captured by the drone's monocular vision sensor are input into the established statistical model in real time, and the model outputs the corresponding false detection probability according to the current environmental conditions and image features.

[0050] Through this embodiment, by calculating the false detection probability, the drone's monocular vision sensor can determine whether the currently recognized target QR code is credible. Combining the false detection probability and the first pose (the precise position and attitude information of the drone relative to the hangar landing point), the drone state machine can generate a first instruction to effectively control the drone landing process. The introduction of the false detection probability enables the state machine to more cautiously evaluate the information from the drone's monocular vision sensor when generating instructions, avoiding landing errors caused by single false detections. Thus, when facing a complex external environment, it can more accurately and reliably identify the hangar QR code, optimize the information fusion strategy, formulate more reasonable landing instructions, and ultimately achieve the precise landing of the drone under various conditions, significantly improving the robustness and safety of the landing.

[0051] In an exemplary embodiment, the first monocular image includes the hangar, which means that the first monocular image contains the pixels of the hangar. The image range formed by all the pixels of the hangar is the area range of the hangar.

[0052] In some embodiments, the drone monocular vision sensor is further configured to convert the coordinates of the corner points of the reference two-dimensional code in the hangar two-dimensional code coordinate system to the image pixel coordinates of the drone monocular vision sensor, so as to obtain the corner pixel coordinates of the corner points of the reference two-dimensional code in the image pixel coordinates; the reference two-dimensional code refers to any two-dimensional code in the hangar two-dimensional code; the drone monocular vision sensor is further configured to determine whether the target two-dimensional code is within the area range of the hangar in the first monocular image when the corner pixel coordinates of the corner points of each two-dimensional code in the hangar two-dimensional code are outside the image range of the drone monocular vision sensor; when the target two-dimensional code is within the area range of the hangar in the first monocular image, determine that the false detection probability of the target two-dimensional code is the first probability; the first probability indicates that the target two-dimensional code is credible; when the target two-dimensional code is not within the area range of the hangar in the first monocular image, determine that the false detection probability of the target two-dimensional code is the second probability; the second probability indicates that the target two-dimensional code is not credible.

[0053] Among them, the hangar two-dimensional code coordinate system refers to a coordinate system established with the center of the hangar two-dimensional code as the origin according to the right-hand rule, which is used to represent the true physical positions of multiple two-dimensional codes inside the hangar. For example, the hangar two-dimensional code coordinate system is as Figure 2 shown. During the landing process, the hangar two-dimensional code coordinate system provides an absolute reference frame, enabling the drone monocular vision sensor to convert the detected two-dimensional code information into a unified coordinate system, thereby accurately calculating the pose deviation of the drone relative to the landing point. Any two-dimensional code in the hangar two-dimensional code can be used as the reference two-dimensional code to improve the flexibility and adaptability of recognition. It can be understood that the drone monocular vision sensor selects the clearest and most easily recognizable two-dimensional code as the reference two-dimensional code in different environments, thereby optimizing the positioning accuracy during the landing process.

[0054] The image pixel coordinates of the drone monocular vision sensor refer to the position coordinates of each pixel point in the image captured by the drone monocular vision sensor. The origin of the image pixel coordinate system is usually located in the upper left corner of the image, and the coordinate values change with the position of the pixel point in the image. During the landing process, the image pixel coordinates are used to represent the position of the hangar two-dimensional code in the field of view of the drone, which is a basic representation method in the detection and recognition process. By converting the coordinates of the corner points of the reference two-dimensional code in the hangar two-dimensional code coordinate system to the image pixel coordinates of the drone monocular vision sensor, the pixel coordinate positions of the four corner points of the reference two-dimensional code in the image of the drone monocular vision sensor are obtained.

[0055] The first probability indicates that the QR code recognized by the drone's monocular vision sensor is completely trustworthy and there is no possibility of false detection. When the drone's monocular vision sensor can accurately match the pixel coordinates of the corner points of the preset reference QR code and the detected corner points, and the corner points are within the expected image area, the false detection probability is set to the first probability (such as 0). This indicates that the recognition result of the QR code is completely consistent with the actual situation and can be used as a reliable basis for landing control.

[0056] The second probability indicates that the confidence level of the recognition result by the drone's monocular vision sensor is very low, that is, the recognition result may be incorrect. When the detected corner points are not within the preset image range, or the existence of the corner points cannot be verified through image processing within the preset area, the false detection probability is set to the second probability (such as 0.8), indicating that the drone's monocular vision sensor believes that the recognized QR code is very likely to be a false detection. Therefore, this result is hardly referred to in the landing decision to avoid making a landing decision based on incorrect information and ensure the safety and accuracy of landing.

[0057] In this embodiment, the setting of the first probability and the second probability is determined based on experience. The first probability represents the credibility of the target QR code, that is, the possibility of its false detection is relatively low, and it can be used for subsequent pose calculation and landing decision. The second probability means a higher false detection rate, indicating that the current detection result is unreliable and should not be used for generating control instructions.

[0058] Optionally, Figure 5 is a flowchart of an optional method for determining the false detection probability according to an embodiment of the present application. As Figure 5 shown, the drone's monocular vision sensor determines any QR code (i.e., the reference QR code) in the captured first monocular image used for identifying the hangar and determines the true physical coordinates of its four corner points in the hangar QR code coordinate system. For example, the reference QR code is denoted as reference QR code M, and M is one of the five QR codes A, B, C, D, and E in the hangar QR code as Figure 2 shown. The true physical coordinates of the four corner points of the reference QR code M are denoted as , and when the reference QR code M is transformed into the drone coordinate system as , where () represents the inverse function. The drone's monocular vision sensor uses the known internal parameter matrix K and distortion function Q of the drone's monocular vision sensor to transform the true physical coordinates of the four corner points of the reference QR code M from the hangar QR code coordinate system to the image pixel coordinate system of the drone's monocular vision sensor, and obtains the corner point pixel coordinates of the four corner points of the reference QR code M in the image pixel coordinate system This conversion process ensures that the corner information of the QR code can be aligned with the current visual information of the UAV, providing a basis for subsequent pose calculation and landing decision-making. When the UAV's monocular vision sensor attempts to convert the corner pixel coordinates of all the corners of the QR codes in the hangar QR code into the image, it will check whether the corner of each QR code is within the visible range of the first monocular image captured by the UAV's monocular vision sensor. If the corner pixel coordinates of the corners of each QR code are outside the image range, the UAV's monocular vision sensor will determine that the currently detected target QR code N may not be correctly recognized, or the hangar is not within the field of view. The UAV's monocular vision sensor further analyzes whether the target QR code N is still within the area range of the hangar in the first monocular image. If the target QR code N is indeed within the area range of the hangar in the first monocular image, it means that although other QR codes are not recognized, the detection of the target QR code N may be correct. The UAV's monocular vision sensor determines that the false detection probability of the target QR code N is the first probability (determined by experience, such as 0), indicating that the target QR code N is credible. Conversely, if the target QR code N is not within the area range of the hangar in the first monocular image, the UAV's monocular vision sensor determines that the false detection probability of the target QR code N is the second probability (determined by experience, such as 0.8), indicating that the target QR code N is not credible.

[0059] Through this embodiment, converting the corner coordinates of any reference QR code in the hangar from the hangar coordinate system to the image pixel coordinate system of the UAV's monocular vision sensor can verify whether the detected QR code actually exists in the hangar and whether its position and orientation meet the expectations. This coordinate conversion increases the accuracy and reliability of recognition. Especially in adverse environmental conditions, such as low light or strong reflection, it can reduce the possibility of misrecognition, thus ensuring that the UAV can accurately identify the hangar position even in a complex environment. Further, an evaluation mechanism for false detection probability is introduced. By judging whether the corner of the hangar QR code is within the UAV's monocular vision image range and whether it is within the area range of the hangar in the first monocular image, it can evaluate whether the currently detected target QR code is credible. If the target QR code is within the area range of the hangar in the first monocular image, even if the corners of other QR codes are outside the UAV's monocular vision image range, it can be determined that the detection of the target QR code is the first probability, indicating a high credibility. Conversely, if the target QR code is not within the area range of the hangar in the first monocular image, it is determined that its detection is the second probability, with low credibility. This evaluation of false detection probability based on position verification can make a more reasonable landing decision in the face of possible false detections.

[0060] In an exemplary embodiment, the monocular vision sensor of the UAV is further configured to calculate the absolute value of the coordinate difference between the corner pixel coordinates of the reference QR code and the detected pixel coordinates of the corner of the reference QR code when the corner pixel coordinates of the corner of the reference QR code are within the image range of the monocular vision sensor of the UAV; the detected pixel coordinates of the corner of the reference QR code refer to the pixel coordinates of the corner of the reference QR code obtained by the monocular vision sensor of the UAV from the first monocular image.

[0061] Among them, the detected pixel coordinates refer to the pixel position coordinates of the corner of the reference QR code determined by the monocular vision sensor of the UAV from the first monocular image through image processing and recognition algorithms. The detected pixel coordinates of the corner of the reference QR code directly reflect the recognition result of the monocular vision sensor of the UAV under the current environmental conditions, and are an important basis for evaluating the QR code detection accuracy, calculating the pose deviation, and generating landing instructions. By calculating the absolute value difference between the corner pixel coordinates and the detected pixel coordinates of the corner of the reference QR code, the monocular vision sensor of the UAV can judge the deviation between the detection result and the expectation, and then evaluate the credibility of the detection result. The absolute value of the coordinate difference between the corner pixel coordinates and the detected pixel coordinates of the corner of the reference QR code characterizes the gap between the actually detected corner position and the ideal position to evaluate the detection accuracy. The smaller the absolute value of the coordinate difference, the closer the detection result is to the expectation, and the higher the detection accuracy.

[0062] In an embodiment, the monocular vision sensor of the UAV is further configured to determine that the false detection probability of the target QR code is the first probability when the absolute value is less than the preset absolute value threshold; when the absolute value is greater than or equal to the preset absolute value threshold, perform binarization processing, gradient clustering processing, edge extraction, and convex quadrilateral fitting processing on the pixel points in the preset area of the corner pixel coordinates of the reference QR code in sequence.

[0063] Among them, the absolute value being less than the preset absolute value threshold indicates that the gap between the corner position of the reference QR code and the ideal position is within a reasonable error range. On the contrary, it indicates that the gap between the corner position of the reference QR code and the ideal position exceeds the reasonable error range. In this case, it is necessary to further comprehensively judge based on the pixel points in the preset area of the corner pixel coordinates of the reference QR code. Specifically, it includes performing binarization processing on the pixel points in the preset area of the corner pixel coordinates of the reference QR code to convert the preset area of the corner pixel coordinates of the reference QR code into a black-and-white image to enhance the contrast; calculating the gradients of the pixel points in the black-and-white image and clustering the pixels with similar gradients to locate potential edge or corner areas; extracting the edge pixels around the corner from the clustering result, removing noise, and accurately positioning the corner position; fitting the extracted edge pixels to find the best-matching convex quadrilateral.

[0064] In one embodiment, the monocular vision sensor of the unmanned aerial vehicle is further configured to determine that the false detection probability of the target two-dimensional code is the first probability when a fitted convex quadrilateral is obtained; when a fitted convex quadrilateral is not obtained, determine whether the corner pixel coordinates of the reference two-dimensional code are within the area of the hangar in the first monocular image; when the corner pixel coordinates of the reference two-dimensional code are within the area of the hangar in the first monocular image, determine that the false detection probability of the target two-dimensional code is the third probability; the third probability represents the uncertainty of the target two-dimensional code, and the third probability is greater than the first probability and less than the second probability; when the corner pixel coordinates of the reference two-dimensional code are not within the area of the hangar in the first monocular image, determine that the false detection probability of the target two-dimensional code is the second probability.

[0065] Among them, the third probability indicates that the monocular vision sensor of the unmanned aerial vehicle has a certain confidence in the recognition result, but there is still a certain risk of false detection. When the detected corner pixel coordinates deviate greatly from the expected coordinates, but approximate matching corners can be found through binarization processing, gradient clustering, edge extraction, and convex quadrilateral fitting processing within a preset area, the false detection probability is set to the third probability (such as 0.2), which indicates that although there is uncertainty in the recognition process, based on the verification of image processing, this result is still highly likely to be correct.

[0066] Optionally, as Figure 5 shown, when the corner pixel coordinates of the reference two-dimensional code M are within the visible range of the image of the monocular vision sensor of the unmanned aerial vehicle, the monocular vision sensor of the unmanned aerial vehicle captures an image of the current hangar (i.e., the first monocular image), and uses a deep network and the Apriltag detection method to identify the corners of the reference two-dimensional code. The specific positions of these corners in the image are the detected pixel coordinates. The monocular vision sensor of the unmanned aerial vehicle converts the corners of the reference two-dimensional code from the hangar two-dimensional code coordinate system to the image pixel coordinate system of the monocular vision of the unmanned aerial vehicle to obtain the corner pixel coordinates of the corners of the reference two-dimensional code; the monocular vision sensor of the unmanned aerial vehicle calculates the absolute value of the coordinate difference between the corner pixel coordinates of the corners of the reference two-dimensional code and the detected pixel coordinates, and adjusts the false detection probability of the target two-dimensional code based on the calculated absolute value of the coordinate difference. Specifically, taking the reference two-dimensional code M as an example, if |corner pixel coordinate - detected pixel coordinate | < σ (where σ is a preset absolute value threshold, determined according to experience), then the monocular vision sensor of the unmanned aerial vehicle considers the detection result to be credible, and the false detection probability of the target two-dimensional code is set to the first probability (such as 0); conversely, if within the area formed by the corner pixel coordinates of the reference two-dimensional code (by the parameter After binarization, gradient clustering, and edge extraction are performed (it is determined), a convex quadrilateral can be successfully fitted, which also indicates that the detection result is credible. The false detection probability is also set to the first probability (such as 0). If the above conditions are not all met, the unmanned aerial vehicle (UAV) monocular vision sensor will set the false detection probability to a relatively high third probability (such as 0.2) according to whether the target QR code is in the hangar and the environmental conditions to reflect the uncertainty of the detection result.

[0067] Through this embodiment, by calculating the absolute value of the coordinate difference between the corner pixel coordinates and the detected pixel coordinates, the deviation between the recognition result and the expected position can be quantified, so as to identify and screen out the QR code that highly matches the expected position, ensuring the use of high-precision position information during the UAV landing process and avoiding landing deviation caused by inaccurate QR code recognition; by setting a preset absolute value threshold and refining the false detection probability into the first probability, the third probability, and the second probability, the accurate grading of the confidence level of the recognition result is achieved. When the absolute value of the coordinate difference is less than the threshold, it indicates that the detection result of the UAV monocular vision sensor is highly credible, and the false detection probability is set to the first probability, which helps to improve the accuracy of the landing decision; when the absolute value of the coordinate difference is greater than or equal to the threshold, the existence of the corner is further verified through binarization, gradient clustering, edge extraction, and convex quadrilateral fitting. If a convex quadrilateral is successfully fitted, the false detection probability is also determined to be the first probability, which indicates that the UAV monocular vision sensor can correct the recognition deviation through image processing technology and ensure that the landing instruction is generated based on true and valid information; in the case where the convex quadrilateral cannot be successfully fitted, but the corner pixel coordinates are still within the hangar area, the false detection probability is set to the third probability, indicating that although the UAV monocular vision sensor has a certain degree of uncertainty about the detection result, based on the fact that the corner is still within the hangar range, this result can still be used as a reference for the landing decision, but its information will be treated with caution and the weight in the decision-making will be reduced. If the corner pixel coordinates are completely outside the hangar area, the false detection probability is the second probability, meaning that the detection result is extremely uncredible, and the UAV monocular vision sensor will tend to ignore this result to avoid making a landing decision based on incorrect information, thus ensuring the safety of the landing.

[0068] In an exemplary embodiment, the hangar monocular vision sensor is further configured to identify markers on the UAV from the second monocular image to obtain the corner points of the markers; calculate the fourth pose of the markers relative to the hangar monocular vision sensor based on the corner points of the markers; identify the specified two-dimensional code in the second monocular image to obtain the corner points of the specified two-dimensional code; calculate the fifth pose of the hangar monocular vision sensor relative to the specified two-dimensional code based on the corner points of the specified two-dimensional code; compensate the fifth pose according to the position difference between the landing point and the center point of the hangar two-dimensional code to obtain the sixth pose of the hangar monocular vision sensor relative to the landing point; the specified two-dimensional code is the two-dimensional code located at the center of the hangar two-dimensional code among multiple two-dimensional codes; determine the second pose based on the fourth pose and the sixth pose.

[0069] Among them, specific markers are provided on the UAV, such as battery knobs (the battery knobs are white, the batteries are black, and the shape is rectangular). These markers have fixed shape and color characteristics, which are convenient for the vision sensor to identify. Marker identification is a key function of the hangar monocular vision sensor. It can accurately detect these markers from the second monocular image, and then calculate the accurate position and attitude information of the markers relative to the hangar monocular vision sensor, that is, the fourth pose. By identifying the markers on the UAV, the accurate positioning of the UAV can be ensured even when the vision is blocked under the UAV.

[0070] Optionally, such as Figure 4As shown, the hangar monocular vision sensor captures the second monocular image from the perspective of the hangar. Using a deep learning-based model such as YOLOv7, it identifies the UAV marker (battery knob) in the second monocular image. It performs contour detection on the identified marker, and through adaptive threshold adjustment and statistical analysis, determines the positions of the four corner points of the marker. The hangar monocular vision sensor preferentially outputs the position of the upper left corner among the four corner points of the marker, and in a clockwise direction, sequentially outputs the remaining corner points of the marker. At this time, the positions of the four corner points of the marker are defined as (-W / 2, -L / 2, 0), (W / 2, -L / 2, 0), (W / 2, L / 2, 0), (-W / 2, L / 2, 0) respectively, where W and L are the width and length of the knob respectively. Then, it uses the Pnp (Perspective-n-Point) method to solve the fourth pose (including the rotation matrix Rs and the translation vector Ts) of the UAV marker relative to the hangar monocular vision sensor. The hangar monocular vision sensor obtains the approximate pixel position of the specified QR code in the second monocular image based on the position of the specified QR code in the hangar, and then uses the Apriltag QR code detection method to detect the specified QR code and its four corner points. At this time, the hangar monocular vision sensor defines the four corner points of the specified QR code as (-LE / 2, -LE / 2, 0), (LE / 2, -LE / 2, 0), (LE / 2, LE / 2, 0), (-LE / 2, LE / 2, 0) respectively, where LE is the size of the specified QR code. The hangar monocular vision sensor still uses the Pnp method to solve the fifth pose of the hangar monocular vision sensor relative to the specified QR code, and compensates for the position difference between the landing point and the center point of the hangar QR code on the fifth pose, and finally obtains the sixth pose of the hangar monocular vision sensor relative to the landing point (which may include the rotation matrix and the translation vector ). The hangar monocular vision sensor determines the second pose of the marker on the UAV relative to the landing point according to the fourth pose and the sixth pose, where, , .

[0071] Through this embodiment, the hangar monocular vision sensor can identify a specified QR code located at the center of the hangar. By accurately detecting and obtaining the corner points of the specified QR code, the pose of the hangar monocular vision sensor relative to the specified QR code (the fifth pose) can be calculated. The fifth pose provides global positioning information between the UAV and the hangar, which helps to comprehensively judge the landing position of the UAV. When calculating the fifth pose, considering the position deviation between the center point of the hangar QR code and the actual landing point, the hangar monocular vision sensor performs corresponding pose compensation to accurately reflect the actual position of the UAV relative to the landing point (the sixth pose). This compensation process ensures that even when using the QR code as a positioning reference, the actual position of the takeoff and landing point can be considered, improving the positioning accuracy. Subsequently, the hangar monocular vision sensor fuses the fourth pose of the UAV marker with the compensated sixth pose to obtain the comprehensive pose information of the UAV relative to the landing point (the second pose). This process makes full use of the advantages of the two positioning methods and improves the reliability of the UAV landing decision.

[0072] In an exemplary embodiment, the hangar monocular vision sensor is further configured to identify a marker on the UAV in the second monocular image to obtain a marker area, perform histogram statistics on the marker area to obtain a median gray value; divide the marker area into grids to obtain a plurality of grids, and determine at least one grid that meets the first preset condition among the plurality of grids as a group of first grids, and determine at least one grid that does not meet the preset condition among the plurality of grids as a group of second grids; the first preset condition refers to the condition that the pixel difference between the maximum pixel value and the minimum pixel value in the grid is greater than the first pixel threshold, the pixel value of each pixel point in the grid is greater than the second pixel threshold, and the pixel value of each pixel point in the grid is greater than the median gray value; modify the pixel value of each pixel point in each grid of a group of first grids to the first pixel value, and modify the pixel value of each pixel point in each grid of a group of second grids to the second pixel value; perform gradient clustering on the group of first grids with modified pixel values and the group of second grids with modified pixel values to obtain a plurality of first clustering points, calculate the variance of the pixel points within the preset area of each first clustering point among the plurality of first clustering points to obtain the variance corresponding to each first clustering point, and select a preset number of first clustering points in descending order of variance as the corner points of the marker.

[0073] Among them, in the second monocular image, the area enclosed by the contour of the UAV marker (such as the battery knob) recognized by the hangar monocular vision sensor is the marker area. The UAV marker can be extracted from the marker area, and then the precise position and attitude of the marker can be calculated for the positioning and guided landing of the UAV. Within the marker area, the median of the gray values of all pixel points is the median gray value. The calculation of the median gray value helps to determine the main color of the marker and maintain the stability of marker recognition under complex lighting conditions. In the embodiment, the median gray value is used as the benchmark for threshold processing to distinguish the internal and external pixels of the marker and improve the recognition accuracy.

[0074] To determine the contour of the marker, in this embodiment, the marker area is divided into grids, and the relative magnitude relationship between the difference between the maximum pixel value and the minimum pixel value in each grid and the first pixel threshold is compared. Here, the first pixel threshold is used to judge the threshold of the pixel difference inside the marker. A pixel difference greater than the first pixel threshold indicates that the pixel values in the grid change significantly, with an obvious brightness difference. This difference means that the analyzed grid is likely to contain the boundary of the marker or feature points with high contrast, which can help the hangar monocular vision sensor detect and locate the marker more accurately. In this embodiment, by comparing the difference between the maximum pixel value and the minimum pixel value in the grid with the first pixel threshold, the hangar monocular vision sensor can screen out grids with sufficient contrast and variation, which are more likely to be part of the marker contour, helping to improve the accuracy of contour detection and ensure the accurate positioning of the marker corner points. Specifically, if the difference between the maximum pixel value and the minimum pixel value in the grid is less than or equal to the first pixel threshold, then the grid is an internal grid of the marker; otherwise, the relative magnitude relationship between each pixel value in the grid and the second pixel threshold and the above median gray value is further judged to determine whether the grid is an edge grid of the marker. Here, the second pixel threshold is used to judge whether a pixel point belongs to the marker, which ensures the accuracy of marker contour detection. In this embodiment, by comparing the pixel value in the grid with the second pixel threshold and the median gray value, the internal and external pixels of the marker can be further distinguished, thereby improving the robustness and accuracy of marker contour recognition. Specifically, a pixel point value greater than the second pixel threshold indicates that the pixel brightness in the grid is higher than a specific threshold. The setting of the second pixel threshold can help the hangar monocular vision sensor exclude the interference of low-brightness backgrounds on marker detection and improve the recognition accuracy. The median gray value is the median of all pixel gray values in the entire marker area, representing the main color or brightness level of the marker. When the values of all pixels in the grid are higher than the median gray value, it means that the pixels in the grid are likely to belong to the bright part or feature part of the marker, rather than the background or other non-marker areas. In an image with a complex background, this condition can help the hangar monocular vision sensor more accurately define the scope of the marker and avoid misrecognition.

[0075] Accurately identifying and locating the corners of drone landmarks (such as battery knobs) is crucial for achieving precise landings. However, the edges of landmarks in images can become blurred due to factors such as lighting variations, reflections, and occlusion, resulting in reduced corner detection accuracy. To improve the robustness and accuracy of corner detection under these complex conditions, this embodiment proposes a processing flow based on grid analysis, binarization, clustering, and variance calculation. The landmark region may contain a large number of grayscale levels, which increases the complexity of subsequent processing and increases sensitivity to noise. To address this issue, this embodiment performs binarization on the landmark region (i.e., converting the pixel values in a set of first and second grids into first and second pixel values). This binarization simplifies the landmark region to only two grayscale levels (background and foreground), highlighting the landmark and edge features, simplifying the subsequent clustering and edge detection processes, and improving processing efficiency and corner detection accuracy. Within the binarized landmark region, the modified pixel value grids are processed using gradient clustering to identify and isolate the landmark boundary or corner region. Gradient clustering, based on the gradient changes between pixels, can effectively identify areas of high gradient change near the outlines and corners of landmarks. These areas are often important features of corners or edges. The benefit of clustering is that it can group pixels with similar characteristics, helping the hangar monocular vision sensor identify and locate the corners of landmarks in complex images, maintaining a high recognition rate even in conditions of uneven lighting or reflections. Variance calculation is performed on the pixels within the preset area of the first cluster point to further select the cluster points most likely to represent corners. The variance calculation reflects the degree of variation in pixel values within the area. Due to edge characteristics, the pixel values in the corner area will change more dramatically than in other areas. Therefore, cluster points with higher variance are more likely to be the corners of the landmark. Selecting a preset number of first cluster points as corner points in descending order of variance ensures that the corner points selected by the system have sufficient contrast and stability, improving the accuracy and robustness of corner point positioning.

[0076] Optionally, during the marker recognition process, the hangar monocular vision sensor first identifies the UAV marker (battery knob) from the second monocular image and determines its contour as the marker region B. Then, a histogram statistics of the pixels in the marker region B is performed to obtain the median gray value g. Subsequently, the marker region B is divided into multiple grids, and the size of each grid (such as 2×2 pixels) is determined by the parameters of the hangar monocular vision sensor. The hangar monocular vision sensor checks each grid to determine whether the difference between the maximum pixel value and the minimum pixel value in the grid is greater than the first pixel threshold PVT, and whether each pixel value V in the grid is greater than the second pixel threshold VT and the median gray value g, and based on this, a group of first grids (edge grids) and a group of second grids (internal grids) are distinguished. The hangar monocular vision sensor performs binarization processing on the group of first grids and the group of second grids, modifies the pixel value of each pixel point in each grid of the group of first grids to the first pixel value (usually set to 255, and 255 usually represents white or the brightest color), and modifies the pixel value of each pixel point in each grid of the group of second grids to the second pixel value (usually set to 0, and 0 usually represents black or the darkest color) to obtain a binarized image. The hangar monocular vision sensor calculates the gradient change between pixel points in the binarized image, finds the high-gradient regions that may represent edges or corners, clusters these high-gradient regions, that is, classifies the pixel points with similar gradient characteristics into the same group to form multiple first cluster points, calculates the variance of the pixel points within the preset region (such as a certain range centered on the cluster point) of each first cluster point, sorts the multiple first cluster points from largest to smallest according to the calculated variance, and selects a preset number of first cluster points as the corners of the marker.

[0077] In this embodiment, by performing histogram statistics on the pixel values of the marker region in the second monocular image, the median gray value is obtained. The median gray value can represent the brightness level of the marker body, thereby helping the hangar monocular vision sensor to more accurately identify the marker region in an environment with uneven illumination. By dividing the marker region into multiple grids and setting a first preset condition for pixel condition screening, it is possible to effectively distinguish the internal and external pixels of the marker, ensuring that the hangar monocular vision sensor only processes those grids containing high contrast and brightness details, thereby improving the accuracy of marker corner recognition. The pixel values of all pixel points in a group of first grids are modified to a first pixel value (such as 255, representing white), while the pixel values of all pixel points in a group of second grids are modified to a second pixel value (such as 0, representing black). Through this binary processing of black and white contrast, the hangar monocular vision sensor can enhance the contrast between the marker and the background, highlight the marker contour, and facilitate subsequent corner detection. Gradient clustering is performed on the binary image to identify the marker edge and potential corner regions. Subsequently, the variance of the pixel points within the preset region of each first clustering point is calculated. A region with a large variance indicates that the pixel values change violently and is very likely to be a corner region. Sorting according to the variance from large to small, and selecting a preset number of clustering points as the corners of the marker. This strategy can still maintain the accuracy of corner detection when the illumination and reflection conditions change.

[0078] In an exemplary embodiment, during the corner recognition process of a UAV marker (such as a battery knob), changes in illumination, reflection interference, or minor deformations of the marker itself may affect the accuracy and coverage rate of corner recognition. To improve the robustness of the hangar monocular vision sensor in a complex environment and achieve more stable corner detection, this embodiment introduces a dynamic adjustment mechanism for the first pixel threshold, which is closely related to the corner recognition coverage rate. That is, the first pixel threshold is dynamically updated following the corner recognition coverage rate of the hangar monocular vision sensor for the marker, which means that the first pixel threshold will be dynamically adjusted according to the corner recognition coverage rate of the hangar monocular vision sensor for the marker (i.e., the ratio of the number of successfully recognized corners to the theoretical number of corners). If the recognition coverage rate is low, the first pixel threshold will be automatically lowered, enabling the hangar monocular vision sensor to detect more pixel changes and improve the corner recognition rate; conversely, if the recognition coverage rate is high, the first pixel threshold will be raised to reduce false detections and optimize the recognition accuracy. This dynamic update mechanism enhances the adaptability and robustness of the hangar monocular vision sensor under different environmental conditions, ensuring the stability and accuracy of corner detection.

[0079] In some embodiments, the hangar monocular vision sensor is further configured to, when the recognition coverage rate of the corner points of the marker is greater than or equal to a preset coverage rate threshold, increase the first pixel threshold by a first step value to obtain an updated first pixel threshold; when the recognition coverage rate of the corner points of the marker is less than the preset coverage rate threshold, decrease the first pixel threshold by a second step value to obtain an updated first pixel threshold; wherein, during the process of updating the first pixel threshold, the updated first pixel threshold is always greater than the preset pixel value.

[0080] The recognition coverage rate refers to the ratio of the number of successfully recognized corner points to the number of corner points that the marker should theoretically have. The preset coverage rate threshold is a standard set by the hangar monocular vision sensor to determine whether the current corner point recognition coverage rate is high enough. When the recognition coverage rate is higher than or equal to the preset coverage rate threshold, the hangar monocular vision sensor considers the current first pixel threshold to be appropriate and does not need to overly reduce the threshold to reduce false detections; while when the recognition coverage rate is lower than the preset coverage rate threshold, the hangar monocular vision sensor will automatically adjust the first pixel threshold to improve the recognition rate of corner points. The setting of the preset coverage rate threshold ensures the stability of corner point recognition in different environments and avoids false detections or missed detections caused by too high or too low threshold settings.

[0081] The first step value refers to the fixed step length used by the hangar monocular vision sensor to increase the first pixel threshold when the recognition coverage rate of the corner points of the marker is higher than or equal to the preset coverage rate threshold. In this embodiment, when the hangar monocular vision sensor recognizes that the corner point coverage rate is high enough, it indicates that the current pixel threshold setting is reasonable and the internal features of the marker are obvious. At this time, by gradually increasing the first pixel threshold by the first step value, some false detections in edge cases can be automatically filtered out, thereby improving the accuracy of corner point recognition. The size of the first step value is determined by experience, which ensures that the adjustment of the first pixel threshold is carried out step by step and avoids a sudden drop in the recognition rate caused by excessive adjustment. For example, the first step value can be 1.

[0082] The second step value is the fixed step length used by the hangar monocular vision sensor to decrease the first pixel threshold when the recognition coverage rate of the corner points of the marker is lower than the preset coverage rate threshold. The introduction of this parameter is to address the situation where the corner point recognition rate decreases in a complex environment. When the environmental light is insufficient or there is reflective interference, the contrast of the corner points of the marker may weaken, resulting in a decrease in the recognition rate. At this time, by gradually decreasing the first pixel threshold by the second step value, the hangar monocular vision sensor can detect more pixel changes with weak contrast, thereby improving the recognition rate of corner points. The second step value is also determined by experience, and its use ensures that the hangar monocular vision sensor can quickly adjust the parameters in the case of a low recognition rate and restore the corner point recognition ability. For example, the second step value can be 5.

[0083] The updated first pixel threshold is always greater than the preset pixel value to ensure the distinction between the internal area of the marker and the background, avoid misidentifying too much background as marker features, and maintain the robustness of corner recognition. For example, the first pixel threshold PVT at the initial moment is 20. If the four points of the knob can be recognized subsequently, PVT is increased by 1, otherwise PVT is decreased by 5, but ensure that PVT is always greater than 10.

[0084] In this embodiment, when the hangar monocular vision sensor recognizes the marker of the drone, the first pixel threshold is automatically updated according to the corner recognition coverage rate, which enhances the corner recognition ability of the hangar monocular vision sensor under different lighting and reflection conditions; when the recognition coverage rate is high, the first pixel threshold is increased according to the first step value, reducing false detection and optimizing the recognition accuracy; when the recognition coverage rate is low, the first pixel threshold is decreased according to the second step value, increasing the detection sensitivity and improving the corner recognition rate; ensuring that the updated first pixel threshold is always greater than the preset pixel value, maintaining the distinction between the marker and the background, avoiding a large number of background misidentifications caused by too low a threshold setting, and maintaining the stability of corner recognition.

[0085] In an exemplary embodiment, the change in light intensity will affect the visual contrast of the marker, and a fixed second pixel threshold is difficult to adapt to all scenarios. Therefore, to solve this problem, in this embodiment, an adaptive parameter is introduced, and the second pixel threshold is dynamically updated following the adaptive parameter that cyclically takes values from multiple parameter values to optimize the corner detection effect, where the adaptive parameter is used to scale the second pixel threshold. The dynamic update of the second pixel threshold aims to improve the accuracy and stability of corner recognition of the drone marker (battery knob) under different environmental lighting conditions.

[0086] In some embodiments, the hangar monocular vision sensor is further configured to, according to the recognition result of the corners of the marker by the hangar monocular vision sensor, extract the parameter value that matches the recognition result from multiple parameter values as the adaptive parameter; and calculate the second pixel threshold according to the preset binarization threshold formula, where the binarization threshold formula represents the correlation relationship between the maximum pixel value, the minimum pixel value and the adaptive parameter in the grid.

[0087] Among them, the recognition result of the corner points of the marker by the hangar monocular vision sensor refers to whether the hangar monocular vision sensor recognizes the corner points of the marker. The parameter value matching the recognition result means that when the corner point recognition is stable, the hangar monocular vision sensor will select a parameter value to make the second pixel threshold appropriate, avoiding excessive filtering of the marker details or introducing too much background noise. On the contrary, in an environment with a low recognition rate, the hangar monocular vision sensor will automatically adjust the parameter value to reduce the second pixel threshold to improve the sensitivity of corner detection and ensure stable recognition of the corner points of the UAV marker under various conditions. For example, the adaptive parameter k is cyclically selected among the three parameter values of 7, 8, and 9. If the current adaptive parameter k is 9 and the current knob corner point is not recognized, then k is taken as 7; otherwise, k is always 9.

[0088] There is a certain correlation relationship among the maximum pixel value, the minimum pixel value, and the adaptive parameter in the grid, and this correlation relationship is the binarization threshold formula. The binarization threshold formula is used to associate the maximum pixel value, the minimum pixel value, and the adaptive parameter in the grid to calculate the second pixel threshold. For example, the binarization threshold formula can adopt formula (4). It can be understood that the binarization threshold formula can ensure that the second pixel threshold can not only highlight the marker features but also suppress the background noise.

[0089] (4)

[0090] Optionally, the hangar monocular vision sensor performs image processing to recognize the corner points of the UAV marker (battery knob). According to the recognition result of the current corner point, it selects a parameter value that best matches the current recognition result from a preset multiple parameter values. The selection of this parameter value is based on empirical rules, that is, in the case of a high recognition coverage rate, a larger parameter value may be selected to reduce false detections; in the case of a low recognition coverage rate, a smaller parameter value is selected to improve the detection sensitivity. The selected parameter value is the adaptive parameter, and using the preset binarization threshold formula (4), the second pixel threshold is calculated .

[0091] In this embodiment, the second pixel threshold follows and dynamically updates an adaptive parameter cyclically selected from multiple preset parameter values. The hangar monocular vision sensor can adapt to different environmental lighting conditions and scene complexities. In particular, it solves the problem of low corner recognition rate of UAV markers in low-light or strong-reflection environments. The adaptive parameter is used to adjust the second pixel threshold, which means that the hangar monocular vision sensor can adjust the strictness of the second pixel threshold according to the current corner recognition result of the marker. That is, when the corner recognition coverage rate is high, the second pixel threshold is increased to reduce false detections and ensure the recognition accuracy. When the recognition coverage rate is low, the second pixel threshold is decreased to increase the detection sensitivity and improve the robustness in complex or adverse environments. The preset binarization threshold formula combines the maximum pixel value , the minimum pixel value and the adaptive parameter . It not only considers the brightness difference in the image but also introduces a mechanism for dynamically adjusting according to the corner recognition result, effectively dealing with the influence of contrast changes on corner detection in different scenarios and ensuring the performance of corner detection in a changing environment.

[0092] In an exemplary embodiment, one of the main challenges faced by the precise landing of a UAV is that in adverse weather conditions, such as low-light or strong-reflection environments, the monocular vision system may not be able to reliably identify the landing marker, resulting in a decrease in landing accuracy. To solve this problem, a joint use scheme of the hangar-side millimeter-wave radar and vision is proposed in this embodiment. Due to its characteristics, the millimeter-wave radar can penetrate adverse weather conditions such as fog and rain and provide accurate position information of the target. Therefore, through radar detection and clustering, combined with the micro-Doppler technology, the UAV can be effectively identified and tracked, and guidance information can be provided even when the vision system fails.

[0093] In some embodiments, the hangar radar is also used to cluster a set of object point cloud data obtained from each scan in multiple scans to obtain a set of second clustering points, and determine the conditional probability that each second clustering point in the set of second clustering points identified in each scan is a UAV.

[0094] Among them, as Figure 4 shown, in multiple scans of the hangar radar, each set of object point cloud data in multiple sets of object point cloud data corresponds to one scan in multiple scans, that is, the hangar radar can obtain a set of object point cloud data every time it scans. After a set of object point cloud data obtained from each scan is processed by a clustering algorithm, a set of point sets classified as potential UAV targets is formed, and this point set is called a set of second clustering points. A set of second clustering points is identified through clustering algorithms such as DBSCAN based on characteristics such as the physical position, speed, and radar cross-section of radar detection points, and is considered to be target points that may belong to the UAV.

[0095] Under each radar scan, for each point in a set of identified second clustering points, based on its micro-Doppler feature and other radar attributes (such as RCS, Doppler velocity, signal-to-noise ratio, etc.), the conditional probability that it is a drone is calculated by the Bayesian method. The conditional probability that a second clustering point is a drone reflects the likelihood that this clustering point is a drone under the current radar detection conditions and is an important basis for judging and tracking drones.

[0096] In some embodiments, the hangar radar is further configured to determine the cumulative probability that each second clustering point identified by the hangar radar is a drone based on the conditional probability that each second clustering point is a drone under each scan, and remove the second clustering points with a cumulative probability less than a first preset probability threshold from the set of second clustering points, and determine the second clustering points with a cumulative probability greater than a second preset probability threshold in the set of second clustering points after removal as the target points of the drone.

[0097] Among them, since there may be false detections and missed detections in the hangar radar detection results, in order to ensure system performance, it is necessary to distinguish which are the clustering points (i.e., drones) that really need attention and which may be noise or misidentifications. In this embodiment, the cumulative probability that each second clustering point is a drone is used to evaluate and confirm whether the detected target is indeed a drone, so as to avoid the system making decisions based on incorrect information. As Figure 4 shown, the cumulative probability that each second clustering point is a drone refers to the cumulative value of the conditional probability that it is a drone for the same second clustering point (potential drone target) in multiple radar scans. The cumulative probability reflects the long-term estimate of the likelihood that this clustering point is a drone under continuous observations. It is obtained by iterative calculation based on the micro-Doppler technology and in a Bayesian manner. The calculation formula of the cumulative probability is shown in formula (5):

[0098] (5)

[0099] Among them, represents the conditional probability that the current target is a drone under the 1st to Nth measurements; represents the conditional probability that the current target is a drone at the Nth measurement; represents the conditional probability that the current target is a drone under the 1st to N - 1th measurements; represents the probability that the current target is a drone.

[0100] In this embodiment, the second clustering points with cumulative probability less than the first preset probability threshold are determined as false targets (i.e., noise points), and the noise points are removed. The first preset probability threshold is used to remove those second clustering points with cumulative probability lower than this threshold, which are considered to have a greater possibility of false detection and are not used as the basis for subsequent tracking and guidance. Setting the first preset probability threshold can effectively reduce false detection and improve the accuracy and efficiency of the hangar radar detection. The specific value of the first preset probability threshold can be set according to experience. For example, the first preset probability threshold can be set to 0.3.

[0101] The first preset probability threshold is used to initially remove false detections and reduce unnecessary calculations and processing; while the second preset probability threshold is used to further screen out clustering points with high confidence to ensure the accuracy of subsequent tracking and guidance. The second preset probability threshold is used to screen out the second clustering points with cumulative probability higher than this threshold, which are considered to have a higher possibility of being an unmanned aerial vehicle (UAV) and can be used as UAV target points for subsequent tracking and guidance. The set of points above this threshold is regarded as a more reliable detection result, which is used to improve the robustness and landing accuracy of the system. Through these two-level probability screenings, the hangar radar can track the UAV more precisely and improve the landing success rate and safety in harsh environments.

[0102] Among a set of second clustering points, the clustering points that meet the conditions screened out by the cumulative probability and the preset probability threshold are the target points of the UAV. The target points of the UAV are considered as the positions of the UAV with high confidence and are used for subsequent UAV tracking and guidance.

[0103] In some embodiments, the hangar radar is further configured to determine the UAV position based on the target points of the UAV, the pose transformation matrix between the hangar radar and the hangar monocular vision sensor, and the sixth pose of the hangar monocular vision sensor relative to the landing point.

[0104] Among them, the pose transformation matrix between the hangar radar and the hangar monocular vision sensor is obtained by joint calibration between the hangar radar and the hangar monocular vision sensor, which will not be elaborated here. As Figure 4 shown, the pose transformation matrix between the hangar radar and the hangar monocular vision sensor includes a rotation matrix and a translation vector . The sixth pose of the hangar monocular vision sensor relative to the landing point includes a rotation matrix and a translation vector , and the calculation process of the sixth pose has been explained in the above embodiments and will not be elaborated here.

[0105] The hangar radar can perform relative position conversion according to formula (6) to obtain the UAV position:

[0106] (6)

[0107] Among them, is the position information of the target point where the cumulative probability that the current target is a drone is greater than the second preset probability threshold under the 1st to Nth measurements.

[0108] Through this embodiment, a group of object point cloud data obtained by each scan is clustered to obtain a group of second clustering points, and the conditional probability that each clustering point is a drone is calculated. This calculation of the conditional probability, based on the radar features of the clustering points, can identify and track possible drone targets in various complex environments, improving the reliability of landing guidance; based on the conditional probability of the second clustering points identified under each scan, the cumulative probability that each second clustering point is a drone is further calculated and updated. The calculation of the cumulative probability is obtained through the accumulation of historical observation data and iterative Bayesian methods, which can reflect the accumulation and adjustment of the confidence level of the hangar radar in a certain clustering point being a drone over time. By removing the second clustering points with a cumulative probability less than the first preset probability threshold, false detections are reduced, and the second clustering points with a cumulative probability greater than the second preset probability threshold after removal are determined as the target points of the drone, ensuring that the radar information for subsequent guidance is more accurate and effective, which is particularly important in complex or adverse environments and helps the drone land in the correct position; finally, based on the target points of the drone, the hangar radar combines the pose transformation matrix between the hangar radar and the hangar monocular vision sensor, and the sixth pose of the hangar monocular vision sensor relative to the landing point to determine the position of the drone in the hangar QR code coordinate system, with the origin of this coordinate system being the landing point. This process is the fusion of multi-sensor information. The position information provided by the radar is used to calibrate and enhance the positioning result of the vision system, and vice versa. Through this fusion, even when a single sensor (such as vision) is restricted by the environment, the hangar radar can use the all-weather performance of the radar to accurately estimate the relative position of the drone and provide more precise landing guidance.

[0109] In an exemplary embodiment, the hangar radar is further configured to use each scan in multiple scans as the current scan and perform the following tracking operations to obtain the conditional probability of identifying a drone under each scan:

[0110] 1. Cluster a group of object point cloud data obtained under the current scan to obtain multiple third clustering points; each third clustering point among the multiple third clustering points includes the polar coordinates of each third clustering point, the Doppler velocity of each third clustering point relative to the hangar radar, and the signal-to-noise ratio of each third clustering point.

[0111] The coordinates of each cluster point relative to the radar position in radar detection are expressed using polar coordinates. Polar coordinates are a two-dimensional coordinate system that, unlike rectangular coordinates, uses distances and angles to define points on a plane. For the third cluster point, the polar coordinates specifically include the distance R from the third cluster point to the radar, the angle A of the third cluster point with respect to the X-axis of the radar coordinate system, and the angle E of the third cluster point with respect to the Z-axis of the radar coordinate system.

[0112] Due to its physical properties, hangar radars operate stably in all weather and lighting conditions, providing information such as target distance, angle, and velocity. However, radar detection results may include echoes from non-target objects. To distinguish drone echoes from interference, Doppler velocity and signal-to-noise ratio are used as key metrics. Cluster analysis and probability calculations are used to improve the accuracy and robustness of drone target recognition.

[0113] In radar systems, Doppler velocity refers to the radial velocity component of a target relative to the radar, determined by analyzing the frequency changes in the radar echo. In this embodiment, by detecting the Doppler velocity of each third cluster point, it is possible to distinguish between truly moving drone targets and static or slow-moving interfering objects. This is because static objects or background noise in a hangar environment do not produce a significant Doppler shift, while the motion of a drone does. Using Doppler velocity, drones can be effectively identified and tracked, maintaining high recognition rates and tracking performance even in complex environments.

[0114] In millimeter-wave radar systems, the signal-to-noise ratio represents the contrast between the radar echo signal strength and the background noise level. In this embodiment, the Doppler velocity is combined with the signal-to-noise ratio to further improve the accuracy of drone identification. During the radar point cloud data clustering process, by analyzing the signal-to-noise ratio of each third cluster point, it is possible to distinguish between high-signal-to-noise ratio echoes with signal strength sufficient to represent drone targets and low-signal-to-noise ratio echoes from background noise or non-target objects. In severe weather conditions, echoes with high signal-to-noise ratios are more likely to come from drones. Therefore, the signal-to-noise ratio is a key discriminant in the identification process, which helps to eliminate false detections and improve the reliability of drone target identification.

[0115] 2. Select at least one third cluster point from multiple third cluster points, whose Doppler velocity relative to the hangar radar is within a preset velocity range and whose signal-to-noise ratio is within a preset signal-to-noise ratio range, to obtain a group of third cluster points; convert each third cluster point from the polar coordinate system to the radar Cartesian coordinate system, and cluster the group of third cluster points in the radar Cartesian coordinate system to obtain a group of fourth cluster points, where each fourth cluster point in the group of fourth cluster points includes the initial motion state parameters of the UAV.

[0116] Among them, the object point cloud data detected by the radar may contain echoes from different objects. To ensure that the information finally used for UAV tracking and positioning is accurate and relevant, in this embodiment, a preset speed range, a preset signal-to-noise ratio range are set, and the identification of the fourth cluster points is carried out to screen out valid points.

[0117] During the millimeter-wave radar detection process, the Doppler velocity of the UAV should conform to a preset reasonable range, and this reasonable range is the preset speed range. The preset speed range is set based on the estimation of the maximum Doppler velocity and the minimum Doppler velocity of the UAV during the landing process, and it is used to screen out the point cloud data that conforms to the motion characteristics of the UAV. For example, the maximum Doppler velocity threshold DT is the upper limit of the preset speed range, ensuring that only the radar detection points within the preset speed range (e.g., 0.1 to DT) are considered, thereby eliminating background noise and irrelevant targets.

[0118] The preset signal-to-noise ratio range is used to screen out the radar detection points with a high enough signal-to-noise ratio, that is, good signal quality. In this embodiment, the points with a signal-to-noise ratio between 0 and SNRT (preset signal-to-noise ratio threshold) are selected. SNRT is also an empirically set value. This step helps to eliminate low-quality signals, improve the accuracy of the cluster points, and ensure that they are more likely to be the echoes of the UAV.

[0119] Optionally, after initially screening multiple third clustering points (based on Doppler velocity and signal-to-noise ratio), the remaining third clustering points can be transformed from the polar coordinate system to the radar Cartesian coordinate system (X, Y, Z) for further clustering analysis, where X = R * sin(E) * cos(A), Y = R * sin(E) * sin(A), Z = R * cos(E), R represents the distance from the third clustering point to the radar; A represents the angle between the third clustering point and the X-axis of the radar coordinate system; E represents the angle between the third clustering point and the Z-axis of the radar coordinate system. These points are clustered using a density-based clustering algorithm (such as DBSCAN), and a new set of tighter clustering points obtained are called fourth clustering points. The association distance is determined by the size of the UAV. Each fourth clustering point output after clustering contains clustering information such as the position (Xc, Yc, Zc) and length-width of the fourth clustering point, the RCS (taking the maximum RCS of all clustering points), and the Doppler velocity (selecting the Doppler velocities with the top few largest signal-to-noise ratios). At the initial stage of target tracking, using the clustering information of each fourth clustering point output after clustering, the initial motion state parameters {x, y, z, Vx, Vy, Vz, Ax, Ay, Az} of the UAV are obtained, where x, y, z represent the initial position of the UAV, directly replaced by the position (Xc, Yc, Zc) of the fourth clustering point; Ax, Ay, Az are the accelerations of the UAV in the horizontal, vertical, and vertical directions respectively, and Ax, Ay, Az are all assigned zero; Vx, Vy, Vz are the velocities of the UAV in the horizontal, vertical, and vertical directions respectively, Vx is assigned the average value of the clustering Doppler velocity in the horizontal direction, and Vy, Vz are assigned zero.

[0120] III. According to the initial motion state parameters of each fourth clustering point, predict the future motion state parameters of each fourth clustering point, and fuse the future motion state parameters of each fourth clustering point with the radar Cartesian coordinates of each fourth clustering point to obtain each updated fourth clustering point.

[0121] Among them, during the target tracking process, a uniformly accelerated model is used to predict the position of the UAV. Specifically, the initial motion state parameters of each fourth clustering point are input into the trained uniformly accelerated model to obtain the future motion state parameters of each fourth clustering point (including the future position, future speed, and future acceleration of each fourth clustering point). When new radar scan data arrives, the Cartesian coordinate information of the fourth clustering point is compared with the future motion state parameters, and the deviation between the Cartesian coordinate information and the future motion state parameters is calculated through the Kalman gain, and the future motion state parameters are corrected. Through the information fusion of the Kalman filter, the future motion state parameters of each fourth clustering point will be updated to obtain the updated each fourth clustering point. This means that for each fourth clustering point, the hangar radar will obtain updated position and speed information that is more accurate and closer to the true motion state.

[0122] IV. Statistically calculate the Doppler velocities of all third clustering points within the preset range of each updated fourth clustering point relative to the hangar radar to obtain the maximum Doppler velocity, minimum Doppler velocity, and Doppler velocity distribution variance corresponding to each fourth clustering point; determine the conditional probability that each fourth clustering point is a UAV under the current scan based on the maximum Doppler velocity, minimum Doppler velocity, and Doppler velocity distribution variance corresponding to each fourth clustering point.

[0123] Among them, statistically calculating the Doppler velocities of all third clustering points within the preset range of each updated fourth clustering point relative to the hangar radar is to more carefully analyze and confirm whether the fourth clustering point truly represents a UAV.

[0124] The maximum Doppler velocity D1, minimum Doppler velocity D2, and Doppler velocity distribution variance D3 obtained in each scan will change according to the specific scan situation. This means that at different stages of the UAV's landing or under different environmental conditions, these statistics will vary. For example, when the UAV approaches the hangar, its speed may change, resulting in changes in the Doppler velocity range and variance. In addition, weather conditions (such as wind speed, rain, and snow) and the surrounding environment (such as obstacles and reflectors) will also affect the detected Doppler velocity, thereby affecting the statistical results and the calculation of the conditional probability.

[0125] Based on the maximum Doppler velocity, minimum Doppler velocity, and Doppler velocity distribution variance corresponding to each fourth clustering point, the hangar radar will determine the conditional probability that each fourth clustering point is a UAV under the current scan. This probability calculation is based on the association between the Doppler velocity characteristics and the known UAV motion patterns.

[0126] In some embodiments, the hangar radar is further configured to determine that the conditional probability of each fourth clustering point being a drone under the current scan is the first probability when each fourth clustering point meets the second preset condition; and determine that the conditional probability of each fourth clustering point being a drone under the current scan is the second probability when each fourth clustering point does not meet the second preset condition; wherein, the second preset condition refers to the condition that the velocity difference between the maximum Doppler velocity and the minimum Doppler velocity corresponding to each fourth clustering point is greater than a preset velocity threshold, and the variance of the Doppler velocity distribution is greater than a preset variance threshold.

[0127] Among them, the preset velocity threshold is a Doppler velocity difference standard set to distinguish the dynamic signal of the drone from the environmental static signal or interference signal. During the landing or flight of the drone, its velocity will change, which is reflected as a change in the Doppler velocity in the radar echo. The preset velocity threshold is used to determine whether the velocity change of the fourth clustering point conforms to the dynamic characteristics of the drone, so as to assist in judging whether it is the signal of the drone. This threshold is set based on experience and is designed to screen out dynamically changing signals and improve the recognition rate of drones.

[0128] The preset variance threshold is used to evaluate the uniformity or dispersion degree of the Doppler velocity distribution of the fourth clustering point to further confirm whether it represents the target characteristics of the drone. The Doppler velocity distribution of the drone echo usually has certain characteristics, which are different from the distribution of static objects or background noise. The preset variance threshold is set according to this characteristic to distinguish the dynamic target signal with relatively high dispersion (such as a drone) from the static signal or interference signal with relatively uniform distribution. By comparing the variance of the Doppler velocity distribution with the preset variance threshold, the drone can be more accurately identified and the false detection rate can be reduced.

[0129] During the detection process of the hangar radar, if a fourth clustering point meets the second preset condition (that is, the velocity difference between its maximum Doppler velocity and minimum Doppler velocity is greater than the preset velocity threshold, and the variance of the Doppler velocity distribution is greater than the preset variance threshold), then this clustering point is regarded as a signal highly likely to represent a drone. The first probability is the conditional probability under this judgment, that is, the probability that this fourth clustering point is a drone under the current scan. It reflects the high confidence of the hangar radar that the target is a drone. Usually, this probability will be relatively high to distinguish it from static objects or mis-detected signals in the environment.

[0130] The second probability is relative to the first probability. If a fourth clustering point does not meet the second preset condition, that is, the velocity difference between its maximum Doppler velocity and minimum Doppler velocity does not exceed the preset velocity threshold, or the variance of the Doppler velocity distribution is lower than the preset variance threshold, then the hangar radar will evaluate the probability of it being a drone as low, that is, the second probability. The second probability reflects the lower confidence of the hangar radar that the target is a drone, which may mean that this clustering point is caused by environmental interference, reflection of static objects, or other non-drone echoes.

[0131] For example, Taking 0.5, when the maximum Doppler velocity D1 - the minimum Doppler velocity D2 is greater than the preset velocity threshold DT and the variance D3 of the Doppler velocity distribution is greater than the preset variance threshold PT, is 0.9, otherwise is 0.4, representing the conditional probability that the target is a drone under the Nth measurement.

[0132] Through this embodiment, by using the hangar radar to preliminarily cluster the object point cloud data, multiple third clustering points are obtained, and then points that meet the preset velocity range and preset signal-to-noise ratio range are further screened out from these third clustering points, and finally fourth clustering points are obtained, which helps to quickly identify potential drone signals from a large number of radar echoes, reduces the interference to drone recognition in low-light or strong-reflection environments, and enhances the system's identification ability in complex environments; based on the initial motion state parameters of the fourth clustering points, the hangar radar can predict the future motion state, and fuse the prediction result with the actual radar coordinate data through algorithms such as Kalman filtering to update and optimize the motion model of the drone. This process improves the accuracy of the drone's position and speed estimation, enabling the hangar radar to more accurately guide the drone to land, especially when the vision system is affected by the environment; by statistically updating the Doppler velocity of each fourth clustering point, including the maximum value, minimum value, and distribution variance, the hangar radar can utilize the micro-Doppler feature, that is, the Doppler frequency shift generated when the drone's propeller rotates, to dynamically evaluate the probability that each fourth clustering point is a drone under the current scan, thereby effectively distinguishing drone signals from background noise and improving the accuracy and environmental adaptability of target recognition; according to the second preset condition, that is, whether the Doppler velocity difference and variance exceed the preset threshold, the hangar radar dynamically adjusts the conditional probability that each fourth clustering point is a drone to the first probability or the second probability, enabling the hangar radar to intelligently evaluate the credibility of the target according to the motion characteristics of the drone (such as speed change) and the quality of the radar signal (such as variance size), so as to maintain stable recognition performance in different environments and improve the robustness of drone landing.

[0133] In an exemplary embodiment, in the scenario where a drone precisely lands in a hangar, especially under low light, strong reflection, or adverse weather conditions, the effectiveness of the drone's monocular vision sensor may be significantly affected, resulting in the inability to accurately identify the QR code on the hangar, and thus the inability to use visual information for positioning and guidance. To address this challenge, the embodiment of the present application proposes a drone landing assistance system that integrates millimeter-wave radar and monocular vision, and introduces the concept of a second instruction. The second instruction is used to control the drone to land at a landing point to ensure that the drone can still be effectively guided to land precisely when the drone's monocular vision sensor fails.

[0134] In some embodiments, the drone state machine is further configured to generate a second instruction according to the second pose in the case where the drone's monocular vision sensor fails to detect the QR code on the hangar, and control the drone to land at the landing point according to the second instruction.

[0135] Among them, the second instruction is a control instruction generated through data fusion and state estimation based on the detection results of the millimeter-wave radar and monocular vision sensor at the hangar end in the case where the drone's monocular vision sensor fails to detect the QR code on the hangar. When the drone's monocular vision sensor cannot obtain effective landing reference information due to environmental factors, the sensors at the hangar end (hangar radar and hangar monocular vision sensor) will play a more crucial role. The drone state machine will calculate how the drone should adjust its motion state to land safely at the predetermined landing point according to the second pose of the drone relative to the hangar provided by these sensors. The second instruction includes speed adjustments in the X, Y, and Z directions of the drone and possible heading angular velocity instructions, ensuring that the drone can land along the optimal path and achieve safe and precise landing through the guidance of the millimeter-wave radar and hangar monocular vision even when the vision sensor is unreliable.

[0136] Optionally, if the drone's monocular vision sensor fails to detect the QR code on the hangar, the detection result of the hangar monocular vision sensor, that is, the second pose, is used to generate the second instruction. The second instruction is similar to the first instruction and both include the heading angular velocity and a vector (representing the speed of the drone in the X, Y, and Z directions), which will not be elaborated here. Among them, according to formula (7), the heading angular velocity is calculated, and according to formula (8), the vector is calculated:

[0137] (7)

[0138] (8)

[0139] Among them, is the heading angular velocity, is the heading angular velocity gain, which is set according to experience, (W) represents the heading angle of W, () represents the inverse function, represents the rotation matrix of the set drone knob to the landing point (that is, before takeoff, when the drone is at the landing point, the rotation matrix of the drone's monocular vision sensor to the landing point), which is set according to experience, is a vector representing the velocities in the X, Y, and Z directions, represents the velocity gain, which is set according to experience, represents the translation vector of the set knob to the landing point (that is, before takeoff, when the drone is at the landing point, the translation vector of the drone's monocular vision sensor to the landing point), which is set according to experience.

[0140] Through this embodiment, the performance of the drone's monocular vision sensor will drop significantly in a harsh environment, resulting in the inability to recognize the QR code on the hangar, thus affecting the landing guidance. By introducing a drone state machine to generate a second instruction based on the second pose, the drone state machine can still more accurately guide the drone to adjust its flight attitude and speed in the case of the failure of the drone's monocular vision sensor, so as to achieve a more precise landing.

[0141] In an exemplary embodiment, in the scenario where the drone precisely lands on the hangar, extreme environments (such as low light, strong reflection, or bad weather) pose severe challenges to the landing process, especially when both the drone's monocular vision sensor and the hangar's monocular vision sensor may be unable to obtain normal information due to environmental factors. To solve this problem, the embodiment of this application proposes a drone landing assistance system integrating millimeter-wave radar and monocular vision, in which the concept of a third instruction is introduced. The third instruction is used to control the drone to land at the landing point to ensure that the drone can still be effectively guided to land precisely when the drone's monocular vision sensor and the hangar's monocular vision sensor fail.

[0142] In some embodiments, the drone state machine is further configured to generate a third instruction according to the drone's position in the case where the hangar's monocular vision sensor cannot detect the second pose, and control the drone to land at the landing point according to the third instruction.

[0143] Among them, the third instruction is a control instruction generated by the UAV state machine based on the UAV position information when the hangar monocular vision sensor fails to detect the second pose, and is used to directly guide the UAV to land at a pre-set landing point. When the UAV monocular vision sensor and the hangar monocular vision sensor cannot provide the first and second poses of the UAV relative to the hangar due to environmental restrictions, the UAV state machine will rely on the UAV position information provided by the hangar millimeter-wave radar and generate the third instruction through algorithm processing. The third instruction includes adjustment instructions for the UAV's motion state, including but not limited to speed adjustments in three directions (X, Y, Z) and possible heading angular velocity adjustments, to ensure that the UAV can land safely and accurately at the hangar's landing point even in the absence of visual information. This mechanism, as the final line of defense of the landing guidance system, improves the adaptability of the UAV in the face of various adverse environmental conditions.

[0144] Optionally, if the hangar monocular vision sensor fails to detect the hangar QR code, the detection result of the hangar radar, i.e., the UAV position, is used to generate the third instruction, where the third instruction is similar to the first instruction and both include the heading angular velocity and a vector (representing the UAV's speed in the three directions of X, Y, and Z), which will not be elaborated here. Among them, according to formula (9), the vector is calculated as:

[0145] (9)

[0146] Among them, is a vector, representing the speed in the three directions of X, Y, and Z, represents the speed gain, set according to experience, represents the UAV position in the QR code coordinate system. At this time, the heading angular velocity is zero.

[0147] In some embodiments, the UAV state machine is further configured to control the UAV to be stationary relative to the hangar when the hangar radar fails to detect the UAV position, that is, if the UAV monocular vision sensor, the hangar monocular vision sensor, and the hangar radar all fail to detect the corresponding results, the three-direction speed commands ( x, y, z) are zero, and the angular velocity command is zero.

[0148] In some embodiments, the UAV state machine outputs a UAV control instruction (i.e., any one of the first instruction, the second instruction, and the third instruction), including three-direction speed commands x, y, z and the heading angular velocity command A flight control program for a drone, and the drone operates according to the specified command.

[0149] Through this embodiment, the mechanism of the third command is introduced, so that when the monocular vision sensor at the hangar end is also limited and cannot provide the second pose, the drone state machine can still generate the third command for controlling the safe landing of the drone based on the drone position information obtained by the hangar radar, greatly enhancing the environmental adaptability and robustness of the drone state machine, ensuring that even without the precise second pose, it can guide the drone to land in a safe manner, thereby avoiding landing failures or accidents caused by the lack of visual information.

[0150] According to another aspect of the embodiments of the present application, there is also provided a landing control method for a drone. This landing control method for a drone can be applied to any of the drone landing systems in the foregoing embodiments (for example, applied to the drone in the drone landing system), and those that have been described will not be elaborated here.

[0151] As an alternative implementation manner, the landing control method for the drone in the embodiments of the present application can be executed by the drone 102. Figure 6 It is a schematic flowchart of an alternative landing control method for a drone according to the embodiments of the present application, as Figure 6 shown, the process of this method may include the following steps:

[0152] Step S602, when the drone descends to a preset height range, capture the hangar two-dimensional code set on the hangar through the drone's monocular vision sensor on the drone to obtain a first monocular image, wherein the hangar two-dimensional code is set with information of the landing point, and the landing point refers to the landing position of the drone on the hangar.

[0153] Step S604, determine the first pose of the drone's monocular vision sensor relative to the landing point according to the first monocular image, and transmit the first pose to the drone state machine.

[0154] Step S606, obtain the second pose transmitted by the monocular vision sensor on the hangar, wherein the second pose is determined according to the second monocular image obtained by the monocular vision sensor on the hangar capturing an object within a preset viewing angle, and is the pose of the marker on the drone relative to the landing point.

[0155] Step S608, obtain the position of the drone transmitted by the hangar radar on the hangar, wherein the position of the drone is the position obtained by tracking the drone based on the object point cloud data obtained by the hangar radar scanning an object within a preset scattering cross-section.

[0156] Step S610: Generate a control command according to the first pose, the UAV position, and the second pose through the UAV state machine, and control the UAV to land at the landing point according to the control command.

[0157] Through this embodiment, on the premise of retaining the monocular vision sensor of the UAV, by adding a hangar radar and a hangar monocular vision sensor at the hangar end, the tracking and guided landing of the UAV in bad weather are realized. At a short distance, the hangar monocular vision sensor is used to accurately guide the UAV to land precisely. Compared with realizing precise landing only relying on the monocular vision sensor of the UAV, the landing robustness is improved, and the success rate of precise landing in extreme weather is greatly enhanced.

[0158] It should be noted that for the foregoing method embodiments, for the sake of simple description, they are all expressed as a series of action combinations. However, those skilled in the art should know that this application is not limited by the described action sequence, because according to this application, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0159] Through the description of the above embodiments, those skilled in the art can clearly understand that the method according to the above embodiments can be implemented by means of software plus a necessary general hardware platform. Of course, it can also be implemented by hardware, but in many cases, the former is a better implementation method. Based on such an understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM (Read-Only Memory), RAM (Random Access Memory), magnetic disk, optical disk), and includes several instructions to enable a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods described in various embodiments of this application.

[0160] According to another aspect of the embodiments of this application, a UAV is further provided. This UAV can be applied to any UAV landing system in the foregoing embodiments, and those that have been described will not be elaborated here.

[0161] As an optional embodiment, the UAV includes: a UAV monocular vision sensor and a processor, and a UAV state machine runs on the processor. Among them,

[0162] The drone monocular vision sensor is used to capture the hangar QR code set on the hangar when the drone descends to a preset height range, obtaining a first monocular image. The hangar QR code is set with information about the landing point, where the landing point refers to the landing position of the drone on the hangar. According to the first monocular image, determine the first pose of the drone monocular vision sensor relative to the landing point, and transmit the first pose to the drone state machine.

[0163] The drone state machine is used to obtain the second pose transmitted by the hangar monocular vision sensor on the hangar. The second pose is determined based on the second monocular image captured by the hangar monocular vision sensor of an object within a preset viewing angle, which is the pose of the marker on the drone relative to the landing point. Obtain the drone position transmitted by the hangar radar on the hangar, where the drone position is obtained by tracking the drone based on the object point cloud data obtained by the hangar radar scanning objects within a preset scattering cross-section. Generate a control command according to the first pose, the drone position, and the second pose, and control the drone to land at the landing point according to the control command.

[0164] It should be noted that the above-mentioned various modules can be implemented by software or hardware. For the latter, it can be implemented in the following ways, but not limited to: the above-mentioned modules are all located in the same processor; or, the above-mentioned various modules are respectively located in different processors in any combination form.

[0165] According to another aspect of the embodiments of the present application, a computer-readable storage medium is provided. The computer-readable storage medium includes a stored program, where the program, when running, executes the steps in any one of the above method embodiments.

[0166] In an exemplary embodiment, the above computer-readable storage medium may include, but is not limited to: various media such as USB flash drives, ROMs, RAMs, mobile hard disks, magnetic disks, or optical discs that can store computer programs.

[0167] According to another aspect of the embodiments of the present application, an electronic device is provided, including a memory, a processor, and a computer program stored on the memory and executable on the processor. The processor is configured to execute the steps in any one of the above method embodiments through the computer program. In an exemplary embodiment, the above electronic device may further include a transmission device and an input / output device, where the transmission device is connected to the above processor, and the input / output device is connected to the above processor.

[0168] The specific examples in this embodiment may refer to the examples described in the above embodiments and exemplary embodiments, and will not be elaborated here.

[0169] According to another aspect of the embodiments of the present application, a computer program product is further provided. The computer program product includes computer programs / instructions, and the computer programs / instructions contain program codes for executing the methods shown in the flowcharts. In such an embodiment, the computer program can be downloaded and installed from a network through the communication part 709 and / or installed from the removable medium 711. When the computer program is executed by the central processing unit 701, various functions provided by the embodiments of the present application are executed. The serial numbers of the embodiments of the present application above are only for description and do not represent the advantages and disadvantages of the embodiments.

[0170] Figure 7 Schematically shown is a block diagram of a computer system of an electronic device for implementing the embodiments of the present application. As Figure 7 shown, the computer system 700 includes a CPU (Central Processing Unit), a central processing unit 701, which can execute various appropriate actions and processes according to the programs stored in the ROM 702 or the programs loaded into the RAM 703 from the storage part 708. In the random access memory 703, various programs and data required for system operation are also stored. The central processing unit 701, the read-only memory 702, and the random access memory 703 are connected to each other through the bus 704. The I / O (Input / Output) interface 705 is also connected to the bus 704.

[0171] The following components are connected to the I / O interface 705: an input part 706 including a keyboard, a mouse, etc.; an output part 707 including, for example, a CRT (Cathode Ray Tube), an LCD (Liquid Crystal Display), etc. and speakers, etc.; a storage part 708 including a hard disk, etc.; and a communication part 709 including a network interface card such as a local area network card, a modem, etc. The communication part 709 performs communication processing via a network such as the Internet. The drive 710 is also connected to the input / output interface 705 as required. A removable medium 711, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 710 as required so that the computer program read from it can be installed into the storage part 708 as required.

[0172] In particular, according to an embodiment of the present application, the processes described in each method flowchart can be implemented as computer software programs. For example, an embodiment of the present application includes a computer program product that includes a computer program carried on a computer-readable medium, and the computer program includes program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication section 709 and / or installed from a removable medium 711. When the computer program is executed by a central processing unit 701, various functions defined in the system of the present application are executed.

[0173] It should be noted that Figure 7 The computer system 700 of the electronic device shown is only an example and should not impose any limitations on the functions and scope of use of the embodiments of the present application.

[0174] Obviously, those skilled in the art should understand that the above-mentioned modules or steps of the present application can be implemented by a general-purpose computing device. They can be concentrated on a single computing device or distributed on a network composed of multiple computing devices. They can be implemented by program code executable by the computing device. Thus, they can be stored in a storage device and executed by the computing device. And in some cases, the steps shown or described can be executed in a different order from here, or they can be separately fabricated into individual integrated circuit modules, or multiple modules or steps among them can be fabricated into a single integrated circuit module to implement. In this way, the present application is not limited to any specific combination of hardware and software.

[0175] The above are only the preferred embodiments of the present application and are not used to limit the present application. For those skilled in the art, the present application can have various changes and modifications. Any modification, equivalent replacement, improvement, etc. made within the principle of the present application shall be included in the protection scope of the present application.

Claims

1. A drone landing system, characterized in that, Including: A drone, a hangar monocular vision sensor, and a hangar radar. The drone includes a drone monocular vision sensor and a processor, and a drone state machine runs on the processor. The hangar monocular vision sensor and the hangar radar are installed on the hangar. Among them, The drone monocular vision sensor is configured to, when the drone descends to a preset height range, capture a hangar QR code set on the hangar to obtain a first monocular image. Among them, the hangar QR code includes multiple QR codes; the sizes of different QR codes among the multiple QR codes are different; a landing point is set on the hangar QR code, and the landing point refers to the landing position of the drone on the hangar; according to the first monocular image, determine a first pose of the drone monocular vision sensor relative to the landing point, and transmit the first pose to the drone state machine; The hangar monocular vision sensor is configured to capture an object within a preset viewing angle to obtain a second monocular image. Among them, the second monocular image includes the hangar QR code and a marker on the drone; according to the second monocular image, determine a second pose of the marker on the drone relative to the landing point, and transmit the second pose to the drone state machine; the hangar monocular vision sensor is further configured to perform marker recognition on the drone in the second monocular image to obtain corner points of the marker; according to the corner points of the marker, calculate a fourth pose of the marker relative to the hangar monocular vision sensor; The hangar monocular vision sensor is further configured to perform marker recognition on the drone in the second monocular image to obtain a marker area, perform histogram statistics on the marker area to obtain a median gray value; the hangar monocular vision sensor is further configured to perform grid division on the marker area to obtain multiple grids, determine at least one grid that meets a first preset condition among the multiple grids as a group of first grids, and determine at least one grid that does not meet the preset condition among the multiple grids as a group of second grids; the first preset condition refers to the condition that the pixel difference between the maximum pixel value and the minimum pixel value in the grid is greater than a first pixel threshold, the pixel value of each pixel point in the grid is greater than a second pixel threshold, and the pixel value of each pixel point in the grid is greater than the median gray value; the hangar monocular vision sensor is further configured to modify the pixel value of each pixel point in each grid of the group of first grids to a first pixel value, and modify the pixel value of each pixel point in each grid of the group of second grids to a second pixel value; the hangar monocular vision sensor is further configured to perform gradient clustering on the group of first grids with modified pixel values and the group of second grids with modified pixel values to obtain multiple first clustering points, calculate the variance of the pixel points within a preset area of each first clustering point among the multiple first clustering points to obtain the variance corresponding to each first clustering point, and select a preset number of first clustering points in descending order of variance as the corner points of the marker; The hangar monocular vision sensor is also used to identify a specified two-dimensional code in the second monocular image to obtain the corner points of the specified two-dimensional code; calculate the fifth pose of the hangar monocular vision sensor relative to the specified two-dimensional code according to the corner points of the specified two-dimensional code; compensate the fifth pose according to the position difference between the landing point and the center point of the hangar two-dimensional code to obtain the sixth pose of the hangar monocular vision sensor relative to the landing point; the specified two-dimensional code is the two-dimensional code located at the center of the hangar two-dimensional code among the multiple two-dimensional codes; determine the second pose according to the fourth pose and the sixth pose. The hangar radar is used to scan objects within a preset scattering cross-section multiple times to obtain multiple sets of object point cloud data; track the drone based on the multiple sets of object point cloud data to obtain the drone position; and transmit the drone position to the drone state machine. The drone state machine is used to generate a control command according to the first pose, the drone position, and the second pose, and control the drone to land at the landing point according to the control command.

2. The system according to claim 1, characterized in that, The hangar two-dimensional code includes multiple two-dimensional codes; the sizes of different two-dimensional codes among the multiple two-dimensional codes are different from each other. The drone monocular vision sensor is also used to perform two-dimensional code recognition on the first monocular image and determine a target two-dimensional code from a group of recognized two-dimensional codes. The target two-dimensional code refers to the two-dimensional code with the smallest Hamming distance among the group of two-dimensional codes; calculate the third pose of the drone monocular vision sensor relative to the target two-dimensional code, and compensate the third pose according to the position difference between the landing point and the center point of the hangar two-dimensional code to obtain the first pose.

3. The system according to claim 1, wherein The hangar two-dimensional code includes multiple two-dimensional codes; the sizes of different two-dimensional codes among the multiple two-dimensional codes are different from each other; the control command includes a first command; the first command is used to control the drone to land at the landing point. The drone monocular vision sensor is also used to determine the false detection probability of the target two-dimensional code and transmit the false detection probability to the drone state machine; the target two-dimensional code refers to the two-dimensional code with the smallest Hamming distance among a group of two-dimensional codes obtained by the drone monocular vision sensor performing two-dimensional code recognition on the first monocular image; the false detection probability represents the possibility of false detection of the target two-dimensional code. The drone state machine is also used to generate a first command according to the false detection probability and the first pose when the drone monocular vision sensor detects the hangar two-dimensional code, and control the drone to land at the landing point according to the first command.

4. The system according to claim 3, wherein The first monocular image includes the hangar. The drone monocular vision sensor is also used to convert the coordinates of the corner points of the reference two-dimensional code in the hangar two-dimensional code coordinate system to the image pixel coordinates of the drone monocular vision sensor to obtain the corner point pixel coordinates of the reference two-dimensional code in the image pixel coordinates; the reference two-dimensional code refers to any two-dimensional code in the hangar two-dimensional code. The drone monocular vision sensor is further configured to determine whether the target two-dimensional code is within the area range of the hangar in the first monocular image when the corner pixel coordinates of each two-dimensional code in the hangar two-dimensional code are outside the image range of the drone monocular vision sensor; when the target two-dimensional code is within the area range of the hangar in the first monocular image, determining that the false detection probability of the target two-dimensional code is a first probability; the first probability indicates that the target two-dimensional code is credible; when the target two-dimensional code is not within the area range of the hangar in the first monocular image, determining that the false detection probability of the target two-dimensional code is a second probability; the second probability indicates that the target two-dimensional code is not credible.

5. The system according to claim 4, wherein The drone monocular vision sensor is further configured to calculate the absolute value of the coordinate difference between the corner pixel coordinates of the reference two-dimensional code and the detected pixel coordinates of the corner of the reference two-dimensional code when the corner pixel coordinates of the corner of the reference two-dimensional code are within the image range of the drone monocular vision sensor; the detected pixel coordinates of the corner of the reference two-dimensional code refer to the pixel coordinates of the corner of the reference two-dimensional code obtained by the drone monocular vision sensor identifying the reference two-dimensional code from the first monocular image. The drone monocular vision sensor is further configured to determine that the false detection probability of the target two-dimensional code is the first probability when the absolute value is less than a preset absolute value threshold; when the absolute value is greater than or equal to the preset absolute value threshold, performing binarization processing, gradient clustering processing, edge extraction, and convex quadrilateral fitting processing on the pixel points in a preset area of the corner pixel coordinates of the reference two-dimensional code in sequence. The drone monocular vision sensor is further configured to determine that the false detection probability of the target two-dimensional code is the first probability when a fitted convex quadrilateral is obtained; when a fitted convex quadrilateral is not obtained, determining whether the corner pixel coordinates of the corner of the reference two-dimensional code are within the area range of the hangar in the first monocular image; when the corner pixel coordinates of the corner of the reference two-dimensional code are within the area range of the hangar in the first monocular image, determining that the false detection probability of the target two-dimensional code is a third probability; the third probability indicates that there is uncertainty in the target two-dimensional code, and the third probability is greater than the first probability and less than the second probability; when the corner pixel coordinates of the corner of the reference two-dimensional code are not within the area range of the hangar in the first monocular image, determining that the false detection probability of the target two-dimensional code is the second probability.

6. The system according to claim 1, characterized in that, The first pixel threshold is dynamically updated following the recognition coverage rate of the corner of the marker by the hangar monocular vision sensor. The hangar monocular vision sensor is further configured to increase the first pixel threshold by a first step value to obtain the updated first pixel threshold when the recognition coverage rate of the corner points of the marker is greater than or equal to a preset coverage rate threshold; and decrease the first pixel threshold by a second step value to obtain the updated first pixel threshold when the recognition coverage rate of the corner points of the marker is less than the preset coverage rate threshold; wherein, during the process of updating the first pixel threshold, the updated first pixel threshold is always greater than a preset pixel value.

7. The system according to claim 1, wherein The second pixel threshold is dynamically updated following an adaptive parameter that cyclically takes values from multiple parameter values; the adaptive parameter is used to scale the second pixel threshold. The hangar monocular vision sensor is further configured to, according to the recognition result of the corner points of the marker by the hangar monocular vision sensor, select a parameter value that matches the recognition result from the multiple parameter values as the adaptive parameter. The second pixel threshold is calculated according to a preset binarization threshold formula, and the binarization threshold formula represents the correlation relationship among the maximum pixel value, the minimum pixel value and the adaptive parameter in the grid.

8. The system according to claim 1, wherein Each set of object point cloud data in the multiple sets of object point cloud data corresponds to one scan in the multiple scans; the hangar radar is further configured to cluster a set of object point cloud data obtained in each scan in the multiple scans to obtain a set of second clustered points, and determine the conditional probability that each second clustered point in the set of second clustered points recognized in each scan is the UAV. The hangar radar is further configured to, based on the conditional probability that each second clustered point recognized in each scan is the UAV, determine the cumulative probability that each second clustered point recognized by the hangar radar is the UAV, and remove the second clustered points with a cumulative probability less than a first preset probability threshold from the set of second clustered points, and determine the second clustered points with a cumulative probability greater than a second preset probability threshold in the set of second clustered points after removal as the target points of the UAV. The hangar radar is further configured to determine the UAV position according to the target points of the UAV, the pose transformation matrix between the hangar radar and the hangar monocular vision sensor, and the sixth pose of the hangar monocular vision sensor relative to the landing point.

9. The system according to claim 8, characterized in that The hangar radar is further configured to use each scan in the multiple scans as the current scan, and perform the following tracking operation to obtain the conditional probability that the UAV is recognized in each scan: Cluster a set of object point cloud data obtained in the current scan to obtain multiple third clustered points; each third clustered point in the multiple third clustered points includes the polar coordinates of each third clustered point, the Doppler velocity of each third clustered point relative to the hangar radar, and the signal-to-noise ratio of each third clustered point. Select at least one third clustered point from the multiple third clustered points whose Doppler velocity relative to the hangar radar is within a preset velocity range and whose signal-to-noise ratio is within a preset signal-to-noise ratio range to obtain a set of third clustered points. Convert each of the third clustering points from the polar coordinate system to the radar Cartesian coordinate system, and in the radar Cartesian coordinate system, cluster the set of third clustering points to obtain a set of fourth clustering points, where each fourth clustering point in the set of fourth clustering points includes the initial motion state parameters of the drone; According to the initial motion state parameters of each fourth clustering point, predict the future motion state parameters of each fourth clustering point, and fuse the future motion state parameters of each fourth clustering point with the radar Cartesian coordinates of each fourth clustering point to obtain each updated fourth clustering point; Statistically calculate the Doppler velocities of all third clustering points within a preset range of each updated fourth clustering point relative to the hangar radar to obtain the maximum Doppler velocity, minimum Doppler velocity, and Doppler velocity distribution variance corresponding to each fourth clustering point; determine the conditional probability that each fourth clustering point is a drone in the current scan according to the maximum Doppler velocity, minimum Doppler velocity, and Doppler velocity distribution variance corresponding to each fourth clustering point.

10. The system according to claim 9, wherein, The hangar radar is further configured to, when each fourth clustering point meets the second preset condition, determine that the conditional probability that each fourth clustering point is a drone in the current scan is the first probability; when each fourth clustering point does not meet the second preset condition, determine that the conditional probability that each fourth clustering point is a drone in the current scan is the second probability; where the second preset condition refers to the condition that the velocity difference between the maximum Doppler velocity and the minimum Doppler velocity corresponding to each fourth clustering point is greater than a preset velocity threshold, and the Doppler velocity distribution variance is greater than a preset variance threshold.

11. The system according to claim 1, characterized in that, The control instruction includes a second instruction; the second instruction is used to control the drone to land at the landing point; The drone state machine is further configured to, when the drone monocular vision sensor cannot detect the hangar QR code, generate the second instruction according to the second pose, and control the drone to land at the landing point according to the second instruction.

12. The system according to claim 1, characterized in that, The control instruction includes a third instruction; the third instruction is used to control the drone to land at the landing point; The drone state machine is further configured to, when the hangar monocular vision sensor cannot detect the second pose, generate the third instruction according to the drone position, and control the drone to land at the landing point according to the third instruction.

13. The system according to claim 1, wherein, The drone state machine is further configured to, when the hangar radar cannot detect the drone position, control the drone to be stationary relative to the hangar.

14. A landing control method for an unmanned aerial vehicle, characterized in that, Including: When the drone descends to a preset altitude range, the drone's monocular vision sensor on the drone captures the hangar QR code set on the hangar to obtain a first monocular image, where the hangar QR code includes multiple QR codes; the sizes of different QR codes among the multiple QR codes are different; the hangar QR code is provided with information about the landing point, and the landing point refers to the landing position of the drone on the hangar. According to the first monocular image, determine the first pose of the drone's monocular vision sensor relative to the landing point, and transmit the first pose to the drone state machine. Obtain the second pose transmitted by the hangar's monocular vision sensor, where the second pose is the pose of the marker on the drone relative to the landing point determined according to the second monocular image obtained by the hangar's monocular vision sensor capturing an object within a preset viewing angle; the hangar's monocular vision sensor identifies the marker on the drone in the second monocular image to obtain the corner points of the marker; according to the corner points of the marker, calculate the fourth pose of the marker relative to the hangar's monocular vision sensor. The hangar's monocular vision sensor identifies the marker on the drone in the second monocular image to obtain the marker area, performs a histogram statistics on the marker area to obtain the median gray value; the hangar's monocular vision sensor divides the marker area into multiple grids, determines at least one grid that meets the first preset condition among the multiple grids as a group of first grids, and determines at least one grid that does not meet the preset condition among the multiple grids as a group of second grids; the first preset condition refers to the condition that the pixel difference between the maximum pixel value and the minimum pixel value in the grid is greater than the first pixel threshold, the pixel value of each pixel point in the grid is greater than the second pixel threshold, and the pixel value of each pixel point in the grid is greater than the median gray value; the hangar's monocular vision sensor modifies the pixel value of each pixel point in each grid of the group of first grids to the first pixel value, and modifies the pixel value of each pixel point in each grid of the group of second grids to the second pixel value; the hangar's monocular vision sensor performs gradient clustering on the group of first grids with modified pixel values and the group of second grids with modified pixel values to obtain multiple first clustering points, calculates the variance of the pixel points within the preset area of each first clustering point among the multiple first clustering points to obtain the variance corresponding to each first clustering point, and selects a preset number of first clustering points in descending order of variance as the corner points of the marker. Identify the specified two-dimensional code in the second monocular image through the hangar monocular vision sensor to obtain the corner points of the specified two-dimensional code; calculate the fifth pose of the hangar monocular vision sensor relative to the specified two-dimensional code according to the corner points of the specified two-dimensional code; compensate the fifth pose according to the position difference between the landing point and the center point of the hangar two-dimensional code to obtain the sixth pose of the hangar monocular vision sensor relative to the landing point; the specified two-dimensional code is the two-dimensional code located at the center of the hangar two-dimensional code among the multiple two-dimensional codes; determine the second pose according to the fourth pose and the sixth pose; Obtain the position of the unmanned aerial vehicle transmitted by the hangar radar on the hangar, where the position of the unmanned aerial vehicle is obtained by tracking the unmanned aerial vehicle based on the object point cloud data obtained by the hangar radar scanning the objects within a preset scattering cross-section; Generate a control command through the unmanned aerial vehicle state machine according to the first pose, the position of the unmanned aerial vehicle, and the second pose, and control the unmanned aerial vehicle to land at the landing point according to the control command.

15. A drone, characterized in that, Comprising: An unmanned aerial vehicle monocular vision sensor and a processor, on which an unmanned aerial vehicle state machine runs, where The unmanned aerial vehicle monocular vision sensor is used to capture the hangar two-dimensional code set on the hangar to obtain a first monocular image when the unmanned aerial vehicle descends to a preset height range, where the hangar two-dimensional code includes multiple two-dimensional codes; the sizes of different two-dimensional codes among the multiple two-dimensional codes are different; the information of the landing point is set on the hangar two-dimensional code, and the landing point refers to the landing position of the unmanned aerial vehicle on the hangar; determine the first pose of the unmanned aerial vehicle monocular vision sensor relative to the landing point according to the first monocular image, and transmit the first pose to the unmanned aerial vehicle state machine; The unmanned aerial vehicle state machine is used to obtain the second pose transmitted by the hangar monocular vision sensor on the hangar, where the second pose is the pose of the marker on the unmanned aerial vehicle relative to the landing point determined according to the second monocular image obtained by the hangar monocular vision sensor capturing the objects within a preset viewing angle; obtain the position of the unmanned aerial vehicle transmitted by the hangar radar on the hangar, where the position of the unmanned aerial vehicle is obtained by tracking the unmanned aerial vehicle based on the object point cloud data obtained by the hangar radar scanning the objects within a preset scattering cross-section; generate a control command according to the first pose, the position of the unmanned aerial vehicle, and the second pose, and control the unmanned aerial vehicle to land at the landing point according to the control command; Among them, the second pose is obtained by the hangar monocular vision sensor performing the following operations: identifying the markers on the UAV in the second monocular image to obtain the corner points of the markers; calculating the fourth pose of the markers relative to the hangar monocular vision sensor according to the corner points of the markers; identifying the marker area in the second monocular image, performing histogram statistics on the marker area to obtain the median gray value; dividing the marker area into grids to obtain a plurality of grids, determining at least one grid that meets the first preset condition among the plurality of grids as a group of first grids, and determining at least one grid that does not meet the preset condition among the plurality of grids as a group of second grids; the first preset condition refers to the condition that the pixel difference between the maximum pixel value and the minimum pixel value in the grid is greater than the first pixel threshold, the pixel value of each pixel point in the grid is greater than the second pixel threshold, and the pixel value of each pixel point in the grid is greater than the median gray value; modifying the pixel value of each pixel point in each grid of the group of first grids to the first pixel value, and modifying the pixel value of each pixel point in each grid of the group of second grids to the second pixel value; performing gradient clustering on the group of first grids with modified pixel values and the group of second grids with modified pixel values to obtain a plurality of first clustering points, calculating the variance of the pixel points in the preset area of each first clustering point among the plurality of first clustering points to obtain the variance corresponding to each first clustering point, and selecting a preset number of first clustering points in descending order of variance as the corner points of the markers; identifying the specified QR code in the second monocular image to obtain the corner points of the specified QR code; calculating the fifth pose of the hangar monocular vision sensor relative to the specified QR code according to the corner points of the specified QR code; compensating the fifth pose according to the position difference between the landing point and the center point of the hangar QR code to obtain the sixth pose of the hangar monocular vision sensor relative to the landing point; the specified QR code is the QR code located at the center of the hangar QR code among the plurality of QR codes; determining the second pose according to the fourth pose and the sixth pose.

Images