Unmanned aerial vehicle accurate landing method based on image recognition

Abstract

The invention relates to the technical field of unmanned aerial vehicle control, and discloses an unmanned aerial vehicle accurate landing method based on image recognition, which comprises the following steps: an unmanned aerial vehicle receives a guide signal or a GPS coordinate from a landing platform, and flies to the sky of a landing area; the unmanned aerial vehicle vertically and downwards shoots an image containing a preset landing mark through an airborne camera; preprocessing and feature enhancement are carried out on the shot image, then the image is input to a target detection network, and a center pixel coordinate of a landing mark and contour information of the landing mark in the image are identified; and combining flight height sensor data of the unmanned aerial vehicle according to the corresponding relationship between the pixel size of the identified landing mark and the real-world size of the landing mark. The method is switched to a high-frame-rate image acquisition mode in a short-distance landing stage, realizes sub-centimeter-level position alignment precision through real-time visual servo fine adjustment, effectively eliminates accumulated deviation before final grounding, and provides key space alignment guarantee for stable and accurate landing of the unmanned aerial vehicle.

Description

Technical Field

[0001] This invention relates to the field of unmanned aerial vehicle (UAV) control technology, and in particular to a method for precise landing of UAVs based on image recognition. Background Technology

[0002] Existing autonomous landing solutions for drones, whether based on high-precision GPS, visual recognition, or infrared / UWB sensing technologies, typically complete their control loop just before the drone touches the ground. Once the landing gear contacts the ground, the control system usually switches to "shutdown" or simply maintains its attitude. However, actual landing surfaces (such as moving drone decks, uneven terrain, or mobile platforms with cushioning) often possess dynamic, non-rigid, or unknown stiffness characteristics. Relying solely on precise alignment before touchdown cannot solve the problems of drone imbalance, bouncing, or even tipping over due to surface swaying, tilting, or material elasticity at the moment of impact.

[0003] The existing patent CN109911231A, although employing a magnetic landing gear to enhance adsorption, is passive and dependent on a specific iron platform, and cannot actively adapt to ground impact in general scenarios. Summary of the Invention

[0004] This invention provides a precise landing method for unmanned aerial vehicles (UAVs) based on image recognition to solve the existing technical problems, thereby addressing the issues mentioned in the background section.

[0005] To address the aforementioned technical problems, according to one aspect of the present invention, more specifically, a method for precise landing of unmanned aerial vehicles based on image recognition, comprising the following steps: S1. The drone receives guidance signals or GPS coordinates from the landing platform and flies to the airspace above the landing area; S2. The UAV takes a vertically downward image containing a preset landing mark using its onboard camera. After preprocessing and feature enhancement of the captured image, it is input into the target detection network to identify the center pixel coordinates of the landing mark and its contour information in the image. S3. Based on the correspondence between the pixel size of the identified landing marker and its actual world size, and combined with the flight altitude sensor data of the UAV, calculate the horizontal position deviation and altitude value of the UAV relative to the center of the landing marker; control the UAV to reduce the horizontal position deviation and descend to the first preset altitude; S4. After the drone descends to the first preset altitude, switch to a higher frame rate image acquisition mode to perform real-time position fine-tuning to ensure that the drone's landing gear projection coincides with the center of the landing mark; continue descending; S5. At the moment the UAV landing gear contacts the landing plane and throughout the entire touchdown process, the pressure distribution and rate of change of each support point are monitored in real time by an array of pressure sensors installed on the landing gear. Based on real-time pressure data, the thrust output of each rotor and the overall attitude angle of the UAV are dynamically adjusted through an adaptive control algorithm based on mechanical feedback in order to balance the load, absorb the impact, and ultimately achieve a smooth, bounce-free, and precise landing.

[0006] Furthermore, in step S1, the guidance signal is an infrared beacon signal, a UWB ultra-wideband signal, or precise coordinate information transmitted from the landing platform via a 5G network; the GPS coordinates are the landing platform's own GPS coordinates transmitted in real time via a data link.

[0007] Furthermore, in step S2, the image undergoes preprocessing and feature enhancement, specifically including: S201. Perform grayscale conversion and adaptive contrast enhancement on the image; S202. An image enhancement algorithm based on spatiotemporal information is adopted. This algorithm suppresses dynamic background noise and enhances the stability features of the landing marker by fusing image features of the current frame and historical frames. Its core calculation formula is as follows: ; in, Indicates at time ,Location Enhanced image feature values; Represents the original image features of the current frame; For historical frames A weighted mask based on motion estimation is used to distinguish between the background and the foreground. The differences in characteristics between the past and the present; , For fusion weighting coefficients; It is the attenuation factor; Historical data to be considered.

[0008] Furthermore, the target detection network is a lightweight improved convolutional neural network, whose input is the enhanced image features. The output is the bounding box, center coordinates, and vertex coordinates of the outline polygon of the landing mark.

[0009] Furthermore, in step S3, the first preset altitude is 0.5 meters to 2 meters above the landing plane; when the drone's altitude is lower than the first preset altitude, an auxiliary ranging sensor is activated for altitude fusion, and the auxiliary ranging sensor includes an ultrasonic sensor, a lidar, or an infrared ranging module.

[0010] Furthermore, in step S5, the adaptive control algorithm based on mechanical feedback performs the following operations: S501, Real-time acquisition of the pressure value of each sensing unit in the pressure sensing array. ,in , This represents the total number of sensing units; S502, Calculate the pressure distribution unevenness index and overall impact force change rate ; S503, Based on the imbalance index and rate of change of impact force By combining the current attitude angle of the UAV, a pre-trained control strategy model or fuzzy PID controller is used to generate thrust adjustment amounts for each rotor. and attitude angle adjustment commands , ; S504, Execute the thrust adjustment amount. The attitude angle adjustment command is used to make the pressure distribution more even and smooth the rate of change of impact force until the pressure values ​​of all sensing units stabilize within the set threshold range and the drone is completely stationary.

[0011] Furthermore, in step S502, the pressure distribution unevenness index is calculated. and overall impact force change rate The specific formula is: ; In the above formula, for The average pressure at which all sensor units are etched.

[0012] Furthermore, in step S503, the core thrust adjustment amount... The calculation formula is: ; In the above formula, Rotor index; , , To control the gain; , , A nonlinear mapping function designed based on the UAV dynamics model and pressure distribution model.

[0013] Furthermore, the control strategy model is trained using a deep reinforcement learning algorithm in a simulation environment containing different landing plane stiffness, slope, and wind disturbances. Its reward function is designed to encourage pressure equilibrium, low impact force change rate, and rapid stabilization.

[0014] This invention provides a precise landing method for unmanned aerial vehicles (UAVs) based on image recognition. Compared with existing technologies, this method achieves the following advantages: 1. This invention adopts a multi-source fusion guidance method, combining multiple signals such as infrared beacons, UWB, 5G and GPS to work together, which significantly improves the guidance reliability and system robustness of UAVs in complex electromagnetic environments or when a single signal fails, ensuring that UAVs can stably and accurately enter the visual recognition range, laying a solid foundation for subsequent precise landing.

[0015] 2. This invention introduces an image enhancement algorithm based on spatiotemporal information and a lightweight convolutional neural network, which effectively suppresses dynamic background interference and enhances the stability of landing markers. While maintaining a low onboard computing load, it significantly improves the target recognition accuracy and real-time performance of UAVs under different lighting conditions, partial occlusion, or motion blur environments.

[0016] 3. By integrating visual recognition results with UAV altitude sensor data, this invention enables autonomous relative navigation and position calculation without relying on external high-precision positioning equipment, reducing the overall system cost and complexity, while improving the UAV's landing adaptability and autonomy in environments with differential GPS or weak signals.

[0017] 4. This invention switches to a high frame rate image acquisition mode during the close-range landing phase and achieves sub-centimeter-level position alignment accuracy through real-time visual servo fine-tuning, effectively eliminating the accumulated deviation before final touchdown and providing key spatial alignment assurance for the smooth and accurate landing of the UAV.

[0018] 5. This invention introduces a mechanical feedback and adaptive control algorithm based on a pressure sensor array during the ground contact phase, enabling the UAV to actively sense and dynamically adapt to landing planes with different stiffness, slope and dynamic characteristics, effectively suppressing landing impact, preventing the aircraft from bouncing and rolling over, and realizing closed-loop precise control of the entire process from air contact to stable landing on the ground, significantly improving landing safety and adaptability. Attached Figure Description

[0019] Figure 1 This is a flowchart of the present invention. Detailed Implementation

[0020] To make the technical solution of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0021] like Figure 1 As shown, according to one aspect of the present invention, a method for precise landing of a drone based on image recognition is provided, comprising the following steps: Step 1: The UAV receives guidance signals or GPS coordinates from the landing platform and flies to the airspace above the landing area. The guidance signals are infrared beacon signals, UWB ultra-wideband signals, or precise coordinate information transmitted from the 5G network by the landing platform. The GPS coordinates are the landing platform's own GPS coordinates transmitted in real time via data link.

[0022] Employing a multi-source fusion guidance method, the UAV can receive various signals such as infrared beacons, UWB, 5G, or GPS to achieve initial guidance from a distance to the landing area. The principle behind this is to obtain the coordinates or characteristic signals of the landing platform in real time through a data link, and combine this with the UAV's own navigation system for route planning and tracking. The advantage of this design is that it improves the robustness and reliability of the system in complex electromagnetic environments or when a single signal fails, ensuring that the UAV can initially and stably enter the visual recognition range.

[0023] Step 2: The drone uses its onboard camera to capture an image containing a pre-set landing marker vertically downwards; after preprocessing and feature enhancement, the captured image is input into a target detection network to identify the center pixel coordinates of the landing marker and its contour information in the image; the image preprocessing and feature enhancement specifically include: S201. Perform grayscale conversion and adaptive contrast enhancement on the image; S202. An image enhancement algorithm based on spatiotemporal information is adopted. This algorithm suppresses dynamic background noise and enhances the stability features of the landing marker by fusing image features of the current frame and historical frames. Its core calculation formula is as follows: ; in, Indicates at time ,Location Enhanced image feature values; Represents the original image features of the current frame; For historical frames A weighted mask based on motion estimation is used to distinguish between the background and the foreground. The differences in characteristics between the past and the present; , For fusion weighting coefficients; It is the attenuation factor; Historical data to be considered.

[0024] The object detection network is a lightweight, improved convolutional neural network, whose input is the enhanced image features. The output is the bounding box, center coordinates, and vertex coordinates of the outline polygon of the landing mark.

[0025] An image enhancement algorithm based on spatiotemporal information and a lightweight target detection network are introduced. The principle is to suppress dynamic background interference by fusing features of the current frame and historical frames, and to use a lightweight convolutional neural network to identify the outline and center of the landing mark in real time. Its advantage is that while maintaining low computational load, it significantly improves the recognition accuracy and stability under conditions such as changes in lighting, partial occlusion or motion blur.

[0026] Step 3: Based on the correspondence between the pixel size of the identified landing marker and its actual world size, and combined with the data from the drone's flight altitude sensor, calculate the horizontal position deviation and altitude value of the drone relative to the center of the landing marker; control the drone to reduce the horizontal position deviation and descend to the first preset altitude; The first preset altitude is 0.5 meters to 2 meters above the landing plane; when the drone's altitude is lower than the first preset altitude, an auxiliary ranging sensor is activated for altitude fusion. The auxiliary ranging sensor includes an ultrasonic sensor, a lidar, or an infrared ranging module.

[0027] By combining visual recognition results with altitude sensor data, the relative position and altitude between the UAV and the landing marker are calculated in real time. The principle is to calculate the horizontal deviation and altitude through a geometric projection model based on the calibration relationship between the pixel size and the actual size of the marker, and control the UAV to make gradual corrections. The advantage of this method is that it enables autonomous relative navigation without relying on high-precision external positioning equipment, reducing system cost and complexity.

[0028] Step 4: After the drone descends to the first preset altitude, switch to a higher frame rate image acquisition mode to perform real-time position fine-tuning, ensuring that the drone's landing gear projection coincides with the center of the landing mark; continue descending; During the close-range landing phase, the system switches to a high frame rate image acquisition mode for real-time visual servo fine-tuning. The principle behind this is to increase the sampling frequency to quickly detect and compensate for minute deviations between the UAV and the center of the marker, ensuring that the landing gear projection coincides with the center of the marker. Its advantage lies in achieving sub-centimeter-level position alignment before final landing, providing a precise spatial alignment basis for a smooth touchdown.

[0029] Step 5: At the moment the UAV landing gear touches the landing plane and throughout the entire touchdown process, the pressure distribution and rate of change at each support point are monitored in real time using an array of pressure sensors installed on the landing gear. Based on real-time pressure data, the thrust output of each rotor and the overall attitude angle of the UAV are dynamically adjusted through an adaptive control algorithm based on mechanical feedback.

[0030] The adaptive control algorithm based on mechanical feedback performs the following operations: S501, Real-time acquisition of the pressure value of each sensing unit in the pressure sensing array. ,in , This represents the total number of sensing units; S502, Calculate the pressure distribution unevenness index and overall impact force change rate The specific formula is as follows: ; In the above formula, for The average pressure used to etch all sensor units; S503, Based on the imbalance index and rate of change of impact force By combining the current attitude angle of the UAV, a pre-trained control strategy model or fuzzy PID controller is used to generate thrust adjustment amounts for each rotor. and attitude angle adjustment commands , Core thrust adjustment amount The calculation formula is: ; In the above formula, Rotor index; , , To control the gain; , , A nonlinear mapping function designed based on the UAV dynamics model and pressure distribution model; S504, Execute thrust adjustment amount The attitude angle adjustment command is used to make the pressure distribution more even and smooth the rate of change of impact force until the pressure values ​​of all sensing units stabilize within the set threshold range and the drone is completely stationary.

[0031] The control strategy model is trained using a deep reinforcement learning algorithm in a simulation environment with different landing plane stiffness, slope and wind disturbances. Its reward function is designed to encourage pressure equilibrium, low impact force change rate and rapid stabilization.

[0032] During the touchdown phase, mechanical feedback adaptive control based on a pressure sensor array is introduced. The principle is to dynamically adjust the thrust of each rotor and the attitude of the airframe by real-time monitoring of the pressure distribution and rate of change at each pivot point, combined with a pre-trained control strategy or fuzzy PID algorithm. Its advantage is that it can actively adapt to landing planes with different stiffness, slope and dynamic characteristics, effectively suppress landing impact, prevent bouncing and rollover, and achieve a truly smooth and precise landing throughout the entire process.

[0033] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention. Therefore, the scope of protection of this patent should be determined by the appended claims.

Claims

1. An image recognition-based precise landing method for a UAV, characterized in that, Includes the following steps: S1. The drone receives guidance signals or GPS coordinates from the landing platform and flies to the airspace above the landing area; S2. The UAV takes a vertically downward image containing a preset landing mark using its onboard camera. After preprocessing and feature enhancement of the captured image, it is input into the target detection network to identify the center pixel coordinates of the landing mark and its contour information in the image. S3. Based on the correspondence between the pixel size of the identified landing marker and its actual world size, and combined with the flight altitude sensor data of the UAV, calculate the horizontal position deviation and altitude value of the UAV relative to the center of the landing marker; control the UAV to reduce the horizontal position deviation and descend to the first preset altitude; S4. After the drone descends to the first preset altitude, switch to a higher frame rate image acquisition mode to perform real-time position fine-tuning to ensure that the drone's landing gear projection coincides with the center of the landing mark; continue descending; S5. At the moment the UAV landing gear contacts the landing plane and throughout the entire touchdown process, the pressure distribution and rate of change of each support point are monitored in real time by an array of pressure sensors installed on the landing gear. Based on real-time pressure data, the thrust output of each rotor and the overall attitude angle of the UAV are dynamically adjusted through an adaptive control algorithm based on mechanical feedback.

2. The image recognition-based precise landing method for unmanned aerial vehicles according to claim 1, characterized in that: In step S1, the guidance signal is an infrared beacon signal, a UWB ultra-wideband signal, or precise coordinate information transmitted from the landing platform via a 5G network; the GPS coordinates are the landing platform's own GPS coordinates transmitted in real time via a data link.

3. The image recognition-based precise landing method for unmanned aerial vehicles according to claim 1, characterized in that: In step S2, the image is preprocessed and its features are enhanced, specifically including: S201. Perform grayscale conversion and adaptive contrast enhancement on the image; S202. An image enhancement algorithm based on spatiotemporal information is adopted. This algorithm suppresses dynamic background noise and enhances the stability features of the landing marker by fusing image features of the current frame and historical frames. Its core calculation formula is as follows: ； in, Indicates at time ,Location Enhanced image feature values; Represents the original image features of the current frame; For historical frames A weighted mask based on motion estimation is used to distinguish between the background and the foreground. The differences in characteristics between the past and the present; , For fusion weighting coefficients; It is the attenuation factor; Historical data to be considered.

4. The image recognition-based precise landing method for unmanned aerial vehicles according to claim 3, characterized in that: The target detection network is a lightweight modified convolutional neural network, and its input is the enhanced image features. The output is the bounding box, center coordinates, and vertex coordinates of the outline polygon of the landing mark.

5. The image recognition-based precise landing method for unmanned aerial vehicles according to claim 1, characterized in that: In step S3, the first preset altitude is 0.5 meters to 2 meters above the landing plane; when the drone's altitude is lower than the first preset altitude, an auxiliary ranging sensor is activated to perform altitude fusion. The auxiliary ranging sensor includes an ultrasonic sensor, a lidar, or an infrared ranging module.

6. The image recognition-based precise landing method for unmanned aerial vehicles according to claim 1, characterized in that: In step S5, the adaptive control algorithm based on mechanical feedback performs the following operations: S501, Real-time acquisition of the pressure value of each sensing unit in the pressure sensing array. ,in , This represents the total number of sensing units; S502, Calculate the pressure distribution unevenness index and overall impact force change rate ; S503, Based on the imbalance index and rate of change of impact force By combining the current attitude angle of the UAV, a pre-trained control strategy model or fuzzy PID controller is used to generate thrust adjustment amounts for each rotor. and attitude angle adjustment commands , ; S504, Execute the thrust adjustment amount. The attitude angle adjustment command is used to make the pressure distribution more even and smooth the rate of change of impact force until the pressure values ​​of all sensing units stabilize within the set threshold range and the drone is completely stationary.

7. The image recognition-based precise landing method for unmanned aerial vehicles according to claim 6, characterized in that: In step S502, the pressure distribution unevenness index is calculated. and overall impact force change rate The specific formula is: ； In the above formula, for The average pressure at which all sensor units are etched.

8. The image recognition-based precise landing method for unmanned aerial vehicles according to claim 6, characterized in that: In step S503, the core thrust adjustment amount The calculation formula is: ； In the above formula, Rotor index; , , To control the gain; , , A nonlinear mapping function designed based on the UAV dynamics model and pressure distribution model.

9. The image recognition-based precise landing method for unmanned aerial vehicles according to claim 6, characterized in that: The control strategy model is trained using a deep reinforcement learning algorithm in a simulation environment with different landing plane stiffness, slope, and wind disturbances. Its reward function is designed to encourage pressure equilibrium, low impact force change rate, and rapid stabilization.

Images