A Dynamic Obstacle Avoidance and Automatic Landing Optimization System for Unmanned Aerial Vehicles in Multiple Scenarios

Abstract

This invention discloses a dynamic obstacle avoidance and automatic landing optimization system for unmanned aerial vehicles (UAVs) in multiple scenarios and across the entire domain, relating to the fields of UAV defense and environmental perception technology. The method includes: controlling a scanning UAV equipped with a camera array to collect spatial and ground images, simultaneously acquiring attitude and positioning data; a processing center preprocessing and seamlessly stitching the images to generate a panoramic image of the entire environment; fusing multi-source monitoring data to form a complete dataset covering both indoor and outdoor environments; constructing a multi-index evaluation model to identify safe, landable areas through semantic segmentation and target detection; transmitting environmental data to a receiving device, guiding it to automatically aim and send appropriate landing commands. This invention achieves 360-degree, all-around environmental detection without blind spots, expands the coverage of landable areas, and improves the accuracy of safe area identification and the coordination between scanning and landing.

Description

Technical Field

[0001] This invention relates to the field of drone defense and environmental perception technology, specifically to a dynamic obstacle avoidance and automatic landing optimization system for drones in all domains and multiple scenarios. It is applicable to the detection of the entire environment and the identification of safe landing areas during the forced landing of unregistered drones, providing accurate environmental data support for forced landing operations. Background Technology

[0002] With the rapid development of drone technology, drones are increasingly widely used in civilian and commercial fields. However, the disorderly flight of unregistered drones (i.e., "black flight" drones) also poses a serious threat to public safety, protection of important facilities, and airspace management. Currently, the main defense measures against unregistered drones include electromagnetic interference, signal deception, and physical interception. Among these, forced landing is a relatively direct and effective method that can prevent secondary damage caused by drone crashes. In forced landing operations, the core technical challenge lies in how to accurately acquire comprehensive environmental information of the landing area and quickly identify safe landing zones. Existing technologies often use single-view photography or local area scanning to detect the landing environment, resulting in large blind spots and incomplete environmental information. It is difficult to achieve 360-degree full coverage of a single space, and full-view coverage of different areas is even more lacking. At the same time, existing scanning methods only focus on the detection of open outdoor areas, ignoring the usability of enclosed or semi-enclosed spaces such as indoor areas and uninhabited areas inside buildings. This limits the range of selectable landing zones and makes it impossible to accurately determine key safety information such as the distribution of personnel and the status of obstacles within the landing zone. In addition, the receiving devices in existing forced landing systems mostly only have the functions of aiming and transmitting forced landing commands, lacking coordination and linkage with the global environment scanning module. They cannot dynamically adjust the aiming parameters and forced landing strategies based on the environmental information obtained from the scan, which can easily lead to risks such as improper selection of forced landing location and collision with obstacles during forced landing. For example, the low-altitude UAV defense data processing method proposed in patent CN115909111A can visualize the dynamic flight path of UAVs and determine countermeasure commands, but it does not involve full-domain environmental scanning and safe emergency landing area identification, and cannot provide environmental support for forced landing; while the multi-sensor fusion emergency landing method proposed by Beihang University International Innovation Research Institute is mainly aimed at the emergency landing of legal UAVs and does not meet the full-domain environmental detection requirements in the scenario of forced landing of unregistered UAVs. Therefore, the solution needs a scanning method that can achieve full-view, full-domain environmental scanning, integrate indoor and outdoor environmental information from multiple scenarios, and coordinate with a forced landing receiving device to solve problems such as incomplete environmental detection, low accuracy in identifying landable areas, and poor adaptability to forced landing strategies in existing technologies. Summary of the Invention

[0003] The purpose of this invention is to provide a dynamic obstacle avoidance and automatic landing optimization system for unmanned aerial vehicles (UAVs) in all domains and multiple scenarios. Addressing the technical shortcomings of existing systems for forced landing of unregistered UAVs, such as incomplete environmental detection, low accuracy in identifying landable areas, and poor coordination between scanning and forced landing, this invention provides a dynamic obstacle avoidance and automatic landing optimization system for UAVs in all domains and multiple scenarios. This system generates a full-domain environmental image through 360-degree image stitching, integrates indoor and outdoor multi-source monitoring data, achieves accurate identification of safe landing areas, and coordinates with a receiving device to optimize the forced landing strategy, thereby improving the safety and reliability of forced landing operations.

[0004] To achieve the above objectives, the present invention provides the following technical solution: a dynamic obstacle avoidance and automatic landing optimization system for unmanned aerial vehicles (UAVs) across multiple scenarios, applied to a forced landing system for unregistered UAVs. The system includes a scanning drone, a receiving device, a multi-source monitoring terminal, and a back-end processing center. The receiving device has automatic aiming and forced landing command transmission functions. The method includes the following steps: S1: Full-domain image acquisition: Control the scanning drone equipped with a high-definition wide-angle camera array to dynamically acquire 360-degree all-round images of the airspace where the unregistered drone is located and the ground and air areas below it according to a preset trajectory; the cameras dynamically rotate in six directions: front, back, left, right, up, and down of the scanning drone, with an overlap of 15-20 degrees between adjacent images to ensure no blind spots; during the acquisition process, the scanning drone's IMU attitude data (including pitch angle, roll angle, and yaw angle) and GNSS positioning data are acquired simultaneously for image correction.

[0005] S2: Full-Domain Image Stitching and Fusion: The back-end processing center receives multi-channel image data, IMU attitude data, and GNSS positioning data transmitted by the scanning UAV. First, it preprocesses the multi-channel images, including image noise reduction, distortion correction, and motion blur compensation. Then, based on the feature point matching algorithm, it extracts feature points in the overlapping areas of adjacent camera images and performs dynamic viewpoint correction in combination with IMU attitude data to compensate for the impact of flight attitude changes on imaging. Finally, using a parallel processing architecture, it seamlessly stitches the multi-angle images to generate a 360-degree full-domain panoramic image and labels the corresponding geographic coordinate information of the image.

[0006] S3: Multi-source monitoring data fusion: The back-end processing center connects to multi-source monitoring terminals within a preset area via a communication module. The multi-source monitoring terminals include outdoor public area monitoring cameras, building interior monitoring cameras, and enclosed space monitoring equipment. The center acquires real-time image data transmitted by the multi-source monitoring terminals and, through image geographic coordinate matching technology, fuses images of enclosed spaces such as indoor and uninhabited areas within buildings with the panoramic image of the entire environment generated in step S2 to form a complete global environment dataset covering outdoor open areas and indoor enclosed areas.

[0007] S4: Safe Emergency Landing Area Identification: Based on the fused global environmental dataset, a safe emergency landing area assessment model is constructed. The assessment indicators of the model include: area openness, obstacle density, personnel presence, area flatness, and environmental stability. The specific identification process is as follows: S41: Region segmentation: The semantic segmentation algorithm is used to divide the global environment image into regions to obtain candidate emergency landing areas, including open outdoor areas, uninhabited areas inside buildings, and enclosed safe areas indoors. S42: Indicator Detection: Identify obstacles (such as buildings, trees, and power facilities) and people in each candidate area through target detection algorithms, calculate obstacle density and the probability of people presence; calculate area flatness through image grayscale value analysis and edge detection; and analyze the stability of the area environment (such as whether there is temporary construction or fluctuation in pedestrian flow) by combining historical data from multi-source monitoring terminals. S43: Comprehensive assessment: Weighted scores are applied to each assessment indicator, with the weights based on the pre-set requirements for forced landing safety. The presence of personnel has the highest weight, followed by obstacle density. Candidate areas with a score ≥80 are determined as safe forced landing areas and are highlighted in the panoramic image of the entire environment.

[0008] S5: Collaborative Forced Landing Guidance: The back-end processing center transmits the full-area environmental image marked with safe forced landing areas and the corresponding geographic coordinate data to the receiving device. Based on this data, combined with the real-time position and flight attitude of the unregistered drone, the receiving device adjusts the aiming angle through an automatic aiming algorithm to lock onto the nearest safe forced landing area. At the same time, the back-end processing center generates an appropriate forced landing command based on the environmental parameters of the safe forced landing area (such as area size and flatness), and sends it to the unregistered drone through the receiving device to guide it to a precise forced landing.

[0009] Furthermore, in step S1, the preset trajectory of the scanning drone adopts a spiral descent trajectory, with the trajectory radius gradually reduced from 50m to 20m and the descent speed being 2m / s, ensuring full coverage of the air-ground area within 500m around the unregistered drone; the resolution of the acquired images is not less than 1080K and the bit rate is high, with a frame rate not less than 25fps, ensuring image clarity and real-time performance.

[0010] Furthermore, in step S2, the image preprocessing uses an adaptive median filtering algorithm for noise reduction, Zhang's calibration method for lens distortion correction, and a motion estimation-based blur compensation algorithm to eliminate motion blur during flight. The feature point matching algorithm uses the SIFT algorithm to ensure matching accuracy under varying lighting conditions and viewpoint shifts.

[0011] Furthermore, in step S3, before the multi-source monitoring data is fused, the monitoring terminal images need to be time-synchronized, and the synchronization error needs to be controlled within 50ms; an image matching algorithm based on ORB features is used to align the geographic coordinates of the monitoring images with the panoramic images of the whole-domain environment to ensure the spatial consistency of the fused images.

[0012] Furthermore, in step S4, the weights of the safe landing area assessment model can be dynamically adjusted through the back-end processing center. For densely populated areas (such as around schools and hospitals), the weight of the personnel presence index can be increased; for densely built areas, the weight of the obstacle density index can be increased.

[0013] Furthermore, in step S5, the automatic aiming algorithm of the receiving device adopts a target tracking algorithm based on Kalman filtering, predicts the flight trajectory of the unregistered UAV in real time, and dynamically adjusts the aiming parameters in combination with the location of the safe landing area, so that the aiming error is controlled within 0.5°.

[0014] Compared with the prior art, the beneficial effects of the present invention are: 1. This invention achieves full-range, blind-spot-free environmental detection: by acquiring images through a rotating wide-angle camera array, combined with IMU attitude correction and seamless stitching technology, a complete 360-degree panoramic image of the entire environment is generated, solving the problem of blind spots in the field of view of the existing single-view scanning, and ensuring the comprehensiveness of the emergency landing environment information.

[0015] 2. This invention expands the coverage of the forced landing area: by integrating outdoor monitoring data with monitoring data of enclosed spaces such as buildings and indoor spaces, the forced landing area is expanded from the traditional open outdoor areas to indoor safe zones, uninhabited areas inside buildings, and other scenarios, which effectively improves the flexibility and adaptability of forced landing operations, and is especially suitable for the handling of unregistered drones in densely populated urban areas.

[0016] 3. Improve the accuracy of safe emergency landing area identification: Construct a multi-indicator comprehensive evaluation model, and combine semantic segmentation, target detection and other algorithms to accurately identify obstacles and personnel, so as to achieve scientific determination of safe emergency landing areas and reduce the risk of secondary injury caused by improper selection of emergency landing areas.

[0017] 4. Achieve coordinated scanning and forced landing: Real-time linkage between the entire environmental data and the receiving device is achieved through the background processing center. The aiming parameters and forced landing strategies are dynamically optimized based on environmental information, which improves the accuracy and reliability of forced landing and solves the problem of disconnect between scanning and forced landing in the existing technology. Attached Figure Description

[0018] Figure 1 This is a schematic diagram of the process of the present invention. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

[0020] This embodiment provides a dynamic obstacle avoidance and automatic landing optimization system for unmanned aerial vehicles (UAVs) across multiple scenarios, applicable to the defense system against unregistered UAVs in urban core areas. The system includes one scanning UAV, two receiving devices (deployed on the east and west sides of the core area), supports multiple multi-source monitoring terminals (including one or more outdoor monitoring cameras and one or more building interior monitoring cameras), and one back-end processing center. The specific implementation steps are as follows: S1: Full-domain image acquisition: Control the scanning drone (equipped with a high bit rate wide-angle camera of 1080 or higher, with a lens focal length of 2.8mm) to fly in a spiral descent trajectory. The initial flight altitude is 200m and the trajectory radius is 50m. The trajectory radius is reduced by 5m for every 20m descent, and finally descends to an altitude of 100m with a trajectory radius of 20m and a descent speed of 2m / s. During the acquisition process, pitch angle, roll angle, and yaw angle data are acquired simultaneously through the drone's built-in IMU sensor (sampling frequency 100Hz), and latitude, longitude, and altitude data are acquired through the GNSS module (positioning accuracy 1m). The multi-channel image data, IMU data, and GNSS data are transmitted to the background processing center in real time (transmission rate ≥100Mbps).

[0021] S2: Full-Domain Image Stitching and Fusion: The back-end processing center utilizes a high-performance processor and a graphics card with a computing power of no less than FP32 and a floating-point performance of 82.58 TFLOPS to build a parallel processing platform for preprocessing the received data: an adaptive median filtering algorithm is used to eliminate image noise, Zhang's calibration method is used to correct camera distortion (camera calibration is completed in advance, and the intrinsic parameter matrix and distortion coefficients are obtained), and a motion estimation algorithm based on optical flow is used to compensate for motion blur during flight; then, the SIFT algorithm is used to extract feature points from adjacent camera images (feature point threshold is set to 0.03), and the RANSAC algorithm is used to remove mismatched feature points (1000 iterations, interior point threshold 2cm), and the image viewing angle is adjusted in combination with IMU attitude data to ensure horizontal alignment of adjacent images; finally, a multi-threaded parallel stitching algorithm is used to stitch the six-channel images together. The images are stitched together to generate a 360-degree panoramic image of the entire environment with a resolution of 8K×4K, and GNSS positioning data is labeled to the corresponding areas of the image.

[0022] S3: Multi-source monitoring data fusion: The back-end processing center connects to multi-source monitoring terminals via a 5G communication module to acquire real-time monitoring images (1080P resolution, 25fps frame rate); the NTP time synchronization protocol is used to synchronize the monitoring images with the images collected by the scanning drone, with the synchronization error controlled within 25ms; the ORB feature matching algorithm is used to extract common feature points between the monitoring images and the panoramic images of the whole-domain environment, and combined with the geographic coordinate information preset by the monitoring terminal, the spatial alignment of the two types of images is achieved, and the fusion generates a complete whole-domain environment dataset covering outdoor streets, squares, building interior corridors, underground parking lots and other areas.

[0023] S4: Safe Landing Area Identification: Construct a safe landing area assessment model, setting assessment indicators and weights: Presence of personnel (weight 0.4), obstacle density (weight 0.3), area openness (weight 0.15), area flatness (weight 0.1), and environmental stability (weight 0.05). Use the U-Net semantic segmentation model to segment the fused global environment image to obtain candidate landing areas (including city squares, building rooftop platforms, and open areas in underground parking lots, etc.). Use the YOLOv11 object detection algorithm to identify personnel (detection accuracy ≥95%) and obstacles (detection accuracy ≥90%) within each candidate area, calculating obstacle density (obstacle area / total area) and the probability of personnel presence (1 if personnel are present, 0 otherwise). Assess area flatness by calculating the image grayscale variance (variance ≤50 indicates flatness). Combine historical data from the monitoring terminal for the past hour to determine if personnel are present in the area. The system considers factors such as flow fluctuations and temporary construction (fluctuations ≤10% are considered stable). Each indicator is scored (0 points for a probability of 1 person present, 100 points otherwise; 100 points for obstacle density ≤5%, 60 points for 5%-20%, 0 points for >20%; 100 points for area openness ≥80%, 80 points for 60%-80%, 0 points for <60%; 100 points for area flatness, 0 points otherwise; 100 points for environmental stability, 0 points otherwise). A weighted comprehensive score is calculated, and areas with a score ≥80 are designated as safe areas for emergency landings and highlighted with a red border in the panoramic image of the entire environment.

[0024] S5: Coordinated Forced Landing Guidance: The back-end processing center transmits panoramic images of the entire environment marked with safe forced landing areas and their corresponding latitude and longitude coordinates to two receiving devices. The Kalman filter target tracking module built into the receiving devices, combined with the real-time position data of the unregistered UAV (obtained through radar detection, with a positioning accuracy of 0.5m), predicts its flight trajectory for the next 5 seconds, selects the safe forced landing area closest to the predicted trajectory (in this embodiment, it is a rooftop platform of a building, with an area of ​​20m × 15m and a comprehensive score of 92 points), and adjusts the aiming angle (aiming error 0.3°). Based on the area and flatness data of the platform, the back-end processing center generates a forced landing command (including a descent speed of 3m / s and a landing cushioning force of 0.5g), which is transmitted to the unregistered UAV through the 2.4G / 5.8G dual-frequency transmission module of the receiving device, guiding it to make a precise forced landing. Descend to a safe area.

[0025] In this embodiment, the above method was used to achieve a 360-degree full-area scan of the air and ground environment in the core urban area, successfully identifying three safe areas for emergency landing. The unregistered drone was eventually and accurately forced to land in the target area without any casualties or property damage, verifying the effectiveness and reliability of the method of the present invention.

[0026] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A dynamic obstacle avoidance and automatic landing optimization system for unmanned aerial vehicles (UAVs) across multiple scenarios, applied to a forced landing system for unregistered UAVs, the system comprising a scanning UAV, a receiving device, a multi-source monitoring terminal, and a back-end processing center, characterized in that... The method includes the following steps: S1: Full-domain image acquisition: Control the scanning drone equipped with a six-sided high-definition wide-angle camera array to perform 360-degree all-round image acquisition of the airspace where the unregistered drone is located and the ground and air area below it according to a preset trajectory; simultaneously acquire the scanning drone's IMU attitude data and GNSS positioning data; S2: Full-domain image stitching and fusion: The back-end processing center preprocesses the multi-channel image data, performs dynamic viewpoint correction based on the feature point matching algorithm and IMU attitude data, and seamlessly stitches the six-channel images to generate a 360-degree full-domain panoramic image, and marks the geographic coordinate information. S3: Multi-source monitoring data fusion: Connects to multi-source monitoring terminals to obtain real-time image data, and through time synchronization and geographic coordinate matching, fuses images of enclosed spaces such as indoor and uninhabited areas inside buildings with panoramic images of the entire environment to form a complete whole-domain environment dataset; S4: Safe Emergency Landing Area Identification: Construct a multi-index safe emergency landing area assessment model, analyze the entire environmental dataset through semantic segmentation and target detection algorithms, determine safe emergency landing areas and highlight them; S5: Cooperative Forced Landing Guidance: The entire environmental data marked with safe forced landing areas is transmitted to the receiving device. The receiving device adjusts the aiming angle to lock onto the safe forced landing area. The background processing center generates an appropriate forced landing command and sends it to the unregistered drone through the receiving device.

2. The UAV all-domain multi-scenario dynamic obstacle avoidance automatic landing optimization system according to claim 1, characterized in that, In the global image acquisition, the preset trajectory is a spiral descent trajectory with an initial trajectory radius of 50m, a descent speed of 2m / s, and the trajectory radius is reduced by 5m for every 20m descent, with a final trajectory radius of 20m. The overlapping area of ​​adjacent fields of view of the six cameras is 15-20 degrees, the image resolution is no less than 4K, and the frame rate is no less than 30fps.

3. The UAV all-domain multi-scenario dynamic obstacle avoidance automatic landing optimization system according to claim 1, characterized in that, In the full-domain image stitching and fusion, the preprocessing includes noise reduction using an adaptive median filtering algorithm, distortion correction using Zhang's calibration method, and blur compensation based on motion estimation; the feature point matching algorithm uses the SIFT algorithm, and the RANSAC algorithm is used to remove mismatched feature points.

4. The UAV all-domain multi-scenario dynamic obstacle avoidance automatic landing optimization system according to claim 1, characterized in that, In the multi-source monitoring data fusion, the NTP time synchronization protocol is used to achieve time synchronization between the monitoring images and the images collected by the scanning drone, with a synchronization error of ≤50ms; the ORB feature matching algorithm is used to align the geographic coordinates of the two types of images.

5. The UAV all-domain multi-scenario dynamic obstacle avoidance automatic landing optimization system according to claim 1, characterized in that, In the identification of the safe emergency landing area, the evaluation model indicators include the presence of personnel, obstacle density, area openness, area flatness, and environmental stability, with corresponding weights of 0.4, 0.3, 0.15, 0.1, and 0.05, respectively. The U-Net semantic segmentation model is used for area segmentation, and the YOLOv11 object detection algorithm is used to identify personnel and obstacles.

6. The UAV all-domain multi-scenario dynamic obstacle avoidance automatic landing and stopping optimization system according to claim 1, characterized in that, In the coordinated forced landing guidance, the receiving device uses a target tracking algorithm based on Kalman filtering to adjust the aiming angle, with an aiming error ≤0.5°.

Images