Unmanned aerial vehicle low-altitude obstacle detection method and system based on machine vision recognition

Abstract

The present application relates to the field of unmanned aerial vehicle flight management, and particularly relates to a method and system for detecting low-altitude obstacles of unmanned aerial vehicles based on machine vision recognition, in which, real-time flight parameters and health status of the unmanned aerial vehicle are acquired, a dynamic virtual spherical safety detection area with the unmanned aerial vehicle as the center is constructed, a binary mask is generated through three-dimensional spherical projection, an effective pixel area corresponding to the safety area is extracted, gradient edge detection and time difference algorithm are adopted to accurately identify static and dynamic obstacles, a dynamic and static fusion feature matrix is constructed, three-dimensional coordinates of the obstacles are reconstructed, threat level evaluation is performed according to three factors of distance, motion situation and obstacle type, and early warning and obstacle avoidance strategies are executed in stages, so that through adaptive flight state and low-altitude environment, static and dynamic obstacles are effectively distinguished, interference is suppressed, the safety of low-altitude flight of the unmanned aerial vehicle and the operation efficiency are improved, and the method is suitable for autonomous obstacle avoidance of unmanned aerial vehicles in multiple scenes.

Description

Technical Field

[0001] This invention relates to the field of unmanned aerial vehicle (UAV) flight management, and more particularly to a method and system for detecting low-altitude obstacles on UAVs based on machine vision recognition. Background Technology

[0002] In recent years, drones have been rapidly adopted in low-altitude operations such as power line inspection, logistics delivery, security monitoring, and emergency rescue. As flight scenarios become increasingly complex, low-altitude obstacle detection and autonomous obstacle avoidance have become core technologies for ensuring flight safety. Current mainstream drone obstacle avoidance solutions mostly rely on lidar, ultrasonic sensors, or monocular vision, which have significant limitations in complex low-altitude environments.

[0003] Firstly, traditional obstacle avoidance systems mostly use fixed safety distances and do not adjust them in conjunction with dynamic parameters such as the drone's real-time speed, braking performance, wind speed and direction. At high speeds, the safety margin is insufficient and collisions are likely to occur, while at low speeds, the redundancy is too large, affecting operational efficiency and making it difficult to adapt to changing flight conditions.

[0004] Secondly, conventional vision solutions typically process the entire frame of the image globally, resulting in a high proportion of invalid background pixels, large data redundancy, and problems such as high latency, insufficient frame rate, and poor real-time performance, which cannot meet the response requirements of low-altitude rapid obstacle avoidance.

[0005] Thirdly, monocular vision or simple binocular solutions are difficult to effectively distinguish between static obstacles (buildings, trees, utility poles) and dynamic obstacles (birds, drones, projectiles). They are easily affected by changes in lighting, airflow turbulence, and stray light noise, resulting in a large number of false alarms and missed alarms. Threat judgment accuracy is low, and it is impossible to accurately obtain the three-dimensional position and distance of obstacles relative to drones. This leads to inaccurate judgment of safe zone intrusion, rough threat level assessment, and obstacle avoidance strategies that are easily triggered or delayed.

[0006] In summary, traditional UAV low-altitude obstacle detection technology has significant shortcomings in terms of dynamic adaptability, real-time detection, identification accuracy, and intelligent threat assessment, making it difficult to meet the requirements for safe, efficient, and stable autonomous flight in complex low-altitude environments. Therefore, a solution is proposed. Summary of the Invention

[0007] This invention addresses the problems of rigid safe zones, inaccurate braking assessment, redundant image processing, confusion between moving and static targets, and coarse threat classification in traditional UAV low-altitude obstacle detection technologies. It provides a UAV low-altitude obstacle detection method and system based on machine vision recognition. Through dynamic safe zone construction, real-time braking capability calculation, effective pixel extraction, fusion recognition of moving and static obstacles, and threat classification-based obstacle avoidance, it achieves high-precision, high-real-time, and high-robust obstacle detection and autonomous obstacle avoidance in complex low-altitude environments.

[0008] The objective of this invention can be achieved through the following technical solution: a method for detecting low-altitude obstacles in unmanned aerial vehicles based on machine vision recognition, comprising the following steps:

[0009] Step 1: Acquisition of the dynamic safety detection area of ​​the target UAV: ​​The flight parameters of the target UAV are acquired in real time. Combined with the kinematic and dynamic equations, the relevant parameters are substituted to calculate the omnidirectional dynamic safety radius. With the UAV as the center and the radius as the scale, a dynamic virtual sphere is constructed as the dynamic safety detection area of ​​the UAV.

[0010] Step 2: Braking Flexible Matching Calculation: Real-time acquisition of the target UAV's health parameters, combined with the torque-thrust conversion coefficient to calculate the total available reverse thrust, and then the ratio of the total reverse thrust to the airframe mass to obtain the UAV's current maximum braking acceleration;

[0011] Step 3: Obtaining the effective pixel area of ​​the dynamic security detection region: Establish the equation of the dynamic virtual sphere, and simultaneously map the three-dimensional sampling points of the sphere to the imaging plane to obtain the two-dimensional projection boundary. Construct a binary mask matrix, perform a bitwise AND operation with the original image pixel by pixel, and finally extract the effective pixel area corresponding to the dynamic security detection region.

[0012] Step 4: Dynamic and static obstacle recognition and construction of fusion feature matrix: preprocessing of effective pixel areas of multiple cameras, establishing pixel coordinate index, recognizing static and dynamic obstacles, calculating pixel depth based on stereo matching of camera calibration parameters, unifying coordinates and classifying and encoding, and constructing fusion feature matrix;

[0013] Step 5: Threat level feedback of target UAV: ​​reconstruct the three-dimensional coordinates of the obstacle's body coordinate system, determine whether it has intruded into the safe zone, extract three assessment factors to calculate the threat level, and match and execute the corresponding early warning and obstacle avoidance strategy.

[0014] Preferably, the dynamic safety detection area acquisition and analysis process for the target UAV is as follows:

[0015] The target UAV's current flight parameters are acquired in real time, and the omnidirectional dynamic safety radius RS is calculated based on kinematic and dynamic equations.

[0016] With the target drone as the center and the omnidirectional dynamic safety radius RS as the radius, a dynamic virtual spherical region, namely the dynamic safety detection region, is constructed.

[0017] Preferably, the braking flexibility matching calculation and analysis process is as follows:

[0018] The health parameters of the target drone are acquired in real time. The motors in the target drone are marked as g, where g is a natural number greater than zero. Based on the health parameters, the maximum current IZmax allowed to pass through a single battery within the braking window, the maximum current IDmax allowed to pass through the ESC terminal, and the upper limit of the equivalent current ISmax that the peak power of the battery is evenly distributed to each motor are acquired in real time. Then, the maximum usable torque Ng of a single motor is obtained, where Ng = motor torque constant × min(IZmaxg, IDmaxg, ISmaxg).

[0019] The preset torque-thrust conversion coefficient is retrieved. The total available reverse thrust is calculated based on the sum of the preset torque-thrust conversion coefficient and the maximum available torque Ng of all motors. The maximum braking acceleration amax of the current target UAV is calculated based on the total available reverse thrust / the mass of the target UAV.

[0020] Preferably, the effective pixel area acquisition and analysis process of the dynamic security detection area is as follows:

[0021] Obtain the equation of the dynamic virtual sphere region with the target UAV as the origin, map the sampling point set of the surface of the dynamic virtual sphere region onto the two-dimensional imaging plane of the camera, and fit the resulting discrete two-dimensional point set into a closed two-dimensional projection boundary Bi by the convex hull algorithm.

[0022] Obtain the distortion coefficient of the i-th camera and perform inverse distortion correction on all pixels within the projection boundary Bi to generate a distortion-free local pixel region Ri′. Using the outer rectangle of Ri′ as the boundary, generate a binary mask matrix. In this binary mask matrix, pixels inside the Ri′ contour are assigned a value of 1, and pixels outside are assigned a value of 0. Only pixels with a mask value of 1 are retained, thus obtaining the effective pixel area corresponding to the dynamic security detection area.

[0023] Preferably, the analysis process for identifying dynamic and static obstacles and constructing a fusion feature matrix is ​​as follows:

[0024] S1: Obtain the effective pixel area of ​​the dynamic security detection region, preprocess the image frames of the effective pixel area of ​​multiple cameras, and construct a full pixel grayscale matrix.

[0025] S2: Establish a global coordinate index table for pixels within the effective pixel area;

[0026] S3: For the preprocessed full-pixel grayscale matrix, the gradient edge detection algorithm is used to calculate the horizontal and vertical grayscale gradient magnitude and gradient direction pixel by pixel, and the extreme pixels of gradient abrupt change are selected as edge feature candidate points.

[0027] The edge feature candidate points are clustered into 8-neighbor connected components to aggregate spatially continuous edge pixels into independent closed contours, and invalid pseudo contours formed by isolated single pixel noise and discrete noise are eliminated.

[0028] Morphological screening is performed on the contours of each connected component, and the contour regions that conform to the scale characteristics of static obstacles are marked as static obstacle pixel mask regions.

[0029] The pixel mask area of ​​static obstacles is marked with full pixel solidification, and this area is locked as a fixed background pixel of the scene.

[0030] Preferably, it also includes S4: continuously caching the effective pixel area images of the current frame, the previous frame, and the two frames before that, performing frame-by-frame grayscale value difference operation only on all pixels in the non-static obstacle mask area, setting an adaptive illumination difference threshold, and determining a moving candidate pixel when the grayscale difference of the same pixel coordinate exceeds the threshold for multiple consecutive frames.

[0031] The neighborhood connectivity of moving candidate pixels is checked, the discretely distributed moving candidate pixels are clustered into continuous pixel blocks, the random jitter interference of single pixels is eliminated, independent dynamic target pixel blocks are divided, and dynamic obstacles are identified.

[0032] S5: For the same edge feature points in the effective pixel area of ​​multiple cameras, match the same feature pixel points in the field of view of different cameras that correspond to the same physical space location. For each successfully matched pair of same feature pixels, calculate the spatial depth distance pixel by pixel and generate a full pixel dense depth mapping table of the effective pixel area to realize that each pixel is bound to a unique spatial depth coordinate.

[0033] The spatial depth coordinates of the pixels are transformed into the UAV body coordinate system and unified with the coordinate system of the constructed dynamic virtual sphere region. All pixels in the effective pixel area are classified and encoded, and the static contour information of each camera's field of view, the position and motion information of dynamic pixel blocks, and the full pixel depth data are integrated to construct a unified format full pixel static and dynamic fusion feature matrix.

[0034] Preferably, the flight threat classification feedback analysis process for the target UAV is as follows:

[0035] Based on the full-pixel dynamic-static fusion feature matrix, for each effective pixel (u, v), combined with the spatial depth value of the corresponding pixel, the two-dimensional image pixel coordinates are inversely solved into three-dimensional spatial coordinates in the camera coordinate system, and then mapped to the UAV body coordinate system (Xb, Yb, Zb) through the external parameter matrix coordinate transformation.

[0036] Complete the body coordinate system coordinate reconstruction of all obstacle pixels, realize the one-to-one correspondence between image pixels, spatial three-dimensional position, and depth distance, and establish a global obstacle spatial point set;

[0037] For each obstacle's three-dimensional spatial point (Xb, Yb, Zb), substitute it into the dynamic virtual sphere region equation to perform distance discrimination: output the target to be assessed or the target outside the security domain.

[0038] Preferably, three core evaluation factors are obtained and normalized. The core evaluation factors include distance factor Fd, motion state factor Fv, and obstacle type factor Fc.

[0039] The obstacle threat coefficient is calculated based on the preset weight coefficient corresponding to Fd×Fd + the preset weight coefficient corresponding to Fv×Fv + the preset weight coefficient corresponding to Fc×Fc. The preset obstacle threat coefficient range [Wmin, Wmax] is retrieved for comparison, and low threat, medium threat, or high threat is output.

[0040] Early warning and intervention decisions are based on matching threat levels.

[0041] The beneficial effects of this invention are as follows:

[0042] (1) The present invention adopts an omnidirectional dynamic safety radius to construct a virtual spherical safety zone in real time, and dynamically adjusts multiple factors. The safety boundary adapts to the flight state and environment in real time. At the same time, the maximum braking acceleration is calculated online based on parameters such as battery health, ESC, and motor torque. The safety radius is strongly bound to the actual braking performance of the UAV, the safety distance calculation is more in line with the real controllable state, and the early warning and obstacle avoidance are more reliable.

[0043] (2) The present invention also generates a binary mask by three-dimensional spherical projection, retains only the effective pixels in the safe area, greatly eliminates the invalid background area, reduces the amount of image processing data, reduces the computing power and power consumption pressure of the airborne terminal, and uses gradient edge detection + connected component clustering to identify static obstacles and three-frame temporal difference to identify dynamic obstacles.

[0044] (3) The present invention also eliminates field of view error and coordinate offset based on the accurate calculation of the three-dimensional position and depth of the obstacle. At the same time, it evaluates the threat level by weighting distance, motion state and obstacle type, and matches low / medium / high three-level early warning and obstacle avoidance strategies. Threat judgment is quantified and response is graded to avoid meaningless maneuvers and take into account both flight efficiency and safety. Attached Figure Description

[0045] The invention will now be further described with reference to the accompanying drawings;

[0046] Figure 1 This is a schematic diagram of the method of the present invention;

[0047] Figure 2 This is a flowchart of the system of the present invention;

[0048] Figure 3 This is a reference diagram for dynamic safety detection area analysis. Detailed Implementation

[0049] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0050] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of the invention. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments;

[0051] Example 1: Please refer to Figures 1 to 3 As shown, this invention is a method for detecting low-altitude obstacles on drones based on machine vision recognition, comprising the following steps:

[0052] Step 1: Acquisition of the dynamic safety detection area of ​​the target UAV: ​​The flight parameters of the target UAV are acquired in real time. Combined with the kinematic and dynamic equations, the relevant parameters are substituted to calculate the omnidirectional dynamic safety radius. With the UAV as the center and the radius as the scale, a dynamic virtual sphere is constructed as the dynamic safety detection area of ​​the UAV.

[0053] Step 2: Braking Flexible Matching Calculation: Real-time acquisition of the target UAV's health parameters, combined with the torque-thrust conversion coefficient to calculate the total available reverse thrust, and then the ratio of the total reverse thrust to the airframe mass to obtain the UAV's current maximum braking acceleration;

[0054] Step 3: Obtaining the effective pixel area of ​​the dynamic security detection region: Establish the equation of the dynamic virtual sphere, and simultaneously map the three-dimensional sampling points of the sphere to the imaging plane to obtain the two-dimensional projection boundary. Construct a binary mask matrix, perform a bitwise AND operation with the original image pixel by pixel, and finally extract the effective pixel area corresponding to the dynamic security detection region.

[0055] Step 4: Dynamic and static obstacle recognition and construction of fusion feature matrix: preprocessing of effective pixel areas of multiple cameras, establishing pixel coordinate index, recognizing static and dynamic obstacles, calculating pixel depth based on stereo matching of camera calibration parameters, unifying coordinates and classifying and encoding, and constructing fusion feature matrix;

[0056] Step 5: Threat level feedback of target UAV: ​​reconstruct the three-dimensional coordinates of the obstacle body coordinate system, determine whether it has intruded into the safe area, extract three evaluation factors to calculate the threat level, match and execute the corresponding early warning and obstacle avoidance strategy;

[0057] Step 1: Acquisition of the dynamic safety detection area of ​​the target UAV: ​​The flight parameters of the target UAV are acquired in real time. Combined with the kinematic and dynamic equations, the omnidirectional dynamic safety radius is calculated by substituting relevant parameters. With the UAV as the center and this radius as the scale, a dynamic virtual sphere is constructed as the dynamic safety detection area of ​​the UAV. The specific analysis process for acquiring the dynamic safety detection area of ​​the target UAV is as follows:

[0058] Real-time acquisition of the target UAV's current flight parameters, which include at least: ground speed vector V, attitude angle, airframe mass m, and current wind speed and direction W;

[0059] Based on kinematic and dynamic equations, the omnidirectional dynamic safety radius RS is calculated.

[0060] Omnidirectional dynamic safety radius RS = V 2 / 2amax+V×transmission delay duration+k×|wind speed and wind direction W|×cosθ+preset static redundancy spacing, where amax represents the maximum braking acceleration, k represents the preset drag coefficient, and θ is the angle between the wind speed vector and the drone velocity vector;

[0061] With the target UAV as the center and the omnidirectional dynamic safety radius RS as the radius, a dynamic virtual spherical region, namely the dynamic safety detection region, is constructed.

[0062] Step Two: Braking Flexibility Matching Calculation: Real-time acquisition of the target UAV's health parameters, combined with the torque-thrust conversion coefficient, calculates the total available reverse thrust. Then, using the ratio of the total reverse thrust to the airframe mass, the current maximum braking acceleration of the UAV is obtained. The specific braking flexibility matching calculation and analysis process is as follows:

[0063] Real-time acquisition of the target drone's health parameters, including remaining battery percentage and battery terminal voltage;

[0064] The motors in the target drone are labeled as g, where g is a natural number greater than zero. Based on health parameters, the maximum current IZmax allowed to pass through a single battery within the braking window, the maximum current IDmax allowed to pass through the ESC terminal, and the upper limit of the equivalent current ISmax that the battery peak power is evenly distributed to each motor are obtained in real time. Then, the maximum usable torque Ng of a single motor is obtained, where Ng = motor torque constant × min(IZmaxg, IDmaxg, ISmaxg).

[0065] The preset torque-thrust conversion coefficient is retrieved, and the total available reverse thrust is calculated based on the sum of the preset torque-thrust conversion coefficient and the maximum available torque Ng of all motors.

[0066] The maximum braking acceleration (amax) of the target UAV is calculated based on the total available reverse thrust / target UAV mass.

[0067] Example 2: Step 3: Acquisition of the effective pixel area of ​​the dynamic security detection region: Establish the equation of a dynamic virtual sphere, and simultaneously map the three-dimensional sampling points of the sphere onto the imaging plane to obtain the two-dimensional projection boundary. Construct a binary mask matrix, and perform a pixel-by-pixel AND operation with the original image to finally extract the effective pixel area corresponding to the dynamic security detection region. The specific analysis process for acquiring the effective pixel area of ​​the dynamic security detection region is as follows:

[0068] Obtain the equation of the dynamic virtual sphere region with the target UAV as the origin: Xb 2 +Yb 2 +Zb 2 =RS 2 Where (Xb, Yb, Zb) are the spatial coordinates of a point in the body coordinate system;

[0069] Using existing technology, the sampling point set of the surface of the dynamic virtual sphere region is mapped onto the two-dimensional imaging plane of the camera. The resulting discrete two-dimensional point set is fitted into a closed two-dimensional projection boundary Bi using the convex hull algorithm. That is, for the i-th (i>0) camera in the target UAV, its accurate intrinsic parameter matrix Ki and the rotation and translation extrinsic parameter matrix [Ri|ti] from the body coordinate system to the camera coordinate system have been obtained through offline calibration. Through the perspective projection equation, the sampling point set of the three-dimensional sphere surface can be mapped onto the two-dimensional imaging plane of the camera: S[u, v, 1]. T =Ki×[Ri|ti]×[Xb, Yb, Zb, 1] T Where S is the scale factor and (u, v) are the image pixel coordinates;

[0070] Obtain the distortion coefficients of the i-th camera and perform inverse distortion correction on all pixels within the projection boundary Bi to generate a distortion-free local pixel region Ri′.

[0071] A binary mask matrix is ​​generated with the bounding rectangle of Ri′ as the boundary. In this binary mask matrix, pixels inside the Ri′ contour are assigned a value of 1, and pixels outside are assigned a value of 0.

[0072] The binary mask is ANDed pixel by pixel with the original full-frame image captured by the camera, and only the pixels with a mask value of 1 are retained, thus obtaining the effective pixel area corresponding to the dynamic security detection area.

[0073] Step 4: Dynamic and Static Obstacle Recognition and Construction of Fusion Feature Matrix: Preprocessing of effective pixel areas from multiple cameras, establishing pixel coordinate indexes, recognizing static and dynamic obstacles, calculating pixel depth based on stereo matching using camera calibration parameters, unifying coordinates and classifying and encoding them, and constructing a fusion feature matrix. The specific analysis process for dynamic and static obstacle recognition and construction of the fusion feature matrix is ​​as follows:

[0074] S1: Obtain the effective pixel area of ​​the dynamic security detection region obtained by each camera after pixel-by-pixel AND operation with mask. Perform preprocessing such as timestamp synchronization calibration and grayscale conversion on the image frames of the effective pixel area of ​​multiple cameras to construct a full pixel grayscale matrix.

[0075] That is, using the UAV's onboard flight control clock as a unified reference, the timestamps of the effective pixel area image frames of multiple cameras are synchronized and calibrated to ensure that the timing reference of multi-field images is consistent at the same time.

[0076] For each effective pixel area, grayscale conversion and Gaussian smoothing filtering are performed frame by frame to filter out random noise caused by low airflow jitter and stray light, while preserving obstacle edges and texture details, and constructing a normalized full-pixel grayscale matrix.

[0077] S2: Establish a global coordinate index table for pixels within the effective pixel area, bind the row and column position of each pixel, its camera number, and imaging plane coordinate information to achieve full pixel addressability and traceability management, and provide a coordinate reference for subsequent pixel-by-pixel calculations;

[0078] S3: For the preprocessed full-pixel grayscale matrix, the gradient edge detection algorithm is used to calculate the horizontal and vertical grayscale gradient magnitude and gradient direction pixel by pixel, and the extreme pixels of gradient abrupt change are selected as edge feature candidate points.

[0079] The edge feature candidate points are clustered into 8-neighbor connected components to aggregate spatially continuous edge pixels into independent closed contours, and invalid pseudo contours formed by isolated single pixel noise and discrete noise are eliminated.

[0080] A prior size constraint threshold for low-altitude obstacles is introduced, and morphological screening is performed on the contours of each connected region. Contour regions that conform to the scale characteristics of static obstacles such as buildings, trees, utility poles, and tower cranes are marked as static obstacle pixel mask areas.

[0081] The pixel mask area of ​​static obstacles is marked with full pixel solidification, and the area is locked as a fixed background pixel of the scene. The area is skipped in the subsequent frame time difference operation to avoid pseudo-moving pixel interference from static background.

[0082] S4: Continuously buffers the effective pixel area images of three frames of time synchronization: the current frame, the previous frame, and the two frames before that. Performs frame-by-frame grayscale value difference operation only on all pixels in the non-static obstacle mask area.

[0083] Set an adaptive illumination difference threshold and dynamically adjust the difference judgment threshold based on the overall average gray level of the real-time image. When the gray level difference of the same pixel coordinate exceeds the threshold for multiple consecutive frames, it is judged as a moving candidate pixel.

[0084] The neighborhood connectivity of moving candidate pixels is checked, the discrete moving candidate pixels are clustered into continuous pixel blocks, the random jitter interference of single pixels is eliminated, and independent dynamic target pixel blocks are divided to identify dynamic obstacles such as birds, low-altitude projectiles, and other moving aircraft.

[0085] The pixel coordinate set, region area, and inter-frame pixel offset of each dynamic target pixel block are statistically analyzed, and the pixel planar motion vector is recorded as the basic data for subsequent motion trend analysis.

[0086] S5: Reuse the camera intrinsic parameter matrix Ki, rotation and translation extrinsic parameter matrix [Ri|ti] and distortion coefficients that have been calibrated offline as fixed reference parameters for geometric solution, without the need for additional model training and calibration;

[0087] For common edge feature points in the effective pixel area of ​​multiple cameras, the traditional feature matching algorithm is used to complete cross-field feature registration, matching common feature pixel points corresponding to the same physical spatial location in different camera fields of view;

[0088] Based on the geometric principle of binocular stereo vision triangulation, combined with the camera intrinsic and extrinsic parameter calibration matrix, for each successfully matched pair of homologous feature pixels, the spatial depth distance is calculated pixel by pixel to generate a full pixel dense depth mapping table of the effective pixel area, so as to bind each pixel with a unique spatial depth coordinate.

[0089] By combining the transformation relationship between the body coordinate system and the camera coordinate system, the spatial depth coordinates of the pixels are transformed to the UAV body coordinate system, which is consistent with the coordinate system of the constructed dynamic virtual sphere region;

[0090] All pixels within the valid pixel area are classified and encoded: static obstacle pixels are bound to contour type, pixel coordinates, and spatial depth value; dynamic target pixels are bound to planar motion vector, pixel block size, and spatial depth value; blank background pixels are marked as invalid feature bits.

[0091] By integrating static contour information, dynamic pixel block position and motion information, and full pixel depth data from each camera's field of view, a unified format full pixel motion-static fusion feature matrix is ​​constructed.

[0092] The standardized output of the full-pixel dynamic and static fusion feature matrix is ​​used in the subsequent threat assessment stage, providing complete original feature input for obstacle spatial location determination, category differentiation, motion trend analysis, and intrusion into the dynamic virtual sphere region.

[0093] Example 3: Step 5: Flight Threat Classification Feedback of Target UAV: ​​Reconstruct the three-dimensional coordinates of the obstacle's body coordinate system, determine whether it has intruded into the safe zone, extract three evaluation factors to calculate the threat level, match and execute the corresponding early warning and obstacle avoidance strategy. The specific flight threat classification feedback analysis process of the target UAV is as follows:

[0094] Based on the full-pixel dynamic-static fusion feature matrix, for each effective pixel (u, v), combined with the spatial depth value of the corresponding pixel, the two-dimensional image pixel coordinates are inversely solved into three-dimensional spatial coordinates in the camera coordinate system through inverse perspective projection transformation, and then uniformly mapped to the UAV body coordinate system (Xb, Yb, Zb) through external parameter matrix coordinate transformation.

[0095] Complete the body coordinate system coordinate reconstruction of all obstacle pixels, realize the one-to-one correspondence between image pixels, spatial three-dimensional position, and depth distance, and establish a global obstacle spatial point set;

[0096] For each obstacle's three-dimensional spatial point (Xb, Yb, Zb), the distance is determined by substituting it into the equation of the dynamic virtual sphere region:

[0097] If the Euclidean distance from the spatial point to the origin of the UAV body is less than or equal to RS, it is determined that the obstacle has intruded into the dynamic security detection area and is marked as a target to be assessed for threat. At the same time, the attribute information of the marked target to be assessed for threat is distinguished, including static obstacles and dynamic obstacles.

[0098] If the Euclidean distance from the spatial point to the origin of the UAV body is greater than RS, it is determined to be a target outside the safe zone and will not be included in the subsequent threat level assessment;

[0099] By utilizing the inter-frame offset of pixels in multiple consecutive frames of images, combined with the physical scale of camera imaging and pixel spatial depth, the relative velocity magnitude and motion direction vector of the dynamic target relative to the UAV are calculated.

[0100] The dynamic target motion direction vector is projected onto the forward and lateral reference directions of the UAV to determine whether the dynamic target is approaching, moving away from, or laterally passing through the safety detection area, and motion trend feature parameters are extracted.

[0101] Static obstacles have no inter-frame pixel offset, their default relative velocity is 0, and only their spatial position and distance attributes are retained.

[0102] Three core evaluation factors were obtained and normalized. The core evaluation factors include distance factor Fd, motion state factor Fv, and obstacle type factor Fc.

[0103] Distance factor: Based on the Euclidean distance from the obstacle to the center of mass of the drone, the closer the distance, the higher the weight of the factor. For example, the distance is divided into three levels: near / medium / far, and high score, medium score and low score are assigned respectively to complete the normalization.

[0104] Motion state factor: the highest value is assigned to approaching, followed by lateral, and the lowest value is assigned to moving away; the value is normalized for direct gear.

[0105] Obstacle type factor: Preset two levels: rigid obstacles (utility poles, buildings) and flexible moving targets (birds, projectiles), with fixed and normalized values;

[0106] The obstacle threat coefficient is calculated based on the preset weight coefficient corresponding to Fd×Fd + the preset weight coefficient corresponding to Fv×Fv + the preset weight coefficient corresponding to Fc×Fc. The preset obstacle threat coefficient range [Wmin, Wmax] is retrieved for comparison. If the obstacle threat coefficient < Wmin, it is judged as low threat; if the obstacle threat coefficient ∈ [Wmin, Wmax], it is judged as medium threat; if the obstacle threat coefficient > Wmax, it is judged as high threat.

[0107] Based on the threat level (low / medium / high threat), the system matches the corresponding early warning and intervention decision, outputs the early warning and intervention decision, and executes it immediately to improve the flight safety and early warning timeliness of the target UAV.

[0108] For example: Low threat level: Only area marking and recording are performed, without triggering flight attitude intervention;

[0109] Medium threat level: Triggering minor adjustments to the flight path, maintaining the original flight speed and making slight detours;

[0110] High threat level: Immediately triggers deceleration braking, hovering obstacle avoidance, or lateral maneuvering avoidance commands.

[0111] Example 4: The present invention also proposes a low-altitude obstacle detection system for UAVs based on machine vision recognition, including a dynamic safe zone construction module, a braking capacity calculation module, an effective pixel extraction module, an obstacle recognition and feature fusion module, and a threat classification and obstacle avoidance module. The dynamic safe zone construction module and the braking capacity calculation module are bidirectionally connected, the braking capacity calculation module and the effective pixel extraction module are unidirectionally connected, the effective pixel extraction module and the obstacle recognition and feature fusion module are unidirectionally connected, and the obstacle recognition and feature fusion module and the threat classification and obstacle avoidance module are unidirectionally connected.

[0112] The dynamic safety zone construction module is used to acquire the flight parameters of the target UAV in real time, acquire the dynamic safety detection zone of the target UAV, and output the dynamic safety detection zone of the target UAV.

[0113] The braking capacity calculation module is used to collect the health parameters of the target UAV in real time, perform braking flexible matching calculation and analysis, calculate the total available reverse thrust in combination with the torque-thrust conversion coefficient, and output the current maximum braking acceleration.

[0114] The effective pixel extraction module is used to obtain and analyze the effective pixel area of ​​the dynamic security detection area. Combining the established dynamic virtual sphere equation and binary mask matrix, it outputs the effective pixel area corresponding to the dynamic security detection area.

[0115] The obstacle recognition and feature fusion module is used for the recognition of dynamic and static obstacles and the construction and analysis of the fusion feature matrix. At the same time, it preprocesses the effective pixel area of ​​multiple cameras and finally outputs the fusion feature matrix.

[0116] The threat classification and obstacle avoidance module is used to reconstruct the three-dimensional coordinates of obstacles in the UAV's body coordinate system, determine whether they have intruded into the dynamic security detection area, provide flight threat classification feedback for the target UAV, and match and execute corresponding early warning and obstacle avoidance strategies.

[0117] In summary, the system employs an omnidirectional dynamic safety radius to construct a virtual spherical safety zone in real time. This system dynamically adjusts multiple factors, and the safety boundary adapts to the flight status and environment in real time. This solves the problem of missed or false alarms caused by traditional fixed safety distances, significantly improving the safety detection accuracy in complex low-altitude scenarios. Furthermore, it calculates the maximum braking acceleration online based on parameters such as battery health, ESC, and motor torque. The safety radius is strongly linked to the actual braking performance of the UAV, and the safety distance calculation is more in line with the real and controllable state, making early warning and obstacle avoidance more reliable. Moreover, by generating a binary mask through 3D spherical projection, only valid pixels within the safety area are retained, significantly eliminating invalid background areas, reducing the amount of image processing data, reducing the computing power and power consumption pressure on the airborne end, and improving the real-time performance of detection.

[0118] It employs gradient edge detection and connected component clustering to identify static obstacles and three-frame temporal difference to identify dynamic obstacles. It solidifies the static background, filters out lighting and jitter noise, and effectively avoids false target interference caused by birds, airflow, and stray light, resulting in higher recognition accuracy. It also accurately calculates the three-dimensional position and depth of obstacles to eliminate field of view errors and coordinate offsets. At the same time, it assesses the threat level by weighting distance, motion status, and obstacle type, and matches low / medium / high warning and obstacle avoidance strategies. Threat judgment is quantified and response is graded to avoid meaningless maneuvers and balance flight efficiency and safety.

[0119] The threshold is set for result comparison and analysis to determine whether it is good or bad. The value of the threshold is determined by a combination of large-scale model analysis of the sample data and human experience, and can also be adjusted appropriately based on seasonal or common-sense influencing factors. The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A method for detecting low-altitude obstacles in unmanned aerial vehicles (UAVs) based on machine vision recognition, characterized in that, Includes the following steps: Step 1: Acquisition of the dynamic safety detection area of ​​the target UAV: ​​The flight parameters of the target UAV are acquired in real time. Combined with the kinematic and dynamic equations, the relevant parameters are substituted to calculate the omnidirectional dynamic safety radius. With the UAV as the center and the radius as the scale, a dynamic virtual sphere is constructed as the dynamic safety detection area of ​​the UAV. Step 2: Braking Flexible Matching Calculation: Real-time acquisition of the target UAV's health parameters, combined with the torque-thrust conversion coefficient to calculate the total available reverse thrust, and then the ratio of the total reverse thrust to the airframe mass to obtain the UAV's current maximum braking acceleration; Step 3: Obtaining the effective pixel area of ​​the dynamic security detection region: Establish the equation of the dynamic virtual sphere, and simultaneously map the three-dimensional sampling points of the sphere to the imaging plane to obtain the two-dimensional projection boundary. Construct a binary mask matrix, perform a bitwise AND operation with the original image pixel by pixel, and finally extract the effective pixel area corresponding to the dynamic security detection region. Step 4: Dynamic and static obstacle recognition and construction of fusion feature matrix: preprocessing of effective pixel areas of multiple cameras, establishing pixel coordinate index, recognizing static and dynamic obstacles, calculating pixel depth based on stereo matching of camera calibration parameters, unifying coordinates and classifying and encoding, and constructing fusion feature matrix; Step 5: Threat level feedback of target UAV: ​​reconstruct the three-dimensional coordinates of the obstacle's body coordinate system, determine whether it has intruded into the safe zone, extract three assessment factors to calculate the threat level, and match and execute the corresponding early warning and obstacle avoidance strategy.

2. The method for detecting low-altitude obstacles in unmanned aerial vehicles based on machine vision recognition according to claim 1, characterized in that, The process of acquiring and analyzing the dynamic safety detection area of ​​the target UAV is as follows: The system acquires the target UAV's current flight parameters in real time and calculates the omnidirectional dynamic safety radius RS based on kinematic and dynamic equations. A dynamic virtual spherical region, i.e., the dynamic safety detection region, is constructed with the target UAV as the center and the omnidirectional dynamic safety radius RS as the radius.

3. The method for detecting low-altitude obstacles in unmanned aerial vehicles based on machine vision recognition according to claim 1, characterized in that, The calculation and analysis process for flexible braking matching is as follows: The health parameters of the target drone are acquired in real time. The motors in the target drone are marked as g, where g is a natural number greater than zero. Based on the health parameters, the maximum current IZmax allowed to pass through a single battery within the braking window, the maximum current IDmax allowed to pass through the ESC terminal, and the upper limit of the equivalent current ISmax that the peak power of the battery is evenly distributed to each motor are acquired in real time. Then, the maximum usable torque Ng of a single motor is obtained, where Ng = motor torque constant × min(IZmaxg, IDmaxg, ISmaxg). The preset torque-thrust conversion coefficient is retrieved. The total available reverse thrust is calculated based on the sum of the preset torque-thrust conversion coefficient and the maximum available torque Ng of all motors. The maximum braking acceleration amax of the current target UAV is calculated based on the total available reverse thrust / the mass of the target UAV.

4. The method for detecting low-altitude obstacles in unmanned aerial vehicles based on machine vision recognition according to claim 1, characterized in that, The process for obtaining and analyzing the effective pixel area of ​​the dynamic security detection region is as follows: Obtain the equation of the dynamic virtual sphere region with the target UAV as the origin, map the sampling point set of the surface of the dynamic virtual sphere region onto the two-dimensional imaging plane of the camera, and fit the resulting discrete two-dimensional point set into a closed two-dimensional projection boundary Bi by the convex hull algorithm. Obtain the distortion coefficient of the i-th camera and perform inverse distortion correction on all pixels within the projection boundary Bi to generate a distortion-free local pixel region Ri′. Using the outer rectangle of Ri′ as the boundary, generate a binary mask matrix. In this binary mask matrix, pixels inside the Ri′ contour are assigned a value of 1, and pixels outside are assigned a value of 0. Only pixels with a mask value of 1 are retained, thus obtaining the effective pixel area corresponding to the dynamic security detection area.

5. The method for detecting low-altitude obstacles in unmanned aerial vehicles based on machine vision recognition according to claim 1, characterized in that, The analysis process for identifying dynamic and static obstacles and constructing a fusion feature matrix is ​​as follows: S1: Obtain the effective pixel area of ​​the dynamic security detection region, preprocess the image frames of the effective pixel area of ​​multiple cameras, and construct a full pixel grayscale matrix. S2: Establish a global coordinate index table for pixels within the effective pixel area; S3: For the preprocessed full-pixel grayscale matrix, the gradient edge detection algorithm is used to calculate the horizontal and vertical grayscale gradient magnitude and gradient direction pixel by pixel, and the extreme pixels of gradient abrupt change are selected as edge feature candidate points. Eight-neighbor connected component clustering is performed on edge feature candidate points to aggregate spatially continuous edge pixels into independent closed contours, and invalid pseudo contours formed by isolated single-pixel noise and discrete noise are eliminated; morphological screening is performed on each connected component contour, and contour regions that conform to the scale characteristics of static obstacles are marked as static obstacle pixel mask regions; full-pixel solidification marking is performed on the static obstacle pixel mask regions to lock the region as fixed background pixels of the scene.

6. The method for detecting low-altitude obstacles in unmanned aerial vehicles based on machine vision recognition according to claim 5, characterized in that, It also includes S4: continuously caching the effective pixel area images of three frames of time synchronization, namely the current frame, the previous frame, and the two frames before that, and only performing frame-by-frame grayscale value difference operation on all pixels in the non-static obstacle mask area, setting an adaptive illumination difference threshold, and determining a moving candidate pixel when the grayscale difference of the same pixel coordinate exceeds the threshold for multiple consecutive frames. The neighborhood connectivity of moving candidate pixels is checked, the discretely distributed moving candidate pixels are clustered into continuous pixel blocks, the random jitter interference of single pixels is eliminated, independent dynamic target pixel blocks are divided, and dynamic obstacles are identified. S5: For the same edge feature points in the effective pixel area of ​​multiple cameras, match the same feature pixel points in the field of view of different cameras that correspond to the same physical space location. For each successfully matched pair of same feature pixels, calculate the spatial depth distance pixel by pixel and generate a full pixel dense depth mapping table of the effective pixel area to realize that each pixel is bound to a unique spatial depth coordinate. The spatial depth coordinates of the pixels are transformed into the UAV body coordinate system and unified with the coordinate system of the constructed dynamic virtual sphere region. All pixels in the effective pixel area are classified and encoded, and the static contour information of each camera's field of view, the position and motion information of dynamic pixel blocks, and the full pixel depth data are integrated to construct a unified format full pixel static and dynamic fusion feature matrix.

7. The method for detecting low-altitude obstacles in unmanned aerial vehicles based on machine vision recognition according to claim 1, characterized in that, The flight threat classification feedback analysis process for the target UAV is as follows: Based on the full-pixel dynamic-static fusion feature matrix, for each effective pixel (u, v), combined with the spatial depth value of the corresponding pixel, the two-dimensional image pixel coordinates are inversely solved into three-dimensional spatial coordinates in the camera coordinate system, and then mapped to the UAV body coordinate system (Xb, Yb, Zb) through the external parameter matrix coordinate transformation. Complete the coordinate reconstruction of the machine coordinate system of all obstacle pixels to achieve a one-to-one correspondence between image pixels, spatial three-dimensional position, and depth distance. Establish a global obstacle spatial point set and substitute each obstacle three-dimensional spatial point (Xb, Yb, Zb) into the dynamic virtual sphere region equation to perform distance discrimination: output the target to be assessed or the target outside the security domain.

8. The method for detecting low-altitude obstacles in unmanned aerial vehicles based on machine vision recognition according to claim 7, characterized in that, Three core assessment factors are obtained and normalized. The core assessment factors include distance factor Fd, motion state factor Fv, and obstacle type factor Fc. The obstacle threat coefficient is calculated based on the preset weight coefficients corresponding to Fd×Fd, Fv×Fv, and Fc×Fc. The preset obstacle threat coefficient range [Wmin, Wmax] is retrieved for comparison, and low threat, medium threat, or high threat is output. The corresponding early warning intervention decision is matched based on the threat level.

9. A machine vision-based low-altitude obstacle detection system for unmanned aerial vehicles (UAVs), wherein the system is applied to the machine vision-based low-altitude obstacle detection method for UAVs as described in any one of claims 1-8, characterized in that, It includes a dynamic safety zone construction module, a braking capacity calculation module, an effective pixel extraction module, an obstacle recognition and feature fusion module, and a threat classification and obstacle avoidance module.

Images