Cluster unmanned aerial vehicle obstacle avoidance method based on millimeter wave radar and binocular vision

Abstract

The invention provides a cluster unmanned aerial vehicle obstacle avoidance method based on millimeter-wave radar and binocular vision, and the method comprises the steps: judging whether an obstacle exists in a current region of an unmanned aerial vehicle or not based on millimeter-wave radar detection data and / or binocular vision images in the flight of one unmanned aerial vehicle in a cluster, so as to control the unmanned aerial vehicle to start an obstacle avoidance strategy. The related position information is synchronously broadcasted to other unmanned aerial vehicles in the cluster; controlling other unmanned aerial vehicles to start an obstacle avoidance strategy based on the real-time position information of other unmanned aerial vehicles; based on the obstacle avoidance strategy execution state of the unmanned aerial vehicle or other unmanned aerial vehicles, the flight state of the unmanned aerial vehicle or other unmanned aerial vehicles is adjusted, obstacle detection is performed by adopting a millimeter-wave radar and binocular vision fusion mode, the sensor setting cost is reduced, and the problem of poor cluster robustness caused by using a single sensor is solved; and meanwhile, cooperative obstacle avoidance of all the unmanned aerial vehicles in the cluster is realized in a manner of sharing position information among the unmanned aerial vehicles, the cluster obstacle avoidance operation and maintenance cost is reduced, and the obstacle avoidance timeliness and effectiveness are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of unmanned aerial vehicles, and in particular to an obstacle avoidance method for clustered unmanned aerial vehicles based on millimeter wave radar and binocular vision. Background Art

[0002] With the continuous development of drone technology, swarm drones are widely used in agriculture, surveying and mapping, power detection and emergency rescue, etc. In these scenarios, swarm drones need to fly in complex environments such as buildings, power lines, mountains and forests. In order to ensure that swarm drones can complete tasks in complex environments, swarm drones need to have the ability to autonomously avoid obstacles.

[0003] At present, there are two main types of obstacle avoidance technologies for cluster drones: active obstacle avoidance based on sensors and active obstacle avoidance based on intra-cluster communication and collaboration. Among them, active obstacle avoidance based on sensors uses lidar sensors to scan the surrounding environment and generate high-precision three-dimensional point cloud data to determine the location, distance, shape and other information of obstacles, so as to plan obstacle avoidance paths based on this information; or uses camera sensors to visually identify the surrounding environment, determine whether there are obstacles and estimate their distance and phase confrontation, so as to achieve active obstacle avoidance. Active obstacle avoidance based on intra-cluster communication and collaboration is to establish a communication link between drones, share their respective positions, speeds and perceived local environmental information in real time, and the entire cluster coordinates to adjust the flight path based on this shared information to avoid collisions with each other and obstacles. However, the lidar sensor itself is expensive and easily affected by bad weather; the camera sensor is limited by the influence of outdoor ambient light and cannot accurately identify objects in low visibility environments. If only a single sensor is set on the drone, the robustness of the cluster drone will be reduced. However, the communication and collaboration method within the cluster is easily affected by the external environment, resulting in problems such as information transmission loss, errors or delays. It cannot guarantee that the drones can obtain accurate obstacle information in a timely manner, affecting the timeliness and effectiveness of obstacle avoidance. At the same time, complex algorithms are required to process information interaction and flight path planning between drones, increasing the operation and maintenance costs of cluster drones. Summary of the invention

[0004] In view of the defects of the prior art, the present invention provides a cluster UAV obstacle avoidance method based on millimeter-wave radar and binocular vision. Based on the millimeter-wave radar detection data and / or binocular vision image of one of the UAVs in the cluster, it is judged whether there is an obstacle in the current area of ​​the UAV, so as to control the UAV to start the obstacle avoidance strategy, and synchronously broadcast the relevant position information to other UAVs in the cluster; based on the real-time position information of other UAVs, the other UAVs are controlled to start the obstacle avoidance strategy; based on the obstacle avoidance strategy execution status of the UAV or other UAVs, the flight status of the UAV or other UAVs is adjusted, and the millimeter-wave radar and binocular vision fusion method is used to detect obstacles, so as to reduce the sensor setting cost and solve the problem of poor cluster robustness caused by using a single sensor; at the same time, by sharing the position information between UAVs, under the condition of small communication volume, the coordinated obstacle avoidance of all UAVs in the cluster is realized, so as to reduce the cluster obstacle avoidance operation and maintenance cost and improve the timeliness and effectiveness of obstacle avoidance.

[0005] The present invention provides a cluster UAV obstacle avoidance method based on millimeter wave radar and binocular vision, comprising: Based on the millimeter-wave radar detection data and / or binocular vision image of one of the UAVs in the cluster, determining whether there is an obstacle in the current area of ​​the UAV; If there is an obstacle in the current area of ​​the drone, the drone is controlled to start an obstacle avoidance strategy and the relevant location information is synchronously broadcast to other drones in the cluster; When the other UAV receives the relevant position information, based on the real-time position information of the other UAV, the other UAV is controlled to start an obstacle avoidance strategy; Based on the obstacle avoidance strategy execution status of the UAV or the other UAV, the flight status of the UAV or the other UAV is adjusted.

[0006] In one embodiment disclosed in the present application, based on the millimeter wave radar detection data and binocular visual image of one of the UAVs in the cluster, before determining whether there is an obstacle in the current area of ​​the UAV, the method includes: Based on the flight mode of one of the UAVs in the cluster, control whether the UAV starts to perform millimeter wave radar detection and binocular vision detection.

[0007] In one embodiment disclosed in the present application, when the drone starts to perform millimeter wave radar detection and binocular vision detection, it includes: Based on the real-time flight status data of the UAV, the execution priority of the millimeter wave radar detection and the binocular vision detection is adjusted.

[0008] In one embodiment disclosed in the present application, based on the millimeter wave radar detection data and / or binocular visual image of one of the drones in flight in the cluster, determining whether there is an obstacle in the current area of ​​the drone includes: Analyze the millimeter wave radar detection data to determine the existence information of the first object in the current area of ​​the drone; Based on the appearance characteristics of the drone, the first object existence information is screened to obtain the first suspicious object information in the current area of ​​the drone; Recognize the binocular visual image to determine the existence information of a second object in the current area of ​​the drone; Based on the image shooting aberration of the drone, the second object existence information is screened to obtain the second suspicious object information in the current area of ​​the drone; Based on the first suspicious object information and / or the second suspicious object information, it is determined whether there is an obstacle in the current area of ​​the drone.

[0009] In one embodiment disclosed in the present application, analyzing the millimeter wave radar detection data to determine the existence information of the first object in the current area of ​​the drone includes: Analyze the millimeter wave radar detection data generated in a continuous number of cycles to obtain the first object existence information corresponding to each cycle; Comparing the first object existence information corresponding to all the periods, identifying the false detection sub-information in the first object existence information corresponding to all the periods; based on the false detection sub-information, converting the first object existence information corresponding to all the periods into the first object existence information in the current area of ​​the drone; as well as, Identifying the binocular visual image to determine the existence information of a second object in the current area of ​​the drone includes: Perform contour recognition on the binocular visual image to obtain three-dimensional contour feature information of the binocular visual image; and determine the existence information of a second object in the current area of ​​the drone based on the three-dimensional contour feature information.

[0010] In one embodiment disclosed in the present application, judging whether there is an obstacle in the current area of ​​the drone based on the first suspicious object information and / or the second suspicious object information includes: Based on the planned flight path of the drone, the first suspicious object information and / or the second suspicious object information are checked for object spatial positions to determine whether there are obstacles in the current area of ​​the drone.

[0011] In one embodiment disclosed in the present application, if there is an obstacle in the current area of ​​the drone, the drone is controlled to start an obstacle avoidance strategy, and the relevant location information is synchronously broadcast to other drones in the cluster, including: If there is an obstacle in the current area of ​​the drone, the drone is controlled to start an obstacle avoidance strategy based on the millimeter-wave radar detection status and binocular vision detection status of the drone; The first positioning information of the UAV and the second positioning information of the obstacle are obtained, the first positioning information and the second positioning information are spatially coordinated to form relevant position information, and the relevant position information is synchronously broadcast to other UAVs in the cluster.

[0012] In one embodiment disclosed in the present application, based on the millimeter wave radar detection state and binocular vision detection state of the drone, controlling the drone to start an obstacle avoidance strategy includes: Determine the difference in detection accuracy between the radar wave detection and binocular vision detection of the drone based on the millimeter radar wave detection state and binocular vision detection state of the drone; Based on the detection accuracy difference, the drone is controlled to start either a millimeter-wave radar-guided obstacle avoidance strategy or a millimeter-wave radar and binocular vision fusion-guided obstacle avoidance strategy.

[0013] In one embodiment disclosed in the present application, when the other drone receives the relevant position information, based on the real-time position information of the other drone, the other drone is controlled to start an obstacle avoidance strategy, including: When the other UAV receives the relevant position information, based on the position relationship between the relevant position information and the real-time position information of the other UAV and the flight dynamic characteristics of the other UAV, determine a suitable position interval for the other UAV to initiate an obstacle avoidance strategy in the planned flight path; When the other UAVs are located in the appropriate position interval, based on the millimeter wave radar detection state and binocular vision detection state of the other UAVs, the other UAVs are controlled to start an obstacle avoidance strategy.

[0014] In one embodiment disclosed in the present application, adjusting the flight state of the drone or the other drone based on the obstacle avoidance strategy execution state of the drone or the other drone includes: Based on the progress of the obstacle avoidance strategy execution of the UAV or the other UAV, updating the millimeter wave radar detection data and binocular vision image of the UAV or the other UAV; Based on the updated millimeter-wave radar detection data and binocular visual images, determine whether the UAV or the other UAV has flown over the obstacle; if so, control the UAV or the other UAV to return to the planned flight path; if not, control the UAV or the other UAV to continue to execute the obstacle avoidance strategy.

[0015] Compared with the existing technology, the cluster drone obstacle avoidance method based on millimeter-wave radar and binocular vision determines whether there is an obstacle in the current area of ​​the drone based on the millimeter-wave radar detection data and / or binocular vision image of one of the drones in flight in the cluster, thereby controlling the drone to start the obstacle avoidance strategy, and synchronously broadcasting the relevant position information to other drones in the cluster; based on the real-time position information of other drones, control other drones to start the obstacle avoidance strategy; also based on the execution status of the obstacle avoidance strategy of the drone or other drones, adjust the flight status of the drone or other drones, use millimeter-wave radar and binocular vision fusion to detect obstacles, reduce the sensor setting cost, and solve the problem of poor cluster robustness caused by using a single sensor; at the same time, by sharing position information between drones, under the condition of small communication volume, collaborative obstacle avoidance of all drones in the cluster can be achieved, reducing the cluster obstacle avoidance operation and maintenance cost and improving the timeliness and effectiveness of obstacle avoidance.

[0016] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention can be realized and obtained by the structures particularly pointed out in the written description, claims, and drawings.

[0017] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0019] Figure 1 A schematic flow chart of a cluster UAV obstacle avoidance method based on millimeter wave radar and binocular vision provided by the present invention. DETAILED DESCRIPTION

[0020] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0021] See also Figure 1 , is a flow chart of a cluster drone obstacle avoidance method based on millimeter wave radar and binocular vision provided by an embodiment of the present invention. The cluster drone obstacle avoidance method based on millimeter wave radar and binocular vision includes: Based on the millimeter-wave radar detection data and / or binocular vision images of one of the drones in the cluster, it is determined whether there are obstacles in the current area of ​​the drone. The cluster includes multiple drones that can fly independently, each of which is equipped with a millimeter-wave radar sensor and a binocular vision sensor, which can perform millimeter-wave radar detection and binocular vision detection on the area where the drone is in flight, preferably the area in front of the flight, and obtain millimeter-wave radar detection data and binocular vision detection data of the corresponding area; and each drone can independently control its own flight parameters, and can also connect to the ground base station to adjust its own flight parameters according to the control instructions from the ground base station; any drone in the cluster can communicate bidirectionally, and any drone can send messages to other drones synchronously in the form of broadcasting to realize information sharing of drones in the cluster.

[0022] If there is an obstacle in the current area of ​​the drone, the drone is controlled to start the obstacle avoidance strategy, and the relevant location information is broadcast to other drones in the cluster simultaneously. When the drone determines that there is an obstacle in the current area through millimeter-wave radar detection and / or binocular vision detection, the drone is controlled to start obstacle avoidance strategies such as deceleration and / or changing the flight path, so that the drone can avoid the obstacle in a timely and stable manner and avoid collision with the obstacle. At the same time, the drone determines the current positioning information through its own positioning sensor, and determines the positioning information of the obstacle based on the millimeter-wave radar detection data and / or binocular vision detection data, and combines the positioning information of the two into relevant location information, so as to share the relevant location information in the form of broadcast and send it to other drones in the cluster, ensuring that the obstacle information detected by any drone in the cluster is obtained in a timely manner, and providing an accurate basis for the coordinated adjustment of the flight status of the cluster drones.

[0023] When other drones receive relevant location information, they control other drones to start obstacle avoidance strategies based on their real-time location information. When other drones receive relevant location information, they can combine their own planned flight paths to predict whether they will enter the obstacle area, so as to adjust their flight parameters in advance. The obstacle detection results of all drones in the cluster are shared within the cluster, so that any drone in the cluster can use the obstacle detection results of other drones as its own obstacle avoidance reference information, ensuring that drones can prepare for obstacle avoidance in advance.

[0024] Adjust the flight status of the drone or other drones based on the execution status of the drone or other drones' obstacle avoidance strategy. When the drones in the cluster start the obstacle avoidance strategy and safely avoid obstacles, adjust the flight path and flight parameters of the drone in time to make the drone return to the original planned flight path, ensuring the continuity of the drone's flight dynamics and the controllability of mission execution.

[0025] The beneficial effects of the above technical solution are as follows: the cluster drone obstacle avoidance method based on millimeter-wave radar and binocular vision determines whether there are obstacles in the current area of ​​the drone based on the millimeter-wave radar detection data and / or binocular vision image of one of the drones in flight in the cluster, thereby controlling the drone to start an obstacle avoidance strategy and synchronously broadcasting the relevant position information to other drones in the cluster; based on the real-time position information of other drones, controlling other drones to start obstacle avoidance strategies; and adjusting the flight status of the drone or other drones based on the execution status of the obstacle avoidance strategy of the drone or other drones, using millimeter-wave radar and binocular vision fusion to detect obstacles, reducing the cost of sensor setting, and solving the problem of poor cluster robustness caused by using a single sensor; at the same time, by sharing position information between drones, under the condition of low communication volume, collaborative obstacle avoidance of all drones in the cluster is achieved, reducing the cluster obstacle avoidance operation and maintenance cost and improving the timeliness and effectiveness of obstacle avoidance.

[0026] Preferably, before judging whether there is an obstacle in the current area of ​​the drone based on the millimeter wave radar detection data and binocular visual image of one of the drones in flight in the cluster, the following steps are included: Based on the flight mode of one of the drones in the cluster, control whether the drone starts to perform millimeter-wave radar detection and binocular vision detection. For example, obtain the flight action parameters of one of the drones in the cluster to determine whether the drone is in take-off mode; if so, send a disable command to the millimeter-wave radar sensor and binocular vision sensor in the drone to control the drone not to start performing millimeter-wave radar detection and binocular vision detection; if not, send a trigger command to the millimeter-wave radar sensor and binocular vision sensor in the drone to control the drone to start performing millimeter-wave radar detection and binocular vision detection; in addition, the trigger command may also include the corresponding working frequency setting command content, so that the millimeter-wave radar sensor and binocular vision sensor in the drone perform periodic millimeter-wave radar detection and binocular vision detection at the corresponding working frequency; the working frequency can be selectively set according to the actual operation of the drone, and is not specifically limited here.

[0027] The beneficial effects of the above technical solution are as follows: each drone in the cluster is equipped with a millimeter-wave radar sensor and a binocular vision sensor; the millimeter-wave radar sensor can perform millimeter-wave radar detection on the current area of ​​the drone (such as the front area), obtain the millimeter-wave data reflected from the current area, and then perform calculation and analysis on the millimeter-wave data to obtain the corresponding millimeter-wave radar detection data; the binocular vision sensor can perform binocular photography on the current area of ​​the drone (such as the front area), obtain the binocular vision image, and then perform recognition and analysis on the binocular vision image to obtain the corresponding binocular vision detection data. During the take-off stage of the drone, the environment around the drone has been checked to ensure that there are no obstacles around the drone that affect take-off. At this time, the drone does not need to start millimeter-wave radar detection and binocular vision detection, which can reduce the drone's power consumption and data processing load.

[0028] Preferably, when the UAV starts to perform millimeter wave radar detection and binocular vision detection, it includes: Based on the real-time flight status data of the drone, the execution priority of millimeter-wave radar detection and binocular vision detection is adjusted. For example, the execution priority of millimeter-wave radar detection and binocular vision detection can be adjusted based on the real-time flight altitude of the drone; specifically, when the real-time flight altitude of the drone is within the preset altitude range, the drone is set to perform millimeter-wave radar detection and binocular vision detection at the same time with the same priority; when the real-time flight altitude of the drone is less than the lower limit of the preset altitude range, the drone is set to prioritize binocular vision detection; when the real-time flight altitude of the drone is greater than the upper limit of the preset altitude range, the drone is set to prioritize millimeter-wave radar detection, thereby setting the execution priority of millimeter-wave radar detection and binocular vision detection in a targeted manner according to the real-time flight environment of the drone, ensuring that the drone can perform accurate obstacle detection under any flight altitude conditions.

[0029] or, When the drone starts to perform millimeter wave radar detection and binocular vision detection, it includes: Based on the hardware status of the UAV, the credibility of the millimeter-wave radar detection data and the binocular vision image is determined to adjust the execution status of the millimeter-wave radar detection and binocular vision detection. The millimeter-wave radar detection and binocular vision detection of the UAV are performed by the millimeter-wave radar sensor and the binocular vision sensor respectively; in the actual working process, the millimeter-wave radar sensor and the binocular vision sensor will inevitably fail, resulting in a certain degree of detection error in the millimeter-wave radar detection and the binocular sight detection. To this end, the working status of the hardware such as the millimeter-wave radar sensor and the binocular vision sensor in the UAV is periodically detected to obtain the hardware status parameters such as the detection sensitivity of the millimeter-wave radar sensor and the binocular vision sensor, so as to determine the credibility of the millimeter-wave radar detection data and the binocular vision image. Generally speaking, the greater the detection sensitivity, the greater the credibility of the corresponding detection result. When the credibility of the millimeter-wave radar detection data or the binocular vision image exceeds the preset credibility threshold, the millimeter-wave radar detection and binocular vision detection are performed; otherwise, the millimeter-wave radar detection or binocular vision detection is not performed, which can reduce the probability of false detection of the millimeter-wave radar detection and binocular vision detection.

[0030] The beneficial effects of the above technical solution are as follows: the millimeter-wave radar sensor and the binocular vision sensor correspond to different detection scenarios and have different detection resolutions. By configuring the millimeter-wave radar sensor and the binocular vision sensor in the drone at the same time, it is not only possible to detect and obtain accurate obstacle data when the drone flies through different scenarios, but also to increase the redundancy of the drone's sensor configuration, ensuring that when the millimeter-wave radar sensor or the binocular vision sensor fails and cannot work, the other sensor can promptly make up for the obstacle detection needs of the drone and improve the system robustness of the drone. In addition, based on the real-time flight status data or hardware status of the drone, adjusting the execution priority of the millimeter-wave radar detection and binocular vision detection or the execution status of the millimeter-wave radar detection and binocular vision detection can select a suitable and reliable obstacle detection method for the drone and improve the reliability of the drone's obstacle avoidance.

[0031] Preferably, judging whether there is an obstacle in the current area of ​​the drone based on the millimeter wave radar detection data and / or binocular vision image of one of the drones in flight in the cluster includes: Analyze the millimeter wave radar detection data to determine the existence information of the first object in the current area of ​​the drone; Based on the appearance characteristics of the drone, the first object existence information is screened to obtain the first suspicious object information in the current area of ​​the drone; Recognize the binocular visual image to determine the existence of a second object in the current area of ​​the drone; Based on the image shooting aberration of the drone, the existence information of the second object is screened to obtain the information of the second suspicious object in the current area of ​​the drone; Based on the first suspicious object information and / or the second suspicious object information, determine whether there are obstacles in the current area of ​​the drone.

[0032] The beneficial effect of the above technical solution is that when the millimeter wave radar sensor in the drone performs millimeter wave radar detection, it scans and transmits millimeter waves to the current area of ​​the drone (such as the front area) and receives the reflected millimeter waves, analyzes the reflected millimeter waves, and generates corresponding millimeter wave radar detection data. Then the millimeter wave radar detection data is analyzed to determine the existence information of the first object in the current area of ​​the drone; wherein the first object existence information may be, but is not limited to, the distance, position, shape and size (such as the horizontal width size and the vertical depth size) of all objects in the current area of ​​the drone. Then, based on the size of the drone itself, the horizontal width threshold and the vertical depth threshold that the drone can safely fly through are set; the shape and size of each object are compared with the set horizontal width threshold and the vertical depth threshold. If the shape and size is greater than or equal to the preset horizontal width threshold or the vertical depth threshold, the corresponding object is determined as the first suspicious object, and the distance, position, shape and size corresponding to the corresponding object are used as the first suspicious object information.

[0033] When the binocular vision sensor in the drone performs binocular vision detection, it takes binocular vision photos of the current area of ​​the drone (such as the area in front) to obtain a binocular vision image; then it performs recognition and analysis on the binocular vision image to determine the existence information of the second object in the current area of ​​the drone; wherein, the existence information of the second object may be, but is not limited to, the distance, position, shape and size (such as horizontal width and vertical depth) of all objects in the current area of ​​the drone. Then, based on the image shooting aberration of the drone, the credibility of each information under the existence information of the second object is determined. If the credibility of a certain information is greater than the preset credibility threshold, the corresponding information is determined as the second suspicious object information, ensuring that the second suspicious object information accurately and comprehensively reflects the spatial distribution information of objects in the current area of ​​the drone that may affect the normal flight of the drone.

[0034] Based on the first suspicious object information and the second suspicious object information, the distance, position, shape and size information corresponding to the same object are merged and corrected, and the global spatial state representation of all objects that may collide with the drone in the current area of ​​the drone is performed from the two aspects of millimeter wave radar detection and binocular vision detection. Then, the merged and corrected suspicious object information is spatially compared with the planned flight path of the drone to determine whether there is a suspicious object on the planned flight path of the drone. If so, it is judged that there is an obstacle in the current area of ​​the drone (that is, there is a suspicious object on the planned flight path); if not, it is judged that there is no obstacle in the current area of ​​the drone, providing a reliable basis for the subsequent launch of the obstacle avoidance strategy.

[0035] Preferably, analyzing the millimeter wave radar detection data to determine the existence information of the first object in the current area of ​​the drone includes: Analyze the millimeter wave radar detection data generated in a continuous number of cycles to obtain the first object existence information corresponding to each cycle; Compare the first object existence information corresponding to all the periods, and identify the false detection sub-information in the first object existence information corresponding to all the periods; based on the false detection sub-information, convert the first object existence information corresponding to all the periods into the first object existence information in the current area of ​​the drone; as well as, Identify the binocular visual image to determine the existence of the second object in the current area of ​​the drone, including: Perform contour recognition on the binocular vision image to obtain three-dimensional contour feature information of the binocular vision image; based on the three-dimensional contour feature information, determine the existence information of the second object in the current area of ​​the drone.

[0036] The beneficial effects of the above technical solution are as follows: the millimeter wave radar sensor is easily affected by the adverse environmental conditions of the outside world. If only a single millimeter wave radar detection is performed on the current area of ​​the drone during the flight of the drone, the corresponding millimeter wave radar detection data will have a large detection error. In order to improve the accuracy and reliability of millimeter wave radar detection, it is necessary to control the millimeter wave radar sensor to perform multiple detections on the current area of ​​the drone, analyze the millimeter wave radar detection data generated for several consecutive cycles, and obtain the first object existence information corresponding to each cycle. Then, the object distance, position, shape and size sub-information of the first object existence information corresponding to all cycles are compared with the same sub-information, and the difference corresponding to the same sub-information in all cycles is obtained, so as to identify the false detection sub-information in the first object existence information corresponding to all cycles. For example, if the difference between the distance detected corresponding to a certain cycle and the distance detected corresponding to all other cycles exceeds the preset difference threshold, the distance detected corresponding to a certain cycle is determined as the false detection sub-information. In fact, comparing the first object existence information corresponding to all cycles and identifying the false detection sub-information in the first object existence information corresponding to all cycles belongs to the conventional technical means in the field of data statistics, which will not be introduced in detail here.

[0037] In addition, the binocular vision sensor in the drone will perform binocular vision shooting of the current area of ​​the drone, especially the front area, to obtain a binocular vision image of the current area, and then perform contour recognition on the binocular vision image to obtain the three-dimensional contour feature information of the binocular vision image; the three-dimensional contour feature information represents the edge or surface three-dimensional contour information of all objects in the binocular vision image. Based on the three-dimensional contour feature information, the distance, position, shape and size of all objects in the current area of ​​the drone are characterized to obtain the existence information of the second object in the current area of ​​the drone.

[0038] In addition, analyzing the millimeter-wave radar detection data generated in a number of consecutive cycles to obtain the first object existence information corresponding to each cycle; comparing the first object existence information corresponding to all cycles to identify the false detection sub-information in the first object existence information corresponding to all cycles; based on the false detection sub-information, converting the first object existence information corresponding to all cycles into the first object existence information in the current area of ​​the drone may include: 1. Setting thresholds and preliminary screening Determine the longitudinal threshold L according to the size of the UAV platform y and the lateral threshold L x ; For example, for small drones, the longitudinal threshold can be set to L y = 2 meters, the lateral threshold is set to L x= 1.5 meters; Based on the size of the drone itself, exclude those detection points that are too close or too far from the drone, and those that deviate too much from the drone's flight plane, and preliminarily screen out the possible millimeter wave detection target set N mmb .

[0039] 2. Filtering algorithm processing 2.1 State vector definition: For each detection target, define the state vector , where x and y are the position coordinates of the target on a two-dimensional plane (relative coordinate plane with the drone as the origin), v x and v y are the speeds of the target in the x and y directions.

[0040] 2.2 State transfer matrix: Define the state transfer matrix F as: ;

[0041] in is the time interval of the millimeter-wave radar detection cycle; this matrix describes the linear relationship between the detection target state in two adjacent detection cycles, that is, based on the uniform motion model, predicts the position and speed of the detection target in the next cycle.

[0042] 2.3 Observation Matrix: The observation matrix H is defined as: ;

[0043] Because the millimeter-wave radar directly observes the position information (x, y) of the detected target, while the speed information is indirectly estimated through the state transfer model.

[0044] 2.4 Process noise covariance matrix Q and observation noise covariance matrix R: According to the measurement accuracy of the millimeter-wave radar and the uncertainty of the target motion, set the appropriate process noise covariance matrix Q and observation noise covariance matrix R. For example, for a millimeter-wave radar with higher accuracy, the observation noise covariance matrix can be set as: ;

[0045] in, and is the standard deviation of the observation noise in the x and y directions, which can be obtained from the radar's technical parameters and experimental calibration; The process noise covariance matrix Q is set according to factors such as the possible acceleration variation range of the target, as follows: ;

[0046] Among them, q x and q y It is a parameter related to the acceleration noise of the detection target in the x and y directions.

[0047] 2.5 Prediction and Update Prediction Steps: ; ;

[0048] in, is the predicted value of the k-th period state based on the k-1-th period state estimate, is the corresponding forecast covariance matrix; Update steps: ; ; ;

[0049] Among them, K k is the gain matrix, Z k is the observation vector of the kth period (i.e., the target position detected by the millimeter-wave radar), is the state estimate after observation update in the kth period, is the corresponding updated covariance matrix.

[0050] After filtering, we get the filtered state estimation value set N of each potential target in each cycle. mmb-f .

[0051] 3. Criteria for determining the same obstacle 3.1 Position distance determination: For two targets in three consecutive periods k-2, k-1, k, calculate the Euclidean distance d between their filtered positions ij (where i and j represent different targets). Let the position of target i in the kth cycle be (x ik ,y ik ), the position of target j in the kth cycle is (x jk ,y jk )but: ; If within three consecutive cycles, , and are all less than a set position matching threshold (for example =0.5 m), the two targets may be the same obstacle.

[0052] 3.2 Speed ​​consistency determination: Calculate the angle between the velocity vectors of two targets in three cycles ; Let the velocity vector of target i be v i =(v ix , v iy ), the velocity vector of target j is v j =(v jx , v jy ), then the cosine of the velocity vector angle is: ;

[0053] if The speed direction is greater than a set matching threshold for three consecutive cycles. (for example = 0.996, indicating that the velocity direction angle is less than about 5 degrees, and the relative difference in the speed of the two targets (i.e. ) is less than a set speed matching threshold for three consecutive cycles (for example = 0.1), further increasing the possibility that the two targets are the same obstacle.

[0054] 3.3 Comprehensive position distance determination and speed consistency determination: If a target and other targets meet the above conditions in three consecutive cycles, they are judged as the same obstacle and marked as valid targets, forming the final valid target set N mmb-A .

[0055] 4. False Detection Target Processing Detection statistics: For each potential target, count the number of times M that it is judged as a valid target within a certain time window (for example, including multiple consecutive groups of three cycles) mmb ; Threshold determination: If M mmb Less than a set minimum detection threshold (For example, =5), the target is considered to be a misdetected target and is removed from the valid target set N mmb-A Removed; Through the above steps, the filtering algorithm and judgment criteria can be used more accurately to screen out valid targets from multiple cycles of millimeter-wave radar detection data, and eliminate misdetected targets, providing reliable target information for the obstacle avoidance decision of the UAV.

[0056] Preferably, judging whether there is an obstacle in the current area of ​​the drone based on the first suspicious object information and / or the second suspicious object information includes: Based on the planned flight path of the drone, the spatial position of the first suspicious object information and / or the second suspicious object information is checked to determine whether there are obstacles in the current area of ​​the drone.

[0057] The beneficial effect of the above technical solution is that all drones in the cluster will fly according to the preset planned flight path during the execution of the mission. If the object is located in the planned flight path of the drone and its adjacent space area, the drone will collide with the object, that is, the object exists as an obstacle in the current area of ​​the drone. Specifically, based on the first suspicious object information and the second suspicious object information, the distance, position, shape, size and other information corresponding to the same object are merged and corrected, and the global spatial state representation of all objects that may collide with the drone in the current area of ​​the drone is performed from the two aspects of millimeter wave radar detection and binocular vision detection, that is, the distance, position, shape, size and other information about the same object in the first suspicious object information and the second suspicious object information are merged and the data error is corrected to obtain the unique corresponding reliable distance, position, shape, size and other information about the same object as the merged and corrected suspicious object information. The merged and corrected suspicious object information is then spatially compared with the planned flight path of the UAV to determine whether there is a suspicious object on the planned flight path of the UAV. If so, it is judged that there is an obstacle in the current area of ​​the UAV (that is, there is a suspicious object on the planned flight path); if not, it is judged that there is no obstacle in the current area of ​​the UAV, providing a reliable basis for the subsequent activation of the obstacle avoidance strategy.

[0058] Preferably, if there is an obstacle in the current area of ​​the drone, the drone is controlled to start an obstacle avoidance strategy and the relevant location information is synchronously broadcast to other drones in the cluster, including: If there are obstacles in the current area of ​​the drone, the drone is controlled to start the obstacle avoidance strategy based on the drone's millimeter-wave radar detection status and binocular vision detection status; if there are no obstacles in the current area of ​​the drone, the drone is controlled to continue flying along the planned flight path, and the millimeter-wave radar sensor and binocular vision sensor in the drone are controlled to continue detecting the area in front of the drone; The first positioning information of the UAV and the second positioning information of the obstacle are obtained, the first positioning information and the second positioning information are spatially unified to form relevant position information, and the relevant position information is synchronously broadcast to other UAVs in the cluster.

[0059] The beneficial effects of the above technical solution are: if there are obstacles in the current area of ​​the drone, it is necessary to control the drone to start the obstacle avoidance strategy and actively avoid the obstacles in the current area. At this time, it is necessary to use the millimeter wave radar sensor and binocular vision sensor in the drone to perform close-range and wide-range detection of the spatial area within the predetermined distance range in front of the drone, obtain the dynamic changes in the relative distance and relative orientation between the drone and the obstacles in the current area, control the drone to gradually adjust and change its own flight state parameters, and achieve stable flight of the drone to avoid obstacles. In addition, obstacles in the current area of ​​the drone may affect other drones in the cluster. If the planned flight paths of other drones in the cluster overlap with the above obstacles, other drones will collide with the above obstacles. To this end, the first positioning information of the drone and the second positioning information of the obstacle are unified in the same spatial coordinate system to obtain relevant position information (that is, the relevant position information includes the position coordinates of the first positioning information and the second positioning information in the same spatial coordinate system), for example, unified into the world coordinate system corresponding to the cluster, and then the unified relevant position information is broadcast to other drones in the cluster through a dedicated wireless communication channel within the cluster, so as to serve as a basis for other drones to avoid obstacles in the corresponding spatial area. In this way, the shared transmission of position information can be realized within the cluster while occupying less communication resources, thereby ensuring the timeliness of the acquisition of position information of all drones in the cluster.

[0060] Preferably, based on the millimeter wave radar detection state and binocular vision detection state of the drone, the drone is controlled to start an obstacle avoidance strategy, including: Based on the UAV's millimeter radar wave detection status and binocular vision detection status, determine the detection accuracy difference between the UAV's radar wave detection and binocular vision detection; Based on the difference in detection accuracy, the drone is controlled to start either the millimeter-wave radar-guided obstacle avoidance strategy or the millimeter-wave radar and binocular vision fusion-guided obstacle avoidance strategy.

[0061] The beneficial effects of the above technical solution are: the UAV actively avoids obstacles in the current area. It needs to gradually adjust and change its own flight state parameters (such as flight altitude, flight speed and flight attitude angle, etc.) based on the distance, position, shape and size of the obstacles without seriously deviating from the planned flight path, so as to avoid the UAV's flight safety being affected by excessive changes in its own flight state in a short period of time. In order to meet the requirement of the UAV to gradually adjust and change its own flight state parameters, it is necessary to use the millimeter-wave radar sensor and / or binocular vision sensor in the UAV to obtain the actual obstacle situation in the area ahead of the UAV in real time, that is, the UAV can gradually change its own flight state parameters under the guidance of the millimeter-wave radar sensor and / or binocular vision sensor to smoothly avoid obstacles. In order to improve the accuracy and reliability of obstacle avoidance of UAV, the detection accuracy difference between radar wave detection and binocular vision detection of UAV is determined based on the millimeter wave radar wave detection state and binocular vision detection state of UAV. If the above detection accuracy difference indicates that the radar wave detection accuracy is higher than the binocular vision detection accuracy, the UAV is controlled to start the millimeter wave radar guided obstacle avoidance strategy (that is, only adjust its own flight state parameters according to the data detected by the millimeter wave radar sensor); if the above detection accuracy difference indicates that the radar wave detection accuracy is lower than the binocular vision detection accuracy, the UAV is controlled to start the binocular vision guided obstacle avoidance strategy (that is, only adjust its own flight state parameters according to the data detected by the binocular vision sensor); if the above detection accuracy difference indicates that the radar wave detection accuracy is equal to the binocular vision detection accuracy, the UAV is controlled to start the millimeter wave radar and binocular vision fusion guided obstacle avoidance strategy (that is, adjust its own flight state parameters according to the data detected by the millimeter wave radar sensor and the binocular vision sensor at the same time). In addition, the drone may change its own flight state parameters (such as flight altitude, flight speed, and flight attitude angle, etc.), including but not limited to increasing or decreasing the flight altitude, reducing the flight speed, or changing at least one of the flight pitch angle, yaw angle, and roll angle. The specific form depends on the relative position and relative orientation dynamic changes between the drone and the obstacle, and will not be described in detail here.

[0062] Alternatively, based on the millimeter-wave radar detection status and binocular vision detection status of the drone, controlling the drone to start the obstacle avoidance strategy can also be: based on the respective performance states of the millimeter-wave radar sensor and binocular vision sensor of the drone, controlling the drone to preferentially execute the millimeter-wave radar-guided obstacle avoidance strategy or the binocular vision-guided obstacle avoidance strategy, the process is as follows: Set the UAV's flight mission category variable T, When the UAV is in a medium-to-high altitude flight mission, T=1; when the UAV is in a low-altitude flight mission, T=2; when the UAV is in a take-off mission, T=3; Set the height variable (in meters), which is the current flight height of the drone. The lower limit of the altitude of the drone in the mid-altitude flight mission is defined as H min-high (For example, H min-high =50m); the upper limit of the altitude of the UAV in low-altitude flight mission is H max-low (For example, H max-low =50m), the lower limit of the altitude for low-altitude flight mission is H min-low (For example, H min-low =1m); Set the obstacle distance variable (in meters), which is the straight-line distance between the drone and the nearest obstacle. Define the safety distance threshold as D safe (For example, D safe =5m), safety distance threshold D safe The speed of the drone is V drone related; Set sensor performance variables, For millimeter wave radar sensors, S mmb It represents the comprehensive performance index of its effective detection range (in meters) and accuracy (value range 0-1, 1 represents the highest accuracy) in the current environment, such as S mmb =Effective detection range × accuracy; For binocular vision sensors, S sm It represents the comprehensive performance index of its effective recognition range (in meters), accuracy (value range 0-1, 1 represents the highest accuracy) and ability to recognize complex obstacles (value range 0-1) in the current environment, such as S mmb =Effective recognition range × accuracy × complex shape recognition capability; The priority is calculated as follows: (1) Task category basic priority P T When T=1, , where k1 is the adjustment coefficient, for example k1=0.1; When T=2, , where k2 and k3 are adjustment coefficients, for example, k2=0.2, k3=0.15; (2) Obstacle distance priority P D When D≤D safe hour, , where k D1 is the adjustment coefficient, such as k D1 =0.2; When D>D safe hour, , where k D2 is the adjustment coefficient, such as k D2 =0.1; (3) Sensor performance priority For millimeter wave radar sensors, , where k S1 Adjustment coefficient, such as k S1 =0.2; For binocular vision sensors, , where k S2 is the adjustment coefficient, such as k S2 =0.2; (4) Comprehensive priority For the millimeter wave radar obstacle avoidance strategy, P mmb =P T ×P D ×P S-mmb ; For binocular vision obstacle avoidance strategy, P sm =P T ×P D ×P S-sm .

[0063] By comparing P mmb and P sm The size of determines the execution priority of the obstacle avoidance strategy. mmb >P sm , the millimeter-wave radar-guided obstacle avoidance strategy is executed first; if P mmb <P sm The binocular vision-guided obstacle avoidance strategy is executed first. mmb With P sm If the values ​​are equal, the obstacle avoidance strategy guided by the fusion of millimeter-wave radar and binocular vision is executed; in addition, in special stages such as takeoff and landing, the priority can be further adjusted according to the special rules and other factors in the mission process, or the obstacle avoidance strategy can be chosen not to be executed, which will not be described in detail here. Through this comprehensive multi-factor quantitative calculation method, the execution priority of the obstacle avoidance strategy of cluster drones can be determined more accurately, and the adaptability and safety of cluster drones in complex flight scenarios can be improved.

[0064] Preferably, when other drones receive relevant position information, based on the real-time position information of other drones, other drones are controlled to start obstacle avoidance strategies, including: When other UAVs receive the relevant position information, based on the position relationship between the relevant position information and the real-time position information of other UAVs and the flight dynamic characteristics of other UAVs, determine the appropriate position interval for other UAVs to start the obstacle avoidance strategy in the planned flight path; When other drones are in the appropriate position range, based on the millimeter-wave radar detection status and binocular vision detection status of other drones, other drones are controlled to start obstacle avoidance strategies.

[0065] The beneficial effects of the above technical solution are: when other drones receive relevant location information, the above relevant location information is compared with the real-time location information of other drones to determine the relative distance and relative azimuth between the two, and combined with the flight dynamic characteristics of other drones such as flight altitude, flight speed and flight attitude angle, predict the appropriate position interval of other drones in their planned flight paths to start obstacle avoidance strategies in order to avoid obstacles, so that other drones have enough space distance and time to implement obstacle avoidance strategies in advance, and improve the effectiveness and reliability of other drones' real-time obstacle avoidance strategies. Also, when other drones are in the appropriate position interval, based on the millimeter-wave radar detection status and binocular vision detection status of other drones, control other drones to start obstacle avoidance strategies, and the process of other drones starting obstacle avoidance strategies is the same as the process of drones starting obstacle avoidance strategies introduced above, and will not be repeated here.

[0066] Preferably, adjusting the flight state of the drone or other drone based on the obstacle avoidance strategy execution state of the drone or other drone includes: Based on the progress of the obstacle avoidance strategy execution of the UAV or other UAV, updating the millimeter wave radar detection data and binocular vision image of the UAV or other UAV; Based on the updated millimeter-wave radar detection data and binocular vision images, it is determined whether the UAV or other UAVs have flown over the obstacle; if so, the UAV or other UAVs are controlled to return to the planned flight path; if not, the UAV or other UAVs are controlled to continue to execute the obstacle avoidance strategy.

[0067] The beneficial effects of the above technical solution are: in the process of each UAV (including UAVs and other UAVs) in the cluster executing the obstacle avoidance strategy, the execution progress of the obstacle avoidance strategy is monitored in real time, so as to periodically update the millimeter wave radar detection data and binocular visual images of the UAV or other UAVs, that is, in the process of executing the obstacle avoidance strategy, the millimeter wave radar detection data and binocular visual images of the UAV are periodically updated at the execution frequency, so as to regularly update the relative position change between the UAV itself and the obstacle during the process of executing the obstacle avoidance strategy. Then, the updated millimeter wave radar detection data and binocular visual images are analyzed to determine whether the UAV has flown over the obstacle (that is, whether it has flown to avoid the obstacle and gradually moved away from the obstacle); if so, the UAV is controlled to return to the original planned flight path; if not, the UAV is controlled to continue to execute the obstacle avoidance strategy, so as to ensure the flight controllability of the UAV.

[0068] It can be seen from the contents of the above embodiments that the cluster drone obstacle avoidance method based on millimeter-wave radar and binocular vision determines whether there is an obstacle in the current area of ​​the drone based on the millimeter-wave radar detection data and / or binocular vision image of one of the drones in flight in the cluster, thereby controlling the drone to start an obstacle avoidance strategy and synchronously broadcasting the relevant position information to other drones in the cluster; based on the real-time position information of other drones, controlling other drones to start obstacle avoidance strategies; and also based on the execution status of the obstacle avoidance strategy of the drone or other drones, adjusting the flight status of the drone or other drones, using millimeter-wave radar and binocular vision fusion to detect obstacles, reducing the cost of sensor setting, and solving the problem of poor cluster robustness caused by using a single sensor; at the same time, by sharing position information between drones, under the condition of low communication volume, collaborative obstacle avoidance of all drones in the cluster is achieved, reducing the cluster obstacle avoidance operation and maintenance cost and improving the timeliness and effectiveness of obstacle avoidance.

[0069] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A cluster UAV obstacle avoidance method based on millimeter wave radar and binocular vision, comprising: Based on the millimeter-wave radar detection data and / or binocular vision image of one of the UAVs in the cluster, determining whether there is an obstacle in the current area of ​​the UAV; If there is an obstacle in the current area of ​​the drone, the drone is controlled to start an obstacle avoidance strategy and the relevant location information is synchronously broadcast to other drones in the cluster; When the other UAV receives the relevant position information, based on the real-time position information of the other UAV, the other UAV is controlled to start an obstacle avoidance strategy; Based on the obstacle avoidance strategy execution status of the UAV or the other UAV, the flight status of the UAV or the other UAV is adjusted.

2. The method for swarm drone obstacle avoidance based on millimeter wave radar and binocular vision as claimed in claim 1, characterized in that: Based on the millimeter wave radar detection data and binocular visual image of one of the UAVs in the cluster, before determining whether there is an obstacle in the current area of ​​the UAV, the method includes: Based on the flight mode of one of the UAVs in the cluster, control whether the UAV starts to perform millimeter wave radar detection and binocular vision detection.

3. The method for avoiding obstacles of clustered UAVs based on millimeter wave radar and binocular vision as claimed in claim 2, characterized in that: When the UAV starts to perform millimeter wave radar detection and binocular vision detection, it includes: Based on the real-time flight status data of the UAV, the execution priority of the millimeter wave radar detection and the binocular vision detection is adjusted.

4. The method for swarm drone obstacle avoidance based on millimeter wave radar and binocular vision as claimed in claim 1, characterized in that: Based on the millimeter wave radar detection data and / or binocular vision image of one of the UAVs in the cluster, determining whether there is an obstacle in the current area of ​​the UAV includes: Analyze the millimeter wave radar detection data to determine the existence information of the first object in the current area of ​​the drone; Based on the appearance characteristics of the drone, the first object existence information is screened to obtain the first suspicious object information in the current area of ​​the drone; Recognize the binocular visual image to determine the existence information of a second object in the current area of ​​the drone; Based on the image shooting aberration of the drone, the second object existence information is screened to obtain the second suspicious object information in the current area of ​​the drone; Based on the first suspicious object information and / or the second suspicious object information, it is determined whether there is an obstacle in the current area of ​​the drone.

5. The method for avoiding obstacles of swarm UAVs based on millimeter wave radar and binocular vision as claimed in claim 4, characterized in that: Analyzing the millimeter wave radar detection data to determine the existence information of the first object in the current area of ​​the drone includes: Analyze the millimeter wave radar detection data generated in a continuous number of cycles to obtain the first object existence information corresponding to each cycle; Comparing the first object existence information corresponding to all the periods, identifying the false detection sub-information in the first object existence information corresponding to all the periods; based on the false detection sub-information, converting the first object existence information corresponding to all the periods into the first object existence information in the current area of ​​the drone; as well as, Identifying the binocular visual image to determine the existence information of a second object in the current area of ​​the drone includes: Perform contour recognition on the binocular visual image to obtain three-dimensional contour feature information of the binocular visual image; and determine the existence information of a second object in the current area of ​​the drone based on the three-dimensional contour feature information.

6. The method for avoiding obstacles of swarm UAVs based on millimeter wave radar and binocular vision as claimed in claim 4, characterized in that: Based on the first suspicious object information and / or the second suspicious object information, determining whether there is an obstacle in the current area of ​​the drone includes: Based on the planned flight path of the drone, the first suspicious object information and / or the second suspicious object information are checked for object spatial positions to determine whether there are obstacles in the current area of ​​the drone.

7. The method for swarm drone obstacle avoidance based on millimeter wave radar and binocular vision as claimed in claim 1, characterized in that: If there is an obstacle in the current area of ​​the drone, the drone is controlled to start an obstacle avoidance strategy and the relevant location information is synchronously broadcast to other drones in the cluster, including: If there is an obstacle in the current area of ​​the drone, the drone is controlled to start an obstacle avoidance strategy based on the millimeter-wave radar detection status and binocular vision detection status of the drone; The first positioning information of the UAV and the second positioning information of the obstacle are obtained, the first positioning information and the second positioning information are spatially coordinated to form relevant position information, and the relevant position information is synchronously broadcast to other UAVs in the cluster.

8. The method for swarm drone obstacle avoidance based on millimeter wave radar and binocular vision as claimed in claim 7, characterized in that: Based on the millimeter wave radar detection state and binocular vision detection state of the drone, the drone is controlled to start an obstacle avoidance strategy, including: Determine the difference in detection accuracy between the radar wave detection and binocular vision detection of the drone based on the millimeter radar wave detection state and binocular vision detection state of the drone; Based on the detection accuracy difference, the drone is controlled to start either a millimeter-wave radar-guided obstacle avoidance strategy or a millimeter-wave radar and binocular vision fusion-guided obstacle avoidance strategy.

9. The method for swarm drone obstacle avoidance based on millimeter wave radar and binocular vision as claimed in claim 1, characterized in that: When the other UAV receives the relevant position information, based on the real-time position information of the other UAV, controlling the other UAV to start an obstacle avoidance strategy includes: When the other UAV receives the relevant position information, based on the position relationship between the relevant position information and the real-time position information of the other UAV and the flight dynamic characteristics of the other UAV, determine a suitable position interval for the other UAV to initiate an obstacle avoidance strategy in the planned flight path; When the other UAVs are located in the appropriate position interval, based on the millimeter wave radar detection state and binocular vision detection state of the other UAVs, the other UAVs are controlled to start an obstacle avoidance strategy.

10. The method for swarm drone obstacle avoidance based on millimeter wave radar and binocular vision as claimed in claim 1, characterized in that: Adjusting the flight state of the drone or the other drone based on the obstacle avoidance strategy execution state of the drone or the other drone includes: Based on the progress of the obstacle avoidance strategy execution of the UAV or the other UAV, updating the millimeter wave radar detection data and binocular vision image of the UAV or the other UAV; Based on the updated millimeter-wave radar detection data and binocular visual images, determine whether the UAV or the other UAV has flown over the obstacle; if so, control the UAV or the other UAV to return to the planned flight path; if not, control the UAV or the other UAV to continue to execute the obstacle avoidance strategy.

Images