Unmanned aerial vehicle cooperative control method based on visual formation

Abstract

The invention relates to the technical field of unmanned aerial vehicle formation, in particular to an unmanned aerial vehicle cooperative control method based on visual formation, which comprises a visual identification assisted positioning algorithm and a formation flight control method. The visual identification assisted positioning algorithm comprises a preprocessing system, a target extraction system, a target scale estimation system and a visual continuous tracking correction system, the visual and LiDAR sensors can be utilized to realize assisted height measurement of an unmanned aerial vehicle cluster and positioning and crossing of a target obstacle, and the technical target of obstacle crossing in a complex outdoor environment is achieved. A continuous tracking and positioning algorithm based on monocular vision and a time domain filtering method based on a laser radar are adopted, and the positioning precision of the unmanned aerial vehicle crossing the obstacle target is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of unmanned aerial vehicle formation, and in particular to a cooperative control method of unmanned aerial vehicles based on visual formation. Background Art

[0002] The environment that drones face when flying in formation is complex and changeable. Factors including airflow, air pressure, lighting, and weather may affect the collision avoidance and obstacle avoidance systems.

[0003] Existing technologies may not be able to maintain stable performance in all environments. UAV formation flight requires good communication and coordination mechanisms to ensure the effectiveness of collision avoidance and obstacle avoidance. However, existing communication and coordination technologies may not fully meet the needs of formation flight, especially in the case of communication delays or interruptions, which may cause the collision avoidance and obstacle avoidance system to fail. The collision avoidance and obstacle avoidance system requires a lot of computing and processing power to analyze sensor data, make decisions and control UAVs in real time. However, existing UAVs may not provide sufficient computing and processing power to support complex collision avoidance and obstacle avoidance systems, which may limit the performance and reliability of the system. The perception range and accuracy of existing UAV formation collision avoidance and obstacle avoidance systems may be limited and cannot fully cover all potential obstacles or dangerous situations. This may cause the system to be unable to effectively avoid collisions or dangers in some cases. To this end, we propose a UAV collaborative control method based on visual formation to solve the above problems. Summary of the invention

[0004] The purpose of the present invention is to provide a UAV cooperative control method based on visual formation to solve the problems raised in the above background technology.

[0005] The technical solution of the present invention is: a UAV cooperative control method based on visual formation, wherein the visual recognition assisted positioning algorithm includes a preprocessing system, a target extraction system, a target scale estimation system and a visual continuous tracking correction system; The preprocessing system comprises a non-local average denoising algorithm module, the non-local average denoising algorithm module signal is connected to a histogram processing color enhancement module, and the histogram processing color enhancement module signal is connected to an HSV color space conversion module; The target extraction system comprises a template correlation filter module and a color space filter module, wherein the template correlation filter module and the color space filter module are signal-connected to the same morphological filter module, and the morphological filter module is signal-connected to an edge extraction module; The target scale estimation system includes a multi-frame monocular matching module and a GPS baseline height determination radar baseline module, and the multi-frame monocular matching module and the GPS baseline height determination radar baseline module are signal-connected to the same simulated binocular triangulation positioning to obtain the target scale module; The visual continuous tracking correction system includes a 2D continuous tracking obstacle target module, and the 2D continuous tracking obstacle target module signal is connected to a multi-frame real-time solution crossing point module; The preprocessing system is connected to the target extraction system network, the target extraction system is connected to the target scale estimation system network, and the target scale estimation system is connected to the visual continuous tracking correction system network; The formation flight control method includes the multi-sensor altitude protection algorithm, the time domain filtering method based on laser data, the formation flight safety distance strategy, and the UAV cluster formation logic; The formation flight control method includes a drone, the drone signal is connected to a laser sensor, the drone signal is connected to a visual sensor, a barometer is arranged inside the drone, a GPS signal sensor is arranged inside the drone, and the laser sensor is connected to the GPS signal sensor signal.

[0006] Preferably, the multi-sensor height setting protection algorithm comprises the following steps: Considering that the height information obtained by fusion of barometer and GPS signals has an error of 0.3 to 0.8 m, the height is estimated simultaneously using laser sensor, barometer and GPS signal sensor: (1) 0-16 m: Under normal flight obstacle avoidance conditions, the laser is used as the main reference, and the filtered laser data is used as the altitude measurement value of the UAV relative to the ground; (2) Above 12m: Pixhawk’s internal fused barometer and GPS information are added as protection. If the laser altitude data is detected to deviate from the Pixhawk’s internal relative altitude data by more than 2m, the laser data is considered abnormal and the system automatically switches to the altitude data fused from the Pixhawk’s internal barometer and GPS signals.

[0007] Preferably, the time domain filtering method based on laser data comprises the following steps: By paying more attention to the height of the drone cluster relative to the ground in the low-altitude area, a time domain filtering method is proposed to solve the problem that the laser altitude data has large jitter when facing ground changes, which may easily cause over-adjustment or oscillation feedback in the flight control loop: ; In the formula, R is the resistance factor of real variable parameters. The smaller the R value, the more obvious the smoothing filtering effect. It represents the slope of the laser height data relative to the horizontal displacement at time n-1. for The desired height of the drones that are always in demand, for The desired altitude of the drone is known at all times; is the height of the laser radar at time n-1, is the horizontal displacement of the UAV, and S is defined as the filtering compensation term.

[0008] Preferably, the formation flying safety distance strategy includes the following steps: A straight-line formation was adopted through the formation, with the host in front and the slaves following, providing more reliable and wide visual input information. Taking into account the GPS error factor, the horizontal safety anti-collision area of ​​a single UAV was set to 2.5m, and the horizontal interval between machines was maintained at 5m. In the vertical direction of the ground, considering the laser sensor, the positioning accuracy is relatively high, and the vertical safety anti-collision area of ​​a single UAV is set to 2m. Taking into account the airflow and pressure factors, the vertical safety distance between machines is 4m.

[0009] Preferably, the drone cluster formation logic steps are as follows: Step S1: After the UAV formation issues a take-off command using the airborne ground station, the UAV formation flies in a line to the vicinity of each door frame obstacle and searches for the door frame target; Step S2: using the host's monocular camera and LiDAR laser sensor to locate the obstacle target, calculate the spatial coordinates of the crossing point and the crossing heading angle, and assign the crossing routes of each UAV; Step S3: Use the ground station software to send the final waypoint to other drones, and the drone formation crosses the obstacle and flies in formation to the next obstacle area; Step S4: Complete the crossing task of the four door frames in sequence and return to the take-off point to complete a safe landing.

[0010] The present invention provides a grass-planting mosaic drone formation through improvement, which has the following improvements and advantages compared with the prior art: First, the present invention utilizes vision and LiDAR sensors to realize auxiliary height measurement of drone clusters and positioning and crossing of target obstacles. Aiming at the technical goal of crossing obstacles in complex outdoor environments, it adopts a continuous tracking and positioning algorithm based on monocular vision and a time domain filtering method based on LiDAR, which improves the positioning accuracy and speed of drones crossing obstacles.

[0011] Second, the present invention uses a visual recognition auxiliary positioning algorithm, so that the system can accurately identify the target and estimate its scale. The preprocessing system effectively removes noise and enhances the image, thereby improving the accuracy of recognition. In addition, the use of morphological filtering and edge extraction modules helps to extract the key features of the target, further improving the accuracy and reliability of recognition. The formation flight control method ensures that the drone cluster can fly in a safe and coordinated manner. The multi-sensor altitude protection algorithm uses laser sensors, barometers and GPS signal sensors to improve the altitude estimation accuracy of the drone, thereby enhancing the stability of the flight. The time domain filtering method based on laser data reduces the over-adjustment phenomenon and oscillation feedback of the flight control loop, further improving the stability of the flight. The formation flight safety distance strategy effectively prevents collisions between drones by setting safe anti-collision areas in the horizontal and vertical directions, ensuring the safety of flight. At the same time, this strategy also takes into account the airflow and air pressure environmental factors, so that the drone can better adapt to environmental changes during flight and maintain a stable flight state. The drone cluster formation logic enables the drone formation to be flexibly adjusted according to different mission requirements. By issuing instructions through the ground station, the drone formation can easily change the flight trajectory and crossing route to adapt to different obstacle areas. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] The present invention will be further explained below in conjunction with the accompanying drawings and embodiments: Figure 1 It is a flow chart of the visual recognition assisted positioning algorithm of the present invention; Figure 2 It is a schematic diagram of filtering results under the conditions of continuous and number potholes appearing on the simulated ground of the present invention; Figure 3 It is a schematic diagram of the result of simulating step-type ground filtering on the ground according to the present invention; Figure 4 It is a three-dimensional schematic diagram of the drone swarm flying through obstacles of the present invention; Figure 5 This is a schematic diagram of a drone swarm flying through obstacles of the present invention; DETAILED DESCRIPTION The present invention is described in detail below, and the technical solutions in the embodiments of the present invention are clearly and completely described. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0013] The present invention provides a method for cooperative control of unmanned aerial vehicles based on visual formation through improvement. The technical solution of the present invention is: like Figure 1 - Figure 5 As shown, a UAV cooperative control method based on visual formation, the visual recognition assisted positioning algorithm includes a preprocessing system, a target extraction system, a target scale estimation system and a visual continuous tracking correction system; The preprocessing system includes a non-local average denoising algorithm module, the signal of the non-local average denoising algorithm module is connected to a histogram processing color enhancement module, and the signal of the histogram processing color enhancement module is connected to an HSV color space conversion module; the application of the non-local average denoising algorithm module can effectively remove noise from the image. Compared with the traditional local mean filtering, the non-local mean filtering can better retain the detail information of the image and effectively reduce the influence of noise. By calculating the similarity and performing weighted averaging, the influence of noise can be reduced, making the image clearer and more natural. Such denoising processing provides a higher quality data foundation for subsequent image processing steps. The role of the histogram processing color enhancement module is to enhance the color effect by processing the histogram of the image. The histogram reflects the distribution of pixels of different gray levels in the image. Adjusting the histogram can change the contrast and brightness properties of the image, thereby enhancing the visual effect of the image, which helps to further improve the clarity and detail of the image and provide more accurate data for subsequent color space conversion. The application of the HSV color space conversion module enables the system to take advantage of the HSV color space. The HSV color space is closer to people's perception of color than RGB, and can express the color hue, vividness and brightness very intuitively. In HSV mode, the hue value can be processed separately without affecting the brightness and saturation; or the brightness and saturation can be changed separately without affecting the color itself, so that in image processing, color positioning tracking and color histogram extraction operations can be performed more conveniently. In certain specific applications, such as object recognition and tracking, the HSV color space also has higher accuracy and efficiency; The target extraction system includes a template correlation filtering module and a color space filtering module. The template correlation filtering module and the color space filtering module are signal-connected to the same morphological filtering module, and the morphological filtering module is signal-connected to an edge extraction module. The template correlation filtering module and the color space filtering module each have different filtering characteristics and can effectively suppress different types of image noise and interference. This dual filtering design can significantly improve the filtering ability of the system, making the extracted targets clearer and more accurate. The morphological filtering module can further smooth the image, eliminate small noise and details, and maintain the shape and structural characteristics of the target by uniformly processing the outputs of the template-related filtering and color space filtering. This unified morphological processing helps to improve the consistency and stability of target extraction. The edge extraction module can accurately extract the edge information of the target on the image after morphological filtering, which is crucial for subsequent recognition and tracking tasks, because the edge information can reflect the shape and outline of the target and help distinguish the target from the background. Since the system uses multiple filtering and processing modules, each module can cope with different image conditions and environmental changes to a certain extent. This multiple processing mechanism makes the entire target extraction system more robust in the face of complex image scenes and noise interference. By adjusting the parameters of the template-related filtering module and the color space filtering module, the system can adapt to images of different types and target characteristics. At the same time, the morphological filtering module and the edge extraction module can also be optimized and adjusted as needed to meet specific application requirements. The target scale estimation system includes a multi-frame monocular matching module and a GPS baseline height determination radar baseline module. The multi-frame monocular matching module and the GPS baseline height determination radar baseline module are connected to the same simulated binocular triangulation positioning module for obtaining target scale. Since the system integrates multiple modules, each module can provide information on the target scale from different angles and data sources. This redundant design enables the system to maintain stable operation in the face of complex environments or data loss. By simulating binocular triangulation positioning, the system can overcome the limitations of monocular vision in scale estimation to a certain extent and improve the adaptability and stability of the system. By connecting the signals of the multi-frame monocular matching module, the GPS baseline height determination module and the radar baseline module to the same simulated binocular triangulation positioning module for obtaining target scale, the system can realize the fusion and optimization processing of multi-source information. This fusion method can make full use of the advantages of various modules and make up for each other's shortcomings, thereby improving the estimation performance of the entire system for the target scale. Since the system adopts a modular design, modules can be added or replaced as needed in the future to adapt to different application scenarios and needs. At the same time, the method of simulating binocular triangulation positioning also has certain flexibility. The visual continuous tracking correction system includes a 2D continuous tracking obstacle target module, and the 2D continuous tracking obstacle target module signal is connected to a multi-frame real-time solution crossing point module; the multi-frame real-time solution crossing point module can more accurately determine the position and motion trajectory of the obstacle target by processing continuous multi-frame image data. This accuracy is crucial to achieve stable tracking, especially when the target moves at a fast speed or the scene is more complex. Real-time is one of the important performance indicators of the tracking system. The multi-frame real-time solution crossing point module can complete the processing and analysis of multiple frames of images in a short time, thereby ensuring that the system can track the obstacle target in real time. This is especially important for application scenarios that require rapid response. By obtaining the crossing point information in multiple frames of images, the system can more comprehensively understand the movement trend and possible path of the obstacle target, which is very beneficial for subsequent decision-making and planning processes, such as predicting the future position of the target and planning obstacle avoidance paths. Since the multi-frame image data provides richer information, the system can show stronger robustness when facing noise, occlusion or other interference factors. Even if there is a problem with a frame of image data, the system can still correct and compensate through the data of other frames; The preprocessing system is connected to the target extraction system through a network, the target extraction system is connected to the target scale estimation system through a network, and the target scale estimation system is connected to the visual continuous tracking correction system through a network. The network connection between the systems allows data to flow freely between them, thereby realizing data sharing and collaborative work. This data sharing can reduce the trouble of repeated data entry and conversion, improve work efficiency, and help reduce errors caused by inconsistent data. The network connection enables each system to acquire and process data in real time. The target extraction system can pass the extracted target information to the target scale estimation system in real time, and the latter can immediately perform scale estimation and pass the result to the visual continuous tracking correction system, thereby realizing real-time tracking and correction of the target. The network connection makes this system architecture more flexible and scalable. New functions or modules can be easily added as long as they can be connected to the existing system through the network. Since data flows in the system, the workflow and data processing methods of each system can be flexibly adjusted as needed. This architecture can make full use of the advantages of each system, thereby improving the accuracy and efficiency of the entire system. The target extraction system may be good at extracting targets from complex backgrounds, while the target scale estimation system may be good at accurately estimating the size and position of the target. Connecting them through the network can give full play to their respective advantages, thereby improving the performance of the entire system; The formation flight control method includes a multi-sensor altitude protection algorithm, a time domain filtering method based on laser data, a formation flight safety distance strategy, and a UAV cluster formation logic; the multi-sensor altitude protection algorithm can more accurately control the flight altitude and improve flight stability by integrating data from different sensors, which helps to avoid altitude deviation and potential safety risks during flight. Secondly, the time domain filtering method based on laser data can effectively process the data obtained by the laser sensor, remove noise and interference, and improve the accuracy and reliability of the data, which is crucial for navigation, positioning and target recognition tasks in formation flight. In addition, the formation flight safety distance strategy can ensure that each member of the formation maintains a safe distance to avoid mutual collision and interference, which helps to maintain the overall stability and safety of the formation and ensure the smooth completion of the flight mission. Finally, the UAV cluster formation logic allows the UAV cluster to form a flexible formation to adapt to different environments and mission requirements. By organically combining the perception, calculation, maneuverability and firepower advantages of the UAVs in the cluster, it can expand the scope of action and improve the autonomous coordination and control capabilities of the UAVs within the cluster. The formation flight control method includes a drone, a drone signal is connected to a laser sensor, a drone signal is connected to a visual sensor, a barometer is arranged inside the drone, a GPS signal sensor is arranged inside the drone, and the laser sensor is connected to the GPS signal sensor signal.

[0014] Furthermore, the multi-sensor height setting protection algorithm includes the following steps: Considering that the height information obtained by fusion of barometer and GPS signals has an error of 0.3 to 0.8 m, the height is estimated simultaneously using laser sensor, barometer and GPS signal sensor: (1) 0-16 m: Under normal flight obstacle avoidance conditions, the laser is used as the main reference, and the filtered laser data is used as the altitude measurement value of the UAV relative to the ground; (2) Above 12m: Add the barometer and GPS information fused inside the Pixhawk as protection. If it is detected that the laser altitude data deviates from the relative altitude data value inside the Pixhawk by more than 2m, it is judged that the laser data is abnormal, and it will automatically switch to the altitude data fused by the barometer and GPS signal inside the Pixhawk. Under normal flight obstacle avoidance conditions of 0 to 16m, the laser is used as the main reference. Its high accuracy and fast response speed enable the drone to more accurately measure the relative altitude to the ground. The laser sensor is not affected by GPS signal interference or changes in air pressure, so it can provide stable and reliable altitude information under these conditions. At altitudes above 12m, since the performance of the laser sensor may be limited by environmental factors (such as atmospheric conditions, reflective characteristics of the target surface, etc.), it is very necessary to add the barometer and GPS information fused inside the Pixhawk as protection. This fusion strategy can automatically switch to the laser data when the laser data is abnormal. Other sensor data ensures that the UAV can continue to fly stably. At the same time, this also improves the adaptability and safety of the UAV in complex environments. By selecting the most suitable sensor data for altitude estimation within different altitude ranges, the UAV can better adapt to various flight scenarios. In tasks that require precise altitude control (such as aerial photography, terrain mapping, etc.), the high-precision characteristics of the laser sensor can ensure that the UAV maintains a stable flight altitude. When long-distance flight or crossing complex terrain is required, the fusion data of the barometer and GPS can provide continuous and reliable altitude information, simplifying system maintenance: The strategy of using multiple sensors for altitude estimation also simplifies the system maintenance of the UAV. Since each sensor has its own unique working principle and applicable environment, it may be necessary to calibrate or adjust the sensor for specific scenarios in actual applications. However, by fusing data from multiple sensors, the dependence on a single sensor can be reduced, thereby reducing maintenance workload and improving system reliability.

[0015] Furthermore, the time domain filtering method based on laser data includes the following steps: By paying more attention to the height of the drone cluster relative to the ground in the low-altitude area, a time domain filtering method is proposed to solve the problem that the laser altitude data has large jitter when facing ground changes, which may easily cause over-adjustment or oscillation feedback in the flight control loop: ; In the formula, R is the resistance factor of real variable parameters. The smaller the R value, the more obvious the smoothing filtering effect. It represents the slope of the laser height data relative to the horizontal displacement at time n-1. for The desired height of the drones that are always in demand, for The desired altitude of the drone is known at all times; is the height of the laser radar at time n-1, is the horizontal displacement of the UAV, and S is defined as the filtering compensation term.

[0016] Furthermore, the formation flying safety distance strategy includes the following steps: The formation adopts a one-line solution, with the host in front and the slaves following, providing more reliable and wide visual input information. Taking into account factors such as GPS errors, the horizontal safety anti-collision area of ​​a single drone is set to 2.5m, and the horizontal interval between drones is maintained at 5m. In the vertical ground direction, considering the presence of laser sensors, the positioning accuracy is relatively high, and the vertical safety anti-collision area of ​​a single drone is set to 2m. Taking into account the airflow and air pressure factors, the vertical safety distance between drones is 4m. By setting clear horizontal and vertical safety anti-collision areas and maintaining a safe distance between drones, collisions between drones can be effectively avoided, improving the safety of the entire formation flight. In the one-line formation, the host is in front and the slaves follow. The layout enables each UAV to obtain a wider range of visual input information, which is conducive to improving the accuracy of flight control and navigation. Since GPS signals may be affected by various factors and have certain errors, the impact of GPS errors on flight safety can be reduced by setting a larger horizontal safe anti-collision area and interval between aircraft. In the vertical direction of the ground, laser sensors are used for positioning, and their positioning accuracy is relatively high, which helps UAVs to maintain a safe distance more accurately in the vertical direction, further improving flight safety. Considering the impact of airflow and air pressure factors on flight, setting a larger vertical safety distance between aircraft can help the formation better adapt to complex meteorological conditions and maintain a stable flight state.

[0017] Furthermore, the logical steps of drone swarm formation are as follows: Step S1: After the UAV formation issues a take-off command using the airborne ground station, the UAV formation flies in a line to the vicinity of each door frame obstacle and searches for the door frame target; Step S2: using the host's monocular camera and LiDAR laser sensor to locate the obstacle target, calculate the spatial coordinates of the crossing point and the crossing heading angle, and assign the crossing routes of each UAV; Step S3: Use the ground station software to send the final waypoint to other drones, and the drone formation crosses the obstacle and flies in formation to the next obstacle area; Step S4: Complete the crossing task of 4 door frames in sequence, return to the take-off point and complete a safe landing. The drone formation adopts a straight-line formation flight, which can quickly and effectively reach the vicinity of each door frame obstacle. Through collaborative work, the efficiency of searching and crossing obstacles can be greatly improved. Using the host's monocular camera and LiDAR laser sensor, the drone can accurately locate the obstacle target, which helps to reduce errors in the crossing process and ensure that the drone can accurately find the crossing point and complete the crossing task. By calculating the spatial coordinates and crossing heading angle of the crossing point and allocating the crossing route of each drone, the drone formation can safely and orderly cross the obstacle, avoiding collisions between drones and reducing the contact between drones and door frame obstacles, thereby improving flight safety. The drone formation can flexibly adjust the crossing route and flight strategy according to different door frame obstacles and flight environments. This flexibility enables the drone formation to adapt to a wider variety of scenarios and task requirements. This formation crossing method has strong scalability and can increase or decrease the number of drones as needed to cope with tasks of different scales and complexities.

[0018] Two types of terrain are simulated and analyzed in the simulation: (1) Terrain with continuous potholes and trees. In this type of terrain, it is usually hoped that the filter can filter out intense data jitter and maintain smooth flight; (2) Terrain with step features. Under such terrain conditions, it is usually hoped that the UAV can maintain its relative height to the ground and make a smooth ascent on the steps to reduce the risk of sudden and violent ascents. (3) The simulated flight altitude is 5 m relative to the ground, and the barometer noise is set to Gaussian noise with a standard deviation of 0.20, a variance of 1, and a mean of 0; (4) The laser noise is Gaussian noise with a standard deviation of 0.01, a variance of 1, and a mean of 0. The filtering output results of the simulated laser data under different resistance factors are as follows: Figure 2 Figure 3 shown.

[0019] Depend on Figure 2 It can be seen that as decreases, the filtering effect becomes more and more obvious, and compared with the slowly changing barometer altitude determination, the relative altitude determination data using laser is more stable. Figure 3 It can be seen that with the decrease of , the smoothing effect of the steep change platform becomes more and more obvious. The smooth transition prevents the excessive adjustment of the UAV control loop, which not only saves energy but also makes the flight process safer and smoother.

[0020] The Rviz simulation tool of the ROS (robot operating system) system was used to verify the UAV formation flight logic. Its visualization feature facilitates the analysis of flight logic. Figure 4 and Figure 5 The logical diagram of formation flight and the simulation trajectory under Rviz are given. The red dots are randomly appearing door frame obstacle areas, the yellow dots are the host aircraft, and the black dots are the slave aircraft.

[0021] The above description enables those skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A UAV cooperative control method based on visual formation, including a visual recognition assisted positioning algorithm and a formation flight control method, characterized in that: The visual recognition assisted positioning algorithm includes a preprocessing system, a target extraction system, a target scale estimation system and a visual continuous tracking correction system; The preprocessing system comprises a non-local average denoising algorithm module, the non-local average denoising algorithm module signal is connected to a histogram processing color enhancement module, and the histogram processing color enhancement module signal is connected to an HSV color space conversion module; The target extraction system comprises a template correlation filter module and a color space filter module, wherein the template correlation filter module and the color space filter module are signal-connected to the same morphological filter module, and the morphological filter module is signal-connected to an edge extraction module; The target scale estimation system includes a multi-frame monocular matching module and a GPS baseline height determination radar baseline module, and the multi-frame monocular matching module and the GPS baseline height determination radar baseline module are signal-connected to the same simulated binocular triangulation positioning to obtain the target scale module; The visual continuous tracking correction system includes a 2D continuous tracking obstacle target module, and the 2D continuous tracking obstacle target module signal is connected to a multi-frame real-time solution crossing point module; The preprocessing system is connected to the target extraction system network, the target extraction system is connected to the target scale estimation system network, and the target scale estimation system is connected to the visual continuous tracking correction system network; The formation flight control method includes the multi-sensor altitude protection algorithm, the time domain filtering method based on laser data, the formation flight safety distance strategy, and the UAV cluster formation logic; The formation flight control method includes a drone, the drone signal is connected to a laser sensor, the drone signal is connected to a visual sensor, a barometer is arranged inside the drone, a GPS signal sensor is arranged inside the drone, and the laser sensor is connected to the GPS signal sensor signal.

2. The method for cooperative control of unmanned aerial vehicles based on visual formation according to claim 1, characterized in that: The multi-sensor height setting protection algorithm includes the following steps: Considering that the height information obtained by fusion of barometer and GPS signals has an error of 0.3 to 0.8 m, the height is estimated simultaneously using laser sensor, barometer and GPS signal sensor: (1) 0-16 m: Under normal flight obstacle avoidance conditions, the laser is used as the main reference, and the filtered laser data is used as the altitude measurement value of the UAV relative to the ground; (2) Above 12m: Pixhawk’s internal fused barometer and GPS information are added as protection. If the laser altitude data is detected to deviate from the Pixhawk’s internal relative altitude data by more than 2m, the laser data is considered abnormal and the system automatically switches to the altitude data fused from the Pixhawk’s internal barometer and GPS signals.

3. The method for cooperative control of unmanned aerial vehicles based on visual formation according to claim 1, characterized in that: The time domain filtering method based on laser data comprises the following steps: By paying more attention to the height of the drone cluster relative to the ground in the low-altitude area, a time domain filtering method is proposed to solve the problem that the laser altitude data has large jitter when facing ground changes, which may easily cause over-adjustment or oscillation feedback in the flight control loop: ； In the formula, R is the resistance factor of real variable parameters. The smaller the R value, the more obvious the smoothing filtering effect. It represents the slope of the laser height data relative to the horizontal displacement at time n-1. for The desired height of the drones that are always in demand, for The desired altitude of the drone is known at all times; is the height of the laser radar at time n-1, is the horizontal displacement of the UAV, and S is defined as the filtering compensation term.

4. The method for cooperative control of unmanned aerial vehicles based on visual formation according to claim 1, characterized in that: The formation flight safety distance strategy includes the following steps: A straight-line formation was adopted through the formation, with the main unit in front and the slave units following, to provide more reliable and wide visual input information. Taking into account the GPS error factor, the horizontal safety anti-collision area of ​​a single UAV was set to 2.5m, and the horizontal interval between units was maintained at 5m. In the vertical direction of the ground, considering the presence of laser sensors, the positioning accuracy is relatively high, and the vertical safety anti-collision area of ​​a single UAV is set to 2m. Taking into account the factors of airflow and air pressure, the vertical safety distance between units is 4m.

5. The method for cooperative control of unmanned aerial vehicles based on visual formation according to claim 1, characterized in that: The logical steps of the drone cluster formation are as follows: Step S1: After the UAV formation issues a take-off command using the airborne ground station, the UAV formation flies in a line to the vicinity of each door frame obstacle and searches for the door frame target; Step S2: using the host's monocular camera and LiDAR laser sensor to locate the obstacle target, calculate the spatial coordinates of the crossing point and the crossing heading angle, and assign the crossing routes of each UAV; Step S3: Use the ground station software to send the final waypoint to other drones, and the drone formation crosses the obstacle and flies in formation to the next obstacle area; Step S4: Complete the crossing task of the four door frames in sequence and return to the take-off point to complete a safe landing.

Images