Vision-based unmanned aerial vehicle swarm formation recognition method

Abstract

The invention discloses an unmanned aerial vehicle bee colony formation recognition method based on vision, aiming at the specific requirements of anti-unmanned aerial vehicle bee colony system formation recognition, an unmanned aerial vehicle image data set construction technology, a Yolov5-based unmanned aerial vehicle target detection technology and a Resnet50-based unmanned aerial vehicle formation classification technology are adopted, so that a photoelectric system can obtain an unmanned aerial vehicle bee colony image after the unmanned aerial vehicle bee colony image is obtained; through the steps of unmanned aerial vehicle image extraction, image binaryzation, formation classification and the like, the unmanned aerial vehicle formation is accurately and quickly identified online, so that the threat degree of the unmanned aerial vehicle formation is judged. The method is easy to implement, low in cost and intelligent, and the problem that a traditional photoelectric system cannot automatically recognize the swarm formation of the unmanned aerial vehicle is solved.

Description

Technical Field

[0001] The present invention belongs to the technical field of anti-UAV swarm, and mainly relates to a UAV swarm formation recognition method based on vision. Background Art

[0002] Drone swarms have the functions of ultra-low-altitude flight, reorganization, and intelligent obstacle avoidance. They will adopt different formations when performing different tasks, such as circular formations for escorting high-value targets, wedge formations for inspections, and vertical formations for attacks. Identifying drone formations can effectively identify their combat modes and judge their threat levels, thereby selecting targets with high tactical value for counterattack, thereby improving the effectiveness and cost-effectiveness of anti-swarm missions. Therefore, the ability to identify formations and select targets is very important for anti-swarm missions.

[0003] Zhou Qiang from China Electronics Technology Group Corporation first used a lightweight model combining SSD and LeNet to detect drone formations. However, this method cannot cope with drone images with complex backgrounds in actual combat missions. There is little research on drone swarm formation recognition in existing technologies, and related research is mostly focused on drone target detection.

[0004] The YOLOv5 algorithm was publicly released in 2020 and has higher detection accuracy and speed than SSD. Resnet has a deeper network structure, so it can better extract image features. The present invention adopts drone image dataset construction technology, Yolov5-based drone target detection technology, and Resnet50-based drone formation classification technology, so that the optoelectronic system can obtain drone swarm images through drone image extraction, image binarization, formation classification and other steps, and can accurately and quickly identify drone swarm formations online. Summary of the invention

[0005] 1. Technical issues to be resolved

[0006] The technical problem to be solved by the present invention is that the traditional photoelectric detection method cannot effectively identify the formation of UAVs.

[0007] (II) Technical solution

[0008] In order to solve the above technical problems, the present invention proposes a method for recognizing drone swarm formation based on vision, which comprises the following steps:

[0009] Step 1: Construction of drone image datasets, mainly including drone target image data, drone formation image data and other real and simulated datasets. The real datasets are mainly obtained by web crawlers and field test shooting. The simulation method mainly adopts the behavior formation control method based on the Flocking obstacle avoidance algorithm, and the drone swarm flight formation is constructed through software simulation to obtain the formation database.

[0010] Step 2: Use YOLOv5 to train the drone image to obtain the drone target detection model. The purpose of this step is to detect all drone targets in the image and give their detection box coordinates in the image.

[0011] Step 3: Based on Resnet50, the drone formation data is trained to obtain a drone formation classification model. This classification model can classify drone formation images into wedges, longitudinal rows, transverse rows, rings, snakes, trapezoids, etc.

[0012] Step 4: Separate all drones detected in the target detection model image in step 2 from the background image, set them to black and white respectively, and classify them through the formation classification model in step 3 to obtain the classification result of the drone formation.

[0013] (III) Beneficial effects

[0014] The beneficial effects of the present invention are embodied in the following aspects:

[0015] (1) High degree of automation and easy to implement

[0016] This can be achieved by modifying the software under the existing optoelectronic platform. The entire formation recognition process is automatically completed by the software and can quickly act on a large number of drone swarms.

[0017] (2) Improving the efficiency of anti-swarm missions

[0018] This method establishes a dataset of drone images and formations, and quickly detects, segmentes, reconstructs and classifies drone formations in images or videos by combining YOLOv5 and Resnet50, so that the optoelectronic system can identify high-value drone targets in a short time. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 This is a formation diagram of the rotor UAV of the present invention. DETAILED DESCRIPTION

[0020] In order to make the purpose, content, and advantages of the present invention more clear, the specific implementation methods of the present invention are further described in detail below in conjunction with the accompanying drawings and examples.

[0021] like Figure 1 As shown, the present invention provides a method for recognizing drone swarm formation based on vision, which comprises the following steps:

[0022] Step 1: Construct a drone image database, including drone target image data and drone formation image data. Data is the core of drone swarm formation recognition and plays a vital role in improving recognition accuracy. The data sources of the present invention are mainly real data obtained through web crawlers or field tests, as well as data obtained through simulation methods.

[0023] (1) Real data collection. Using web crawlers and actual shooting methods of the test site, crawl the videos, images and other data related to drones on the Internet or generated by actual tests, and save them in the local database. The establishment of the dataset includes steps such as video and image acquisition, target marking, and label sorting. The drone swarm target dataset needs to include drone images in different scenes, weather, and distances, and is shot at different angles. When marking, the labeler needs to identify multiple different types of drones and follow the following marking criteria: determine the position of the box from the upper left point to the lower right point, and the highest point, lowest point, leftmost point, and rightmost point of the drone are on the upper boundary, lower boundary, left boundary, and right boundary of the annotation box respectively. After the annotation is completed, a labeling file is generated. The labeling file records the image name, width, height, number of channels, horizontal coordinate of the upper left corner point of the box, vertical coordinate of the upper left corner point of the box, horizontal coordinate of the lower right corner point of the box, vertical coordinate of the lower right corner point of the box, and the annotation category. The annotation information is recorded in a txt format file, which contains the annotation box of the corresponding image. Each annotation box contains the category number, the normalized horizontal and vertical coordinates of the center point, and the width and height of the target box.

[0024] (2) Simulation. The simulation method uses the UAV swarm formation control method based on the Flocking obstacle avoidance algorithm, and uses software simulation to establish a UAV swarm flight model and obtain a formation database. The formation categories are divided into six categories: Figure 1 In the equation, number 1 represents a wedge formation, number 2 represents a horizontal formation, number 3 represents a vertical formation, number 4 represents a circular formation, number 5 represents a serpentine formation, and number 6 represents a trapezoidal formation. The motion state equations (direct assignment) of n drones forming a formation flight are as follows:

[0025]

[0026] Where p i , v i , a i represents the position, velocity and acceleration of the i-th UAV in the formation. During the UAV formation flight, the motion state equation of the formation is as follows:

[0027]

[0028] Where t represents the current time, a=[a 1 , a 2 , ..., a n ] T As you can see, a i Acceleration is the only variable in motion, so acceleration can be designed to control the formation flight state.

[0029] When controlling the drone swarm formation based on the obstacle avoidance algorithm, all drones should meet the safety distance constraints and stable flight constraints:

[0030]

[0031] Where d represents a fixed distance, and Δ is a distance variable that changes with the formation flight time. The distance between drones must be at least greater than the dangerous distance d-Δ, and their maximum distance must be less than d+Δ. The distance between drones tends to stabilize as the formation flies, maintaining a fixed value d.

[0032] Step 2: Build a drone target detection model. Due to the real-time requirements in actual combat, the drone target detection model needs to take into account both lightweight and accuracy. As a single-stage target detection model, Yolov5 has the characteristics of fast detection speed and high detection accuracy. Therefore, it is used as the drone target detection framework and trained and tested on the drone dataset built in step 1.

[0033] Step 3: Construct a UAV formation classification model based on Resnet50. The ResNet series of algorithms introduces a residual structure to eliminate problems such as gradient disappearance and degradation. While continuously increasing the convolution layer, the model accuracy can be improved. Using Resnet50 for formation type classification can enhance the network model's extraction and learning of formation features. When a small amount of internal information about the enemy UAV formation is obtained, the UAV formation formation can be effectively identified. Even if the formation formation is dislocated and disordered, the neural network algorithm can still identify the formation and has good robustness. Train and classify the swarm formation dataset constructed in step 1.

[0034] Step 4: In the task of actually needing to recognize the drone swarm formation, the image is processed as follows.

[0035] (1) Use the Yolov5 model trained in step 2 to detect drone targets and output the detection box position and size matrix of each drone in the image;

[0036] (2) Using the image binarization method, the detection target is set to black, and the image background other than the detection frame is set to white to remove the interference of the background environment;

[0037] (3) Input the image generated by step (2) into the Resnet50 target classification model and output the formation recognition classification result.

Claims

1. A vision-based drone swarm formation recognition method, characterized in that: It includes the following steps: 1) Construct a UAV image dataset, which is a composite dataset of real data and simulated data. The simulated data is obtained by using a behavioral formation control method based on the Flocking obstacle avoidance algorithm, and the real dataset is obtained by web crawlers and field tests. The obstacle avoidance algorithm based on Flocking is as follows: The UAV motion state equation is: Where p i , v i , a i Represents the position, velocity and acceleration of the i-th UAV in the formation. During the UAV formation flight, the motion state equation of the formation is as follows: Where t represents the current time, a=[a1,a2,...,a n ] T When controlling the drone swarm formation based on the obstacle avoidance algorithm, all drones should meet the safety distance constraints and stable flight constraints: Where d represents a fixed distance, Δ is a distance variable, which changes with the formation flight time; the distance between drones is at least greater than the dangerous distance d-Δ, and their maximum distance must be less than d+Δ; the distance between drones tends to be stable as the formation flies, maintaining a fixed value d; 2) Build a UAV target detection model, use the Yolov5-based target detection model to train and verify the UAV image data obtained in the first step, and generate a target detection model file; 3) Construct a UAV formation classification model, use the Resnet50-based classification model to train and verify the UAV formation data obtained in the first step, and generate a target classification model file; 4) In the actual task of UAV swarm formation recognition, the target detection model is used to obtain the specific position of the target in the image, and the image background except the target is set to white and the target is set to black; then it is input into the formation classification model and the formation recognition result is output.

Images