A method for overall target recognition of drone swarms based on deep learning

Abstract

The present invention discloses a method for overall target recognition of drone clusters based on deep learning, which belongs to the field of target detection and includes the following steps: S1, collecting and screening original images of drone clusters; S2, identifying the overall position of the drone cluster; S3, obtaining an overall image of the drone cluster; S4, identifying the individual positions of drones; S5, combining the overall position recognition result of the drone cluster and the individual position recognition result of the drone to obtain the position of the individual drone on the original image of the drone cluster. The present invention adopts the above-mentioned overall target recognition method for drone clusters based on deep learning. When the individual drones in the drone cluster are concentrated, overall target recognition is introduced, and individual drone target recognition is performed based on the overall target recognition. The overall target recognition result and the individual target recognition result are then combined to obtain the final result. While simplifying the steps, the accuracy of the position recognition of individual drones in the cluster is improved.

Description

Technical Field

[0001] The present invention relates to the field of target detection, and in particular to a method for overall target recognition of a drone cluster based on deep learning. Background Art

[0002] With the rapid development of modern technology, drone swarms are increasingly being used in all aspects of daily life. Among them, aerial drone swarm combat is a form of modern warfare. Target identification of aerial drone swarms helps accurately locate and track drones, determine the position changes of individual drones, and analyze the intentions of drone swarms. This helps improve the security of airspace management, enhance military defense capabilities, and safeguard public safety. It plays a vital role in maintaining air order, improving mission execution efficiency, and promoting the advancement of drone technology.

[0003] Currently, target recognition for drone swarms falls under the category of small aerial target recognition. Small targets are defined in an image based on relative scale, defined as the relative area of ​​all target instances within the same category. Specifically, the median ratio of the bounding box area to the image area ranges from 0.08% to 0.58%. Methods for identifying small aerial targets based on visual images face challenges such as weak light signals, complex recognition environments, and high real-time requirements.

[0004] To solve the above problems, the existing technology has conducted research in the following directions:

[0005] For example, in their paper "UAV Optical Small Target Detection Algorithm Based on Dual-Modal Image Fusion," Chang Kaixuan, Huang Jianhua, Sun Xiyan, et al. proposed a dual-channel image fusion detection algorithm based on back-end fusion to enhance the feature information of small target images and improve the accuracy of small target detection. They also designed a back-end fusion detection framework (BFDN) to fuse infrared and visible light images. This method is relatively complex because it requires matching the visible light image and infrared image at corresponding moments to ensure that the corresponding images are as similar as possible.

[0006] In their paper "Real-time Object Detection Based on an Improved YOLOv3 Network," Sun Jia, Guo Dabo, Yang Tiantian, et al. reported on a YOLOv3 network. This approach improves anchor selection by setting a threshold to select appropriate initial cluster centers. For medium and small objects, the method also adds feature maps of varying scales to enhance detection accuracy. However, when the object's area is too small relative to the original image size, adjusting the initial cluster centers of the anchor boxes only slightly improves recognition accuracy for small objects.

[0007] In his paper "Research on Small Target Detection of UAVs in Low-Altitude Airspace at Airports Based on Contrastive Learning," Yuanzihao published a paper using contrastive learning to improve the distinguishability of image background and target features. Specifically, during the process of obtaining positive and negative samples, the team used random image cutouts to capture the target and background images from the sample images. These samples were then enhanced and fed into a contrastive learning network for training. However, this method only works for backgrounds with weak textures, such as clear blue skies. Even after data enhancement, the features of negative samples remain small. Summary of the Invention

[0008] To solve the above problems, the present invention provides a method for overall target recognition of drone clusters based on deep learning. When the individual drones in the drone cluster are concentrated, overall target recognition is introduced, and individual drone targets are recognized based on the overall target recognition. The overall target recognition results and the individual target recognition results are combined to obtain the final result. While simplifying the steps, the accuracy of position recognition of individual drones in the cluster is improved.

[0009] To achieve the above objectives, the present invention provides a method for identifying overall targets of a UAV cluster based on deep learning, comprising the following steps:

[0010] S1. Collecting and screening original images of drone clusters: Collecting original images and screening the original images to determine the overall distribution range of the drone cluster and the size of the original images. The original images that meet the set relative relationship are used as the original images of the drone cluster.

[0011] S2, perform overall target calibration on the drone cluster in the original image of the drone cluster and identify the overall position of the drone cluster;

[0012] S3. Based on the overall position recognition result of the drone cluster obtained in step S2, the original image of the drone cluster is cropped along the overall distribution range of the drone cluster to obtain an overall image of the drone cluster;

[0013] S4, performing individual target calibration on the overall image of the drone cluster cropped in step S3 to identify the individual positions of the drones;

[0014] S5. Combining the overall position recognition result of the drone cluster obtained in step S2 and the individual position recognition result of the drone obtained in step S4, the final target recognition result is obtained according to the image cropping and transformation relationship, and the position of the individual drone on the original image of the drone cluster is obtained.

[0015] Preferably, step S1 specifically includes the following steps:

[0016] S11. Use the visual sensor of the ground base station to capture the drone cluster in the air to obtain the original image;

[0017] S12, randomly select image P from the original image i , i=1,2,···,10%N, N is the total number of original images;

[0018] S13, record the randomly selected i Image P i In the pixel coordinate system, the minimum lateral position u in the UAV cluster is min , maximum horizontal position u max , vertical position minimum value v min , the maximum value of the vertical coordinate v max The drone individual uses the vertex coordinates (u min ,v min )、(u max ,v min )、(u min ,v max )、(u max ,v max ) to determine the distribution range of the drone cluster and calculate the area ratio of the rectangular detection box to the original image. Where S is the area of ​​the original image;

[0019] S14. Calculate the mean of the area ratios of the selected images If β<0.3, it is determined that the set relative relationship is satisfied, and the selected image that satisfies the set relative relationship is regarded as the original image of the drone cluster.

[0020] Preferably, step S2 specifically includes the following steps:

[0021] S21, the original image of the drone cluster is divided into the training set P according to the set ratio train With the test set P test ;

[0022] S22, for the training set P train For each original image of the drone cluster in the image coordinate system, search for the drone cluster in the original image with the minimum lateral position u min , maximum horizontal position u max , vertical position minimum value v min , the maximum value of the vertical coordinate v max The drone individual uses the vertex coordinates (u min ,v min )、(u max ,v min )、(u min ,v max )、(u max ,v max)’s rectangular detection frame frames the drone cluster, and the location of the rectangular detection frame is regarded as the overall location of the drone cluster;

[0023] S23. Use the obtained overall position of the drone cluster to train the YOLOv5 deep learning algorithm model;

[0024] S24, the test set P test Input the YOLOv5 deep learning algorithm model trained in step S23, and output the overall image result of the drone cluster represented by a rectangular detection box.

[0025] Preferably, in step S24, the output result (u min ,u max ,v min ,v max ), representing the four vertices of the rectangular detection box (u min ,v min )、(u max ,v min )、(u min ,v max )、(u max ,v max ).

[0026] Preferably, in step S3, according to (u min ,u max ,v min ,v max ) to crop the image, retain the rectangular detection frame part, and obtain the image P where the drone cluster accounts for a large proportion in the entire image after cropping crop , and the image P crop As an overall image of the drone cluster.

[0027] Preferably, step S4 specifically includes the following steps:

[0028] S41, according to the set ratio, the image P crop Divide into training set With the test set

[0029] S42, training set Use the images in the image to calibrate the individual target of the UAV;

[0030] S43, using the image calibrated in step S42 to train the YOLOv5 deep learning algorithm model;

[0031] S44, the test set Input the YOLOv5 deep learning algorithm model to identify the individual positions of drones in the image.

[0032] Preferably, in step S42, all rectangular detection frames that frame the outer contour of the individual drone are searched, the smallest rectangular detection frame among all the rectangular detection frames is found, and the individual drone is calibrated using the smallest rectangular detection frame.

[0033] Preferably, the YOLOv5 deep learning algorithm models described in step S23 and step S43 include an input structure, a backbone network, a neck structure and a head structure arranged in sequence, wherein the input structure is used for image preprocessing, the backbone network is used for feature extraction, the neck structure is used for feature fusion, and the head structure is used to output detection results.

[0034] Preferably, in step S5, assuming that the original image P of the drone cluster is i After the overall calibration of the drone cluster, the output rectangular detection box is After cropping, we get image P i ', P i 'After the drone individual calibration, the image P i 'The recognition result corresponding to the drone individual j is a rectangular detection box The image P is obtained by combining the results of the overall target recognition of the UAV cluster and the individual target recognition of the UAV. i The position of UAV individual j is

[0035] The present invention has the following beneficial effects:

[0036] When the individual drones in a cluster are concentrated, overall target recognition is introduced, and individual drone target recognition is performed based on the overall target recognition. The overall target recognition results and the individual target recognition results are combined to obtain the final result. While simplifying the steps, the accuracy of individual drone position recognition in the cluster is improved.

[0037] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 This is a flowchart of a method for identifying overall targets of a UAV cluster based on deep learning according to the present invention;

[0039] Figure 2 This is an example original image of an aerial drone cluster in the simulation experiment of the present invention;

[0040] Figure 3 This is a diagram showing the overall target recognition results of the drone cluster in the simulation experiment of the present invention;

[0041] Figure 4 This is a diagram showing the results of identifying individual drone targets in a drone cluster in a simulation experiment of the present invention;

[0042] Figure 5 This is the final target recognition result diagram of the drone cluster in the simulation experiment of the present invention. DETAILED DESCRIPTION

[0043] In order to make the purpose, technical solutions and advantages disclosed in the embodiments of the present invention clearer, the embodiments of the present invention are further described in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the embodiments of the present invention and are not intended to limit the embodiments of the present invention. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application. Examples of the embodiments are shown in the accompanying drawings, where the same or similar numbers throughout represent the same or similar elements or elements with the same or similar functions.

[0044] It should be noted that the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or server that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or are inherent to these processes, methods, products or devices.

[0045] Like reference numerals and letters denote like items in the following drawings, and thus, once an item is defined in one drawing, it does not require further definition or explanation in subsequent drawings.

[0046] Taking into account the accuracy and convenience requirements of aerial drone cluster target recognition, the present invention is designed as follows: Figure 1 As shown in FIG, a method for overall target recognition of a UAV cluster based on deep learning includes the following steps:

[0047] S1. Collecting and screening original images of drone clusters: Collecting original images and screening the original images to determine the overall distribution range of the drone cluster and the size of the original images. The original images that meet the set relative relationship are used as the original images of the drone cluster.

[0048] Step S1 specifically includes the following steps:

[0049] S11. Use the visual sensor of the ground base station to capture the aerial drone cluster to obtain the original image. The image of the aerial drone cluster at a distance generally has the characteristics of the drone cluster occupying a small area of ​​the entire image and the individual drones are densely distributed.

[0050] S12, randomly select image P from the original image i , i=1,2,···,10%N, N is the total number of original images;

[0051] S13, record the randomly selected i-th image P i In the pixel coordinate system, the minimum lateral position u in the UAV cluster is min , maximum horizontal position u max , vertical position minimum value v min , the maximum value of the vertical coordinate v max The drone individual uses the vertex coordinates (u min ,v min )、(u max ,v min )、(u min ,v max )、(u max ,v max ) to determine the distribution range of the drone cluster and calculate the area ratio of the rectangular detection box to the original image. Where S is the area of ​​the original image;

[0052] S14. Calculate the mean of the area ratios of the selected images If β<0.3, it is determined that the set relative relationship is satisfied, and the selected image that satisfies the set relative relationship is regarded as the original image of the drone cluster.

[0053] S2, perform overall target calibration on the drone cluster in the original image of the drone cluster and identify the overall position of the drone cluster;

[0054] Step S2 specifically includes the following steps:

[0055] S21, the original image of the drone cluster is divided into the training set P according to the set ratio train With the test set P test ;

[0056] S22, for the training set P train For each original image of the drone cluster in the image coordinate system, search for the drone cluster in the original image with the minimum lateral position u min , maximum horizontal position u max , vertical position minimum value v min , the maximum value of the vertical coordinate v max The drone individual uses the vertex coordinates (u min ,v min )、(u max ,v min )、(u min ,v max )、(u max ,v max )’s rectangular detection frame frames the drone cluster, and the location of the rectangular detection frame is regarded as the overall location of the drone cluster;

[0057] S23. Use the obtained overall position of the drone cluster to train the YOLOv5 deep learning algorithm model;

[0058] S24, the test set P test Input the YOLOv5 deep learning algorithm model trained in step S23, and output the overall image result of the drone cluster represented by a rectangular detection box.

[0059] In step S24, the result (u min ,u max ,v min ,v max ), representing the four vertices of the rectangular detection box (u min ,v min )、(u max ,v min )、(u min ,v max )、(u max ,v max ).

[0060] S3. Based on the overall position recognition result of the drone cluster obtained in step S2, the original image of the drone cluster is cropped along the overall distribution range of the drone cluster to obtain an overall image of the drone cluster;

[0061] In step S3, according to (u min ,u max ,v min ,v max ) to crop the image, retain the rectangular detection frame part, and obtain the image P where the drone cluster accounts for a large proportion in the entire image after cropping crop , and the image P crop As an overall image of the drone cluster.

[0062] S4, performing individual target calibration on the overall image of the drone cluster cropped in step S3 to identify the individual positions of the drones;

[0063] Step S4 specifically includes the following steps:

[0064] S41, according to the set ratio, the image P crop Divide into training set With the test set

[0065] S42, training set Use the images in the image to calibrate the individual target of the UAV;

[0066] In step S42, all rectangular detection frames that frame the outer contour of the drone individual are searched, the smallest rectangular detection frame among all the rectangular detection frames is found, and the drone individual is calibrated using the smallest rectangular detection frame.

[0067] S43, using the image calibrated in step S42 to train the YOLOv5 deep learning algorithm model;

[0068] S44, the test set Input the YOLOv5 deep learning algorithm model to identify the individual positions of drones in the image.

[0069] The YOLOv5 deep learning algorithm models described in step S23 and step S43 both include an input structure, a backbone network, a neck structure, and a head structure arranged in sequence, wherein the input structure is used for image preprocessing, the backbone network is used for feature extraction, the neck structure is used for feature fusion, and the head structure is used to output detection results. It has the advantages of fast detection speed, lightweight model, and simple training.

[0070] S5. Combining the overall position recognition result of the drone cluster obtained in step S2 and the individual position recognition result of the drone obtained in step S4, the final target recognition result is obtained according to the image cropping and transformation relationship, and the position of the individual drone on the original image of the drone cluster is obtained.

[0071] The relative positions of objects in the image are expressed in pixel coordinates. In the pixel coordinate system, the upper left corner of the image is the origin (0, 0), the horizontal axis (u axis) increases from left to right, and the vertical axis (v axis) increases from top to bottom. Based on this, in step S5, it is assumed that the original image P of the drone cluster is i After the overall calibration of the drone cluster, the output rectangular detection box is After cropping, we get image P i ', P i 'After the drone individual calibration, the image P i 'The recognition result corresponding to the drone individual j is a rectangular detection box The image P is obtained by combining the results of the overall target recognition of the UAV cluster and the individual target recognition of the UAV. i The position of UAV individual j is

[0072] Simulation experiment

[0073] In this simulation experiment, the Figure 2 As shown, a drone cluster consisting of 56 drones is in a triangular formation with a relatively concentrated distribution. There are 99 images in total, and the resolution of each image is 1920*1080. The average area ratio of 10 randomly selected images is calculated. The result is 0.0094, which meets the requirements of overall target recognition, so overall target recognition can be performed on the 10 selected images.

[0074] Table 1 Area ratio of selected images

[0075]

[0076]

[0077] The selected images are processed using the deep learning-based UAV cluster overall target recognition method of the present invention, and the results are as follows: Figure 3-Figure 5 shown.

[0078] Table 2: Target recognition results of example images using the algorithm of the present invention

[0079]

[0080]

[0081]

[0082] Combine Figure 3-Figure 5 As can be seen from Table 2, due to the fact that some drones were cropped at the boundary of the rectangular detection frame and some drones were close to each other in the image, 54 drones were finally identified in the image, which is within a reasonable range.

[0083] However, if we directly perform drone individual calibration on the same drone cluster image and use the YOLOv5 model training to perform drone individual recognition, the model will not be able to recognize the individual drones because the pixels occupied by the drone targets are too small.

[0084] Thus verifying the effectiveness of the design of the present invention.

[0085] Therefore, the present invention adopts the above-mentioned deep learning-based drone cluster overall target recognition method. When the individual distribution of the drone cluster is concentrated, overall target recognition is introduced, and individual drone target recognition is performed based on the overall target recognition. The overall target recognition result and the individual target recognition result are combined to obtain the final result. While simplifying the steps, the accuracy of position recognition of individual drones in the cluster is improved, and it has a broad application prospect.

[0086] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit the same. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that they can still modify or replace the technical solutions of the present invention with equivalents, and these modifications or equivalent replacements cannot cause the modified technical solutions to deviate from the spirit and scope of the technical solutions of the present invention.

Claims

1. A method for identifying overall targets of drone swarms based on deep learning, characterized by: The following steps are involved: S1. Collecting and screening original images of drone clusters: Collecting original images and screening the original images to determine the overall distribution range of the drone cluster and the size of the original images. The original images that meet the set relative relationship are used as the original images of the drone cluster. S2, perform overall target calibration on the drone cluster in the original image of the drone cluster and identify the overall position of the drone cluster; Step S2 specifically includes the following steps: S21. Divide the acquired drone cluster original images into training sets according to the set ratio With the test set ; S22, training set For each original image of the drone cluster in the image coordinate system, search for the drone cluster in the original image with the minimum horizontal position , maximum horizontal position , vertical position minimum , maximum value of vertical coordinate The drone individual uses the vertex coordinates 、 、 、 The rectangular detection frame frames the drone cluster, and the location of the rectangular detection frame is regarded as the overall location of the drone cluster; S23. Use the obtained overall position of the drone cluster to train the YOLOv5 deep learning algorithm model; S24, the test set Input the YOLOv5 deep learning algorithm model trained in step S23, and output the overall image result of the drone cluster represented by a rectangular detection box; S3. Based on the overall position recognition result of the drone cluster obtained in step S2, the original image of the drone cluster is cropped along the overall distribution range of the drone cluster to obtain an overall image of the drone cluster; S4, performing individual target calibration on the overall image of the drone cluster cropped in step S3 to identify the individual positions of the drones; Step S4 specifically includes the following steps: S41, according to the set ratio, the whole image of the drone cluster is Divide into training set With the test set ; S42, training set Use the images in the image to calibrate the individual target of the UAV; S43, using the image calibrated in step S42 to train the YOLOv5 deep learning algorithm model; S44, the test set Input the YOLOv5 deep learning algorithm model to identify the individual positions of drones in the image; S5. Combining the overall position recognition result of the drone cluster obtained in step S2 and the individual position recognition result of the drone obtained in step S4, the final target recognition result is obtained according to the image cropping and transformation relationship, and the position of the individual drone on the original image of the drone cluster is obtained.

2. The method for identifying an entire drone cluster based on deep learning according to claim 1, characterized in that: Step S1 specifically includes the following steps: S11. Use the visual sensor of the ground base station to capture the drone cluster in the air to obtain the original image; S12. Randomly select an image from the original image , , is the total number of original images; S13, record the randomly selected images In the pixel coordinate system, the minimum lateral position of the drone cluster , maximum horizontal position , vertical position minimum , maximum value of vertical coordinate The drone individual uses the vertex coordinates as 、 、 、 The rectangular detection frame is used to determine the distribution range of the drone cluster and the area ratio of the rectangular detection frame to the original image is calculated. ,in is the area of ​​the original image; S14. Calculate the mean of the area ratio of the selected image ,like , it is determined that the set relative relationship is satisfied, and the selected image that satisfies the set relative relationship is regarded as the original image of the drone cluster.

3. The method for identifying an entire drone cluster based on deep learning according to claim 1, characterized in that: In step S24, the result is output , representing the four vertices of the rectangular detection box 、 、 、 .

4. The method for identifying an entire drone cluster based on deep learning according to claim 3, characterized in that: In step S3, according to Crop the image, retain the rectangular detection frame part, and obtain the image in which the drone cluster accounts for a larger proportion of the entire image after cropping , and the image As an overall image of the drone cluster.

5. The method for identifying an entire drone cluster based on deep learning according to claim 4, characterized in that: In step S42, all rectangular detection frames that frame the outer contour of the drone individual are searched, the smallest rectangular detection frame among all the rectangular detection frames is found, and the drone individual is calibrated using the smallest rectangular detection frame.

6. The method for identifying an entire drone cluster based on deep learning according to claim 5, characterized in that: The YOLOv5 deep learning algorithm models described in step S23 and step S43 both include an input structure, a backbone network, a neck structure, and a head structure arranged in sequence, wherein the input structure is used for image preprocessing, the backbone network is used for feature extraction, the neck structure is used for feature fusion, and the head structure is used to output detection results.

7. The method for identifying an entire drone cluster based on deep learning according to claim 6, characterized in that: In step S5, assuming that the original image of the drone cluster After the overall calibration of the drone cluster, the output rectangular detection box is , after cropping, we get the image , After the drone individual calibration, the image Drone individuals The corresponding recognition result is a rectangular detection box , then the results of the overall target recognition of the UAV cluster and the individual target recognition of the UAV are combined to obtain the image Drone individuals The location is .

Images