A method for aerial target detection based on deep learning

Abstract

The present invention provides an aerial target detection method based on deep learning, which relates to the field of machine vision detection technology. The method uses the SSD algorithm as the basic detection algorithm to realize the detection function of aerial targets. A method for collecting aerial target data sets is given, and then the SSD backbone network is improved. By introducing the attention mechanism, FPN feature fusion, K-means clustering algorithm, and adopting the EIOU bounding box regression loss function, the ATD-SSD algorithm is designed and implemented. Then the aerial target data set is trained and the model is saved. Finally, the image is tested. After obtaining the position data and category data related to the prediction, the prediction results are screened, and finally the prediction box is drawn. The present invention can improve the accuracy and real-time performance of aerial targets, and can complete the real-time detection of aerial targets while maintaining high-precision detection effects.

Description

Technical Field

[0001] The present invention relates to the field of machine vision detection technology, and in particular to an aerial target detection method based on deep learning. Background Art

[0002] Deep learning is currently being widely applied across various fields. For example, in natural language processing, deep learning can be used for automated translation, automatically converting between different languages; in text classification, it can be used to categorize and classify documents by type; in email, it can filter out spam; and in speech recognition, it can automatically convert human speech into text. Deep learning is also having a profound impact in fields such as data mining, object detection, and drones. Furthermore, with the rapid development of drones, their use is becoming increasingly common, for example, in disaster-stricken areas, surveying farmland and forests, and in the military, using drones to assist combatants.

[0003] As deep learning-based object detection algorithms continue to improve and drones rapidly develop, the detection of aerial targets has become a hot topic. Aerial targets can be categorized as large and small. Large targets include mountains and houses, while small targets include drones and birds. Large targets are typically more visible and distinct, making them easier to detect. However, small targets have lower visibility and erratic motion, making them difficult to detect in the air. These small aerial targets pose a significant threat to drone flight safety, aerial environmental monitoring, and drone coordinated operations. Therefore, while ensuring accurate detection of large aerial targets, accurately detecting small aerial targets is crucial. Due to their small size and erratic motion, small aerial targets are easily affected by the external environment, resulting in low detection accuracy and even missed or false detections. Furthermore, clusters of small targets can lead to many being misclassified as background, compromising detection effectiveness. Summary of the Invention

[0004] The technical problem to be solved by the present invention is to address the shortcomings of the above-mentioned existing technologies and provide a deep learning-based aerial target detection method. This method, taking aerial targets encountered during drone flight as the research object, studies the problem of deep learning-based aerial target detection. The SSD algorithm is improved to address the problems of low aerial target detection accuracy, missed detections, and incorrect classification of positive and negative samples. While ensuring the algorithm's detection speed, the detection accuracy of aerial targets is improved. Ultimately, an SSD-based aerial target detection algorithm, referred to as the ATD-SSD algorithm, is proposed. The ATD-SSD algorithm improves aerial target detection accuracy while ensuring detection speed.

[0005] In order to solve the above technical problems, the technical solution adopted by the present invention is:

[0006] A method for detecting aerial targets based on deep learning, comprising the following steps:

[0007] Step 1: Collect the aerial target data set according to the characteristics of the aerial target graphics, and expand and process the graphics image;

[0008] Step 2: Prepare the aerial target dataset according to the specifications of the VOC aerial target dataset, and strictly follow the corresponding format for labeling and naming;

[0009] Step 3: Improve the target box design, improve the backbone network, and improve the loss function; read the aerial target dataset, and then use the improved backbone network to obtain the improved target box size. Then process the aerial target dataset, extract and save the information of the images in the aerial target dataset, and use the improved loss function to calculate the loss. Then, classify the positive and negative samples. Finally, obtain the final model through continuous calculation and adjustment of weight parameters and save it.

[0010] Step 4: First process the image to be tested, unify the image size to 300×300, then load the backbone network and the trained model, then load the image into the model, obtain the location data and category data related to the prediction, filter the prediction results, and finally draw the prediction box.

[0011] Furthermore, in step 1, the aerial target dataset is collected using Python-based web crawler technology and autonomous drone photography.

[0012] Furthermore, in step 2, LabelImg labeling software is used for labeling.

[0013] Furthermore, the target frame design in step 3 is improved by using the K-means clustering algorithm to perform cluster analysis on the aerial target dataset used, improve the initial anchor frame, and obtain the optimal length-to-width ratio of the target frame; the K value in the K-means clustering algorithm is 6;

[0014] After clustering, the target frame size of each feature map input to the detection layer is calculated. First, the length and width values ​​of each target frame in the aerial small target dataset are collected. The product of the length and width is the area of ​​the target frame. Then, the ratio of the area of ​​the target frame to the area of ​​the original image is recorded as s. A new array S is formed by the ratios s corresponding to all feature maps. Finally, S is sorted to obtain the first data s of the array S. max and the last data s min The value of the target box size B k Calculate as shown in formula (1);

[0015]

[0016] Where m represents the number of feature maps, k∈[1,m];

[0017] After calculation, the most suitable target box size for the output feature map can be obtained.

[0018] Furthermore, the improvements to the backbone network in step 3 include:

[0019] Add attention mechanism based on the original network SSD;

[0020] When the image enters the SSD algorithm, it is processed by the network to obtain a low-level feature map, but it is not propagated forward immediately. Instead, the low-level feature map is processed using the CBAM attention mechanism before being propagated forward.

[0021] Based on the original network SSD, feature fusion is performed through the FPN network structure; the bilinear interpolation algorithm is used to improve the FPN upsampling method; based on the characteristics of the feature map, two methods, low-level fusion module and high-level fusion module, are used to achieve fusion between feature maps; in the low-level fusion module, the input high-level feature map is first upsampled and then summed with the low-level feature map; the high-level feature fusion module first upsamples the input high-level feature information and then splices it with the low-level feature information. After splicing, the feature map is reduced in dimension using 1×1 convolution;

[0022] When the image enters the network, the size of the input image is first uniformly adjusted to 300×300, and then the input information is processed in a bottom-up path, passing through an attention module and then through other feature layers, and finally outputting eight feature maps of different scales, with scales of 38×38×256, 38×38×512, 19×19×1024, 19×19×1024, 10×10×512, 5×5×256, 3×3×256, and 1×1×256; then they are scaled uniformly through a horizontal channel convolution, and then feature fusion is performed. Finally, a 3×3 convolution is used to reduce the aliasing effect, and finally sent to the detection layer for classification and detection of aerial targets.

[0023] Furthermore, the loss function in step 3 is improved by using the EIOU bounding box regression loss function, as shown in formula (6);

[0024]

[0025] Among them, IOU represents the intersection-union ratio between the real box and the predicted box, b gt, b represent the center points of the real box and the predicted box respectively, ρ represents the Euclidean distance between the two center points of the real box and the predicted box, c represents the diagonal length of the minimum circumscribed rectangle of the real box and the predicted box, w gt , w represent the width of the real box and the predicted box respectively, h gt , h represent the height of the real box and the predicted box respectively, They represent the width and height of the minimum bounding rectangle that can contain both the true box and the predicted box.

[0026] Furthermore, in step 3, the ATD-SSD target detection algorithm is used, and the process is as follows:

[0027] The ATD-SSD network first inputs an image through the input end, and the image size is selected as 300×300 according to actual needs; after applying the K-means clustering algorithm on the aerial target dataset to re-cluster the generated anchor point box, the most appropriate target box size is obtained, and the parameters are modified in the algorithm. Then, preliminary feature extraction is performed on the optimized backbone feature extraction network, and the extracted image features are passed to the improved multi-scale FPN network for feature fusion and enhancement of image features. Finally, predictions of different scales are performed in the prediction layer, and the boundary regression loss function EIOU is added.

[0028] The beneficial effects of adopting the above technical solution are as follows: the aerial target detection method based on deep learning provided by the present invention, first, proposes a target frame clustering algorithm to cluster the aerial target data set and calculates the most appropriate target frame size. In addition, in order to address the problems that small aerial targets are small in size, easily overlooked, and have low attention, a hybrid attention mechanism is introduced into the backbone network of the algorithm to improve the algorithm's attention to small aerial targets. In order to address the problems of insufficient feature information extraction and feature loss, the feature extraction method is improved, and bilinear interpolation is used for upsampling. Secondly, different fusion methods are used for feature fusion to increase the amount of small target feature information and improve detection accuracy. Finally, in order to address the problem of incorrect classification of overlapping small aerial targets, backgrounds, and targets, the selected algorithm is optimized from the two perspectives of target frame size and loss function. The EIOU loss function is used to replace the traditional IOU loss function, improve the algorithm's back propagation problem, and optimize the algorithm's shortcomings of incorrect background and target classification. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 A functional division diagram based on the entire model provided for a specific embodiment of the present invention;

[0030] Figure 2 A diagram of data collected using crawler technology provided in a specific embodiment of the present invention;

[0031] Figure 3A diagram of drone image acquisition data provided by a specific embodiment of the present invention;

[0032] Figure 4 This is a diagram of the LabelImg annotation interface provided for a specific embodiment of the present invention;

[0033] Figure 5 The network model of the ATD-SSD algorithm provided in the specific embodiment of the present invention;

[0034] Figure 6 Detection result diagrams provided for a specific embodiment of the present invention; wherein, Figure (6a) shows the detection result for buildings and trees, Figure (6b) shows the detection result for mountains, and Figure (6c) shows the detection result for drones;

[0035] Figure 7 Flowchart of shallow feature graph of SSD algorithm after adding attention mechanism provided by specific embodiment of the present invention;

[0036] Figure 8 A network structure diagram of the CBAM provided for a specific embodiment of the present invention;

[0037] Figure 9 The FPN structure diagram provided for the specific embodiment of the present invention;

[0038] Figure 10 A schematic diagram of a bilinear interpolation algorithm provided for a specific embodiment of the present invention;

[0039] Figure 11 The processing result diagram of the same image processed by the interpolation algorithm provided in the specific embodiment of the present invention; wherein, Figure (11a) is the original image, Figure (11b) is the nearest neighbor interpolation processing diagram, and Figure (11c) is the bilinear interpolation processing diagram;

[0040] Figure 12 A structural diagram of a fusion module provided for a specific embodiment of the present invention; wherein, FIG (12a) is a structural diagram of a low-level fusion module, and FIG (12b) is a structural diagram of a high-level fusion module;

[0041] Figure 13 A detection accuracy diagram provided for a specific embodiment of the present invention;

[0042] Figure 14 The prediction result diagram provided for the specific embodiment of the present invention; wherein, Figure (14a) is the prediction result diagram of buildings and trees, Figure (14b) is the prediction result diagram of mountains, Figure (14c) is the prediction result diagram of flying birds, and Figure (14d) is the prediction result diagram of drones. DETAILED DESCRIPTION

[0043] The following embodiments of the present invention are described in further detail with reference to the accompanying drawings and examples. The following examples are used to illustrate the present invention but are not intended to limit the scope of the present invention.

[0044] like Figure 1 As shown, the target detection method based on deep learning in this embodiment is mainly divided into a training part and a prediction part.

[0045] The specific implementation of the entire model includes the following steps:

[0046] Step 1: Collect the aerial target data set according to the characteristics of the aerial target graphics, and expand and process the graphics images.

[0047] There are two main sources for obtaining aerial target datasets: collecting them online and capturing them yourself. To ensure the complexity and diversity of the aerial target dataset, we used Python-based web crawler technology and autonomous drone photography to collect the dataset.

[0048] Information crawling mainly refers to using relevant algorithms to search for data or web pages that meet the corresponding conditions on the Internet according to certain rules, and then saving the required data or results. There are many screening strategies for web crawlers, such as depth-first search traversal, breadth-first search traversal, etc. At the same time, Python's request library can directly connect to the web page URL and obtain the response of the web page. In addition, Python's scrapy library allows users to easily monitor logs and crawl and store data. Therefore, this embodiment uses the Python language to realize the crawling and collection of aerial target data sets. Crawler technology crawls some pictures of drones, such as Figure 2 shown.

[0049] Using DJI M100 to maintain a speed of 18m / min, a total of 200G of video materials were shot at different heights and angles. The resolution of each material is 1980×1680. Finally, the relevant algorithms were written in Python to process the video. By cutting the video frames, pictures of different scenes were obtained, and the cut pictures were filtered to remove some irrelevant and unclear pictures, and a total of 6,000 high-definition, usable pictures were obtained. Among them, the pictures finally obtained by drone are as follows: Figure 3 shown.

[0050] The DJI M100 is a professional-grade drone equipped with a DJI 3510 motor, a 650mm wheelbase, and a horizontal flight speed of 22m / s. It also features a high-definition gimbal camera for video capture. Other specific specifications for the DJI M100 are shown in Table 1.

[0051] Table 1 Parameters of M100

[0052]

[0053] Step 2: Prepare the aerial target dataset according to the specifications of the VOC aerial target dataset, strictly label and name it according to the corresponding format, and obtain the final usable aerial target dataset.

[0054] LabelImg is an open source software for image annotation. It uses Python and Qt to design the program logic and interface. It can annotate aerial target datasets in VOC or COCO formats. Figure 4 As shown, the left side of the software is a row of function navigation buttons. For example, Open is to open a picture, Open Dir is to open the folder where the picture is stored, Change Save Dir is the path to store the annotation file, and Next / Pre Image can switch pictures. The pictures annotated by LabelImg will eventually retain three folders, namely Annexes, JPEGImages and ImageSets. Annotations is used to store the annotated XML files; ImageSets stores the divided image path names, and the pictographic images can be found according to the stored information; JPEGImages stores JPG format pictures. The final XML file stores the name, location information, width, height and other information of the objects contained in each picture.

[0055] Step 3: Improve the target box design, backbone network, and loss function in the ATD-SSD algorithm; first, read the aerial target dataset, then use the improved backbone network to obtain the improved target box size, then process the aerial target dataset, extract and save the information of the images in the aerial target dataset, and then use the EIOU loss function to calculate the loss, and classify the positive and negative samples. Finally, obtain the final model through continuous calculation and adjustment of weight parameters and save it.

[0056] The ATD-SSD algorithm is proposed, which can improve the detection accuracy while ensuring the real-time detection. The ATD-SSD algorithm is based on the improved SSD model, which can detect specific targets. The improvements made are: (1) The K-means clustering algorithm is used to design the target frame. (2) The backbone network of the algorithm is improved by adding an attention mechanism, performing multi-scale fusion, enriching feature information, and adding an FPN network structure to enhance the learning ability of the convolutional neural network and improve detection accuracy; (3) The EIOU loss method is introduced to construct a regression loss function to improve the loss function. The network model of the ATD-SSD algorithm is as follows: Figure 5 As shown, the test results are Figure 6 shown.

[0057] Step 3-1: Design of target box.

[0058] Use the K-means clustering algorithm to perform cluster analysis on the aerial target dataset and improve the initial anchor boxes. Set the K value in the K-means clustering algorithm to 6.

[0059] After obtaining the optimal ratio of the length and width of the target frame through the clustering algorithm, the size of the target frame needs to be recalculated. The size of the target frame will not only affect the final detection effect, but also the detection accuracy. If the size of the target frame is significantly different from the target frame size of the aerial target dataset during the design process, the positive and negative ratios of the entire sample will be unbalanced, resulting in low detection accuracy, or even missed detection or wrong detection. In order to solve these problems, the target frame size of each feature map input to the detection layer needs to be calculated. First, collect the length and width values ​​of each target frame in the aerial small target dataset. The product of the length and width is the area of ​​the target frame. Then, the ratio of the area of ​​the target frame to the area of ​​the original image is recorded as s. A new array S is formed by all s. Finally, S is sorted. The sorting algorithm steps are as follows:

[0060] (1) Randomly select a number in the array as an initial number;

[0061] (2) Put the data in the array that is smaller than the initial number to the right of the initial number, and put the data that is larger than the initial number to the left of the initial number, forming two sub-arrays;

[0062] (3) Repeat steps (1) and (2) for the subarrays on the left and right sides of the initial number until all S are ordered or each subarray has only one data;

[0063] (4) The first and last data of array S are s max and s min value.

[0064] After calculating s, we get s max and s min The values ​​are 0.15 and 0.9 respectively, and then the size B of the target box is k Calculate as shown in formula (1).

[0065]

[0066] Where m represents the number of feature maps, k∈[1,m].

[0067] After calculation, the most suitable target box size for the output feature map can be obtained.

[0068] Step 3-2: Improve the backbone network of the algorithm.

[0069] In order to solve the problem of low detection accuracy and poor effect of small targets in aerial target datasets, this embodiment is designed to combine the hybrid domain attention mechanism CBAM with the SSD algorithm. The low-level feature map of the SSD algorithm has a small receptive field, and most of the information extracted is related to some details of the image such as color, edges and corners, which contains more detailed information. Therefore, the low-level feature map is responsible for the detection of small targets. In addition, since large targets in the aerial target dataset are more obvious and easier to detect, and have a higher degree of recognition, considering the parameter amount of the entire algorithm, the attention mechanism is not added after the feature maps of all scales. The attention mechanism is only added to the low layer. This not only improves the detection accuracy of small targets, but also reduces the computational complexity of the algorithm as much as possible. The flowchart of the shallow feature map of the SSD algorithm after adding the attention mechanism is as follows. Figure 7 As shown in Figure 2. When the image enters the SSD algorithm, it is processed by the network to obtain a low-level feature map. However, it is not propagated forward immediately. Instead, the CBAM attention mechanism is used to process the low-level feature map first, and then propagated forward after processing. The network structure of CBAM is as follows: Figure 8 shown.

[0070] In the first step, the input feature map passes through two pooling layers and then through a neural network to obtain two output feature maps. These two feature maps are then fused and activated to generate a channel attention feature map. In the second step, the channel attention feature map is multiplied by the original feature map. The map is then processed through a pooling layer and concatenated to reduce the number of channels to 1. An activation function is then used to generate a spatial attention feature map. Finally, this feature map is multiplied with the channel attention feature map obtained in the first step to obtain the final CBAM output feature map. The most valuable information in the CBAM output feature map receives higher attention. The processed feature map is then propagated upward and processed in the next step of the algorithm to obtain feature maps at multiple scales.

[0071] Since the basic backbone network of the SSD algorithm is not friendly to the detection of small targets in the air, it is necessary to optimize the backbone network. By drawing on the advantages of the Feature Pyramid Network (FPN), the feature extraction network of the algorithm is improved. FPN is mainly divided into three parts: bottom-up processing, horizontal connection processing, and top-down processing. The process of combining the three parts is as follows: Figure 9 As shown in the figure, when an image enters the FPN, it first undergoes bottom-up processing, extracting feature information from the image and propagating it upward layer by layer. Horizontal connection processing modifies the scale of the feature map to a uniform, usable size to facilitate further processing. Top-down processing fuses the feature information. Using FPN to improve the backbone network of the SSD algorithm not only improves the algorithm's efficiency in extracting feature information from aerial target datasets, but also increases the amount of aerial target feature information captured.

[0072] Among them, in the process of FPN fusion, the nearest neighbor interpolation method (Nearest Neighbour Interpola) is used for upsampling. The nearest neighbor interpolation method does not need to recalculate the information of the feature map, and directly uses the original information as the output, and the calculation is relatively simple. Therefore, the nearest neighbor interpolation algorithm is relatively simple to implement. However, because the nearest neighbor interpolation algorithm does not go through calculations but directly uses the information of the original feature map, it is easy to cause the accumulation of color blocks in the feature map after upsampling, which causes great interference to the detection and classification of the target. Therefore, this embodiment proposes to use a bilinear interpolation algorithm to improve the upsampling method of FPN. The principle diagram of the bilinear interpolation algorithm is shown in the figure. Figure 10 shown.

[0073] Assume that we already know the function f in Q 11 (x1,y1),Q 12 (x1,y2),Q 21 (x2,y1),Q 22 If we want to know the value of P(x,y) on the function f, we first perform linear interpolation in the x direction to obtain f(R1) and f(R2) as shown in Equation (2) and Equation (3), respectively.

[0074]

[0075]

[0076] Then perform linear interpolation in the y direction to obtain f(P) as shown in formula (4).

[0077]

[0078] Substituting the results of equations (2) and (3) into equation (4) yields the final result of bilinear interpolation f(x, y), which is P(x, y), as shown in equation (5).

[0079]

[0080] Through the analysis of the two algorithms, the processing results of the same image using the two algorithms are as follows: Figure 11 As shown. Figure (11a), Figure (11b) and Figure (11c) are the original image, the nearest neighbor interpolation method and the bilinear interpolation method processing diagrams respectively. As can be seen from the figure, under the premise of using the same drone picture, Figure (11b) processed by the nearest neighbor interpolation algorithm has many jagged edges and blurred points on the edge of the target, and Figure (11c) processed by the bilinear interpolation algorithm has a clearer edge of the target and a better effect. Although the use of the bilinear interpolation method for upsampling increases the amount of calculation to a certain extent, it reduces the loss of feature information, ensures the grayscale continuity of the feature map, and is beneficial to the detection of aerial targets, so the speed reduction caused by the increase in calculation amount can be ignored. Therefore, this embodiment uses the bilinear interpolation algorithm for upsampling.

[0081] FPN needs to fuse the high-level feature map with the low-level feature map. FPN only uses one fusion method in the process of feature fusion. However, due to the different scales and structures of feature maps at different depths, the characteristics of feature maps at different depths are also different. Some feature maps have more channels, while some feature maps have fewer channels. If only one feature fusion method is used, a large amount of feature information will be lost during the fusion process, which will lead to unsatisfactory final detection and classification effects. Therefore, this embodiment uses two methods, low-level fusion module and high-level fusion module, to achieve fusion between feature maps according to the characteristics of feature maps, making full use of the characteristics of feature maps at different depths. The structural diagrams are shown as follows: Figure 12 As shown in Figures (12a) and (12b) in .

[0082] In the low-level fusion module, the incoming high-level feature maps are first upsampled and then summed with the low-level feature maps. The low-level fusion module is mainly used for feature maps with a large number of channels. The high-level feature fusion module first upsamples the incoming high-level feature information and then concatenates it with the low-level feature information. After concatenation, 1×1 convolution is used to reduce the dimension of the feature map, allowing the concatenated features to be well fused. The high-level fusion module is mainly used for feature maps with a small number of channels.

[0083] When an image enters the network, it is first resized to 300×300. Then, the input information is processed in a bottom-up manner, passing through an attention module and other feature layers. The final output is eight feature maps of different scales: 38×38×256, 38×38×512, 19×19×1024, 19×19×1024, 10×10×512, 5×5×256, 3×3×256, and 1×1×256. Each of these maps then undergoes a horizontal channel convolution to unify the scale, followed by feature fusion. Finally, a 3×3 convolution is used to reduce aliasing effects before being fed into the detection layer for aerial object classification and detection.

[0084] Step 3-3: Improvement of loss function.

[0085] SSD uses IOU as the loss function, using the union of the predicted box and the true box as the denominator, and the intersection of the predicted box and the true box as the numerator. The calculated value is used to represent the intersection of the predicted box and the true box. The calculated results will be between 0 and 1. The closer the final result is to 1, the closer the distance between the predicted box and the true box is, which also means that the generated prediction box is better. If the final value is close to 0, it means that the two are far apart and the generated prediction box has a large error. Although the IOU intersection-union ratio can reflect the intersection of the predicted box and the true box, when the IOU intersection-union ratio is 0, the IOU does not play a role in the forward propagation of the entire network, which will cause the network to converge more slowly and make the final accuracy unsatisfactory.

[0086] In order to make up for the shortcomings of the IOU loss function, the EIOU bounding box regression loss function is adopted, as shown in formula (6). This loss function separates the influencing factors of the aspect ratio and calculates the length and width of the real box and the predicted box respectively, which makes the model converge faster.

[0087]

[0088] In the formula, IOU represents the intersection-union ratio between the real box and the predicted box, b gt , b represent the center points of the real box and the predicted box respectively, ρ represents the Euclidean distance between the two center points of the real box and the predicted box, c represents the diagonal length of the minimum circumscribed rectangle of the real box and the predicted box, w gt , w represent the width of the real box and the predicted box respectively, h gt , h represent the height of the real box and the predicted box respectively, They represent the width and height of the minimum bounding rectangle that can contain both the true box and the predicted box.

[0089] When locating a target, it is necessary to calculate the position information of the target on an image. The confirmation of the position information is mainly calculated through a bounding box regression module to confirm the target position. What the bounding box regression needs to do is to continuously approach the predicted box to the real box, so as to predict the target that needs to be detected in the image. EIOU decomposes the loss function into three parts: overlap loss, center distance loss, and width and height loss. EIOU not only solves the problem that the IOU predicted box and the real box cannot propagate normally when they do not intersect, but also effectively solves the problem that the intersection of the predicted box and the real box is unclear when the intersection-union ratio is the same. EIOU also uses the distance between the center point of the predicted box and the real box as an evaluation indicator to accelerate the convergence of the network.

[0090] Therefore, the EIOU loss function is used instead of the traditional IOU loss function to calculate the intersection between the predicted box and the real box, which increases the calculation speed of the network and improves the network's detection accuracy for aerial target datasets.

[0091] Step 4: First, process the image to be tested and unify the image size to 300x300. Then, load the backbone network and the trained model. Then, load the image into the model, obtain the location data and category data related to the prediction, filter the prediction results, and finally draw the prediction box.

[0092] The ATD-SSD algorithm is improved mainly in two directions. In order to solve the problems of missed detection, inaccurate prediction frame and incorrect classification of positive and negative samples of the SSD algorithm for small overlapping targets in the air, the target frame of the algorithm is designed and the loss function is optimized to improve the detection effect of the algorithm for overlapping small targets. In addition, in order to solve the problem that the SSD algorithm has weak attention to small targets in the air and low detection accuracy, the backbone network of the algorithm is improved by introducing the attention mechanism and improving the feature extraction method and feature fusion method to improve the detection accuracy of the algorithm and enhance the detection effect. The detection accuracy of the aerial target of the ATD-SSD algorithm is A p and m ap The results are shown in the figure Figure 13 shown.

[0093] The prediction results of the ATD-SSD algorithm for aerial targets are as follows: Figure 14 As shown in the figure, the ATD-SSD algorithm achieves higher detection accuracy for single small aerial targets than using only one optimization method. This solves the problems of feature information loss and low attention for small aerial targets, improving small target detection accuracy. Furthermore, the ATD-SSD algorithm can effectively detect multiple overlapping small targets, and its predicted box sizes are relatively accurate, addressing issues such as misclassification of positive and negative samples and incomplete detection of overlapping small targets.

[0094] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or make equivalent replacements for some or all of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope defined by the claims of the present invention.

Claims

1. A method for detecting aerial targets based on deep learning, characterized by: The method comprises the following steps: Step 1: Collect the aerial target data set according to the characteristics of the aerial target graphics, and expand and process the graphics image; Step 2: Prepare the aerial target dataset according to the specifications of the VOC aerial target dataset, and strictly follow the corresponding format for labeling and naming; Step 3: Improve the target box design, improve the backbone network, and improve the loss function; read the aerial target dataset, and then use the improved backbone network to obtain the improved target box size. Then process the aerial target dataset, extract and save the information of the images in the aerial target dataset, and use the improved loss function to calculate the loss. Then, classify the positive and negative samples. Finally, obtain the final model through continuous calculation and adjustment of weight parameters and save it. The improvement of the target frame design adopts the K-means clustering algorithm to perform cluster analysis on the aerial target dataset used, improve the initialization anchor frame, and obtain the optimal length-to-width ratio of the target frame; the value of K in the K-means clustering algorithm is 6; After clustering, the target frame size of each feature map input to the detection layer is calculated. First, the length and width values ​​of each target frame in the aerial small target dataset are collected. The product of the length and width is the area of ​​the target frame. Then, the ratio of the area of ​​the target frame to the area of ​​the original image is recorded as s. A new array S is formed by the ratios s corresponding to all feature maps. Finally, S is sorted to obtain the first data s of the array S. max and the last data s min The value of the target box size B k Calculate as shown in formula (1); Where m represents the number of feature maps, k∈[1,m]; After calculation, the most suitable target box size for the output feature map can be obtained; Improvements to the backbone network include: Add attention mechanism based on the original network SSD; When the image enters the SSD algorithm, it is processed by the network to obtain a low-level feature map, but it is not propagated forward immediately. Instead, the low-level feature map is processed using the CBAM attention mechanism before being propagated forward. Based on the original network SSD, feature fusion is performed through the FPN network structure; the bilinear interpolation algorithm is used to improve the FPN upsampling method; based on the characteristics of the feature map, two methods, low-level fusion module and high-level fusion module, are used to achieve fusion between feature maps; in the low-level fusion module, the input high-level feature map is first upsampled and then summed with the low-level feature map; the high-level feature fusion module first upsamples the input high-level feature information and then splices it with the low-level feature information. After splicing, the feature map is reduced in dimension using 1×1 convolution; When the image enters the network, the size of the input image is first uniformly adjusted to 300×300, and then the input information is processed in a bottom-up path, passing through an attention module and then through other feature layers, and finally outputting eight feature maps of different scales, with scales of 38×38×256, 38×38×512, 19×19×1024, 19×19×1024, 10×10×512, 5×5×256, 3×3×256, and 1×1×256; then they are scaled uniformly through a horizontal channel convolution, and then feature fusion is performed, and finally a 3×3 convolution is used to reduce the aliasing effect, and finally sent to the detection layer for classification and detection of aerial targets; The improvement of the loss function adopts the EIOU bounding box regression loss function, as shown in formula (6); Among them, IOU represents the intersection-union ratio between the real box and the predicted box, b gt , b represent the center points of the real box and the predicted box respectively, ρ represents the Euclidean distance between the two center points of the real box and the predicted box, c represents the diagonal length of the minimum circumscribed rectangle of the real box and the predicted box, w gt , w represent the width of the real box and the predicted box respectively, h gt , h represent the height of the real box and the predicted box respectively, Respectively represent the width and height of the minimum bounding rectangle that can contain both the real box and the predicted box; Step 4: First process the image to be tested, unify the image size to 300×300, then load the backbone network and the trained model, then load the image into the model, obtain the location data and category data related to the prediction, filter the prediction results, and finally draw the prediction box.

2. The deep learning-based aerial target detection method according to claim 1, wherein: In step 1, the aerial target dataset is collected using Python-based web crawler technology and autonomous drone photography.

3. The deep learning-based aerial target detection method according to claim 1, wherein: In step 2, LabelImg labeling software is used for labeling.

4. The deep learning-based aerial target detection method according to claim 1, wherein: The ATD-SSD target detection algorithm is used in step 3, and the process is as follows: The ATD-SSD network first inputs an image through the input end, and the image size is selected as 300×300 according to actual needs; after applying the K-means clustering algorithm on the aerial target dataset to re-cluster the generated anchor point box, the most appropriate target box size is obtained, and the parameters are modified in the algorithm. Then, preliminary feature extraction is performed on the optimized backbone feature extraction network, and the extracted image features are passed to the improved multi-scale FPN network for feature fusion and enhancement of image features. Finally, predictions of different scales are performed in the prediction layer, and the boundary regression loss function EIOU is added.

Images