Aerial small target detection method based on adaptive receptive field enhancement

Abstract

The application relates to a kind of aerial small target detection methods based on adaptive receptive field enhancement, first propose adaptive receptive field enhancement network module, this module not only can improve the context missing problem existing in hollow convolution, and in the same case of the number of convolution layer branch, it can obtain more number of receptive field scale than receptive field block.In addition, the application further processes the output feature map, which can realize adaptive adjustment of receptive field size for each layer neuron.Adding the application to a general target detection algorithm, experiments show that the application can improve the aerial small target detection rate.

Description

Technical Field

[0001] This invention belongs to the field of image processing technology and relates to an adaptive receptive field enhancement network module. Starting from the receptive field, it solves the problem of insufficient information caused by small targets in aerial images and improves the detection performance of small targets in aerial images. Background Technology

[0002] Object detection is one of the classic visual problems. Currently, deep learning-based methods have become a popular direction for general object detection. However, the current cutting-edge deep learning object detection algorithms are all proposed for general datasets. Detection of aerial images is more challenging because (1) the target is smaller in scale than in high-resolution aerial images, (2) the target is sparse, uneven and concentrated in certain areas, and (3) compared with natural image datasets, small targets account for a larger proportion.

[0003] Currently, significant efforts have been made to improve the detection of small targets in aerial images using general object detection algorithms. To address the problem of insufficient information caused by the small size of targets in aerial images, various improved algorithms have been proposed, focusing on increasing input resolution and feature enhancement. Among these, receptive field enhancement is a commonly used method for feature enhancement. The receptive field is an important attribute of convolutional neural networks (CNNs). The size of the input image region corresponding to a point in the feature map of a certain layer in a CNN is the receptive field size of that layer. The size of the receptive field of a convolutional layer is related to the completeness and robustness of target feature extraction, especially for small targets.

[0004] Given the importance of the receptive field for small target detection in aerial photography, and the fact that current general target detection algorithms based on receptive field enhancement still have shortcomings, for example, some scholars use dilated convolution to enhance the receptive field, but the sparse sampling characteristic of dilated convolution cannot capture contextual information; some scholars use receptive field blocks (RFB) to add receptive field enhancement modules to the network, but if it is necessary to increase the number of receptive field scales, the number of convolutional layer branches must be increased, resulting in a significant increase in computation. Summary of the Invention

[0005] Technical problems to be solved

[0006] To address the issues of false positives and false negatives in aerial image detection due to small target size and insufficient information, this invention provides a method for detecting small aerial targets based on adaptive receptive field enhancement. For the first time, an adaptive receptive field enhancement network module is proposed. This module not only improves the context loss problem inherent in dilated convolutions but also achieves a greater number of receptive field scales than receptive field blocks while maintaining the same number of convolutional layer branches. Furthermore, this invention further processes the output feature map, enabling each neuron to adaptively adjust its receptive field size. Integrating this invention into a general target detection algorithm, experiments show that it can improve the detection rate of small aerial targets.

[0007] Technical solution

[0008] A method for detecting small targets in aerial photography based on adaptive receptive field enhancement, characterized by the following steps:

[0009] Step 1: Input the image to be detected and extract the image feature map;

[0010] Step 2: Input the feature map into an adaptive receptive field enhancement network to enhance the receptive field of the feature map. The adaptive receptive field enhancement network: First, the input feature map is passed through a 1×1 convolution module; then, the processed feature map is input into the receptive field block, and the receptive field is enhanced through K-conv convolutional layers of the concatenated branches of the receptive field block; the output feature maps of multiple branches after processing by the receptive field block are input into a selective convolution module, which calculates the weight values ​​of the feature maps of different branches of the receptive field block, and the feature map and the weight values ​​are weighted and summed to obtain the output feature map; the output feature map is passed through a 1×1 convolution again to restore the number of channels to the input size; finally, it is passed through an eSE module and short-circuit connections are used to further enhance the feature map to obtain the final output feature map.

[0011] Step 3: Input the feature map of the enhanced receptive field into the general object detection method for object detection.

[0012] A further technical solution of the present invention: the general object detection method is YOLOx, Faster R-CNN or YOLOv5 algorithm.

[0013] A further technical solution of the present invention: the convolution kernel K' of the K-conv convolutional layer is represented as:

[0014]

[0015]

[0016] Where F is the transformation matrix, I is the identity matrix and O is the zero matrix, r1 is the intermediate expansion factor, r2 is the internal sharing factor, and c2∈[1,C] B ],c1∈[1,CA ], C A C B These represent the number of channels in feature maps A and B, respectively.

[0017] A further technical solution of the present invention: the selective convolution module is specifically as follows:

[0018] First, the feature maps R from multiple branches will be summed and merged:

[0019]

[0020] Then, global information is embedded through global average pooling, which involves summing all pixels in the feature map and averaging the results to obtain a value that represents the corresponding feature map s∈R. C Therefore, after global average pooling, the feature map s of the c-th channel is... c :

[0021]

[0022] Next, a fully connected layer is used to predict the probability vector V∈R of different feature maps. C' :

[0023] V = F fc (s)=δ(B(W s ))

[0024] Where δ is the loss function, B represents regularization, and W∈R C' ;

[0025] After obtaining the probability vector, a self-attention mechanism is used to adaptively select information at different spatial scales; the softmax function is used to obtain the weight value of the c-th channel.

[0026]

[0027] in, This represents the weight value of the output feature map of the i-th branch in the c-th channel. This represents the probability vector of the output feature map of the i-th branch in the c-th channel;

[0028] Finally, the feature maps R obtained from different branches of the convolutional layer are... i The weighted sums of the calculated weight values ​​generate the final output feature map Foutput. output =[F1,F2,...,F C The output feature map of the c-th channel.

[0029]

[0030]

[0031] A further technical solution of the present invention: When the general target detection method is YOLOx, receptive field modules with different expansion rates are added to the output layer of the feature maps for detecting small, medium and large targets.

[0032] Beneficial effects

[0033] This invention proposes an adaptive receptive field enhancement network from the perspective of receptive field enhancement, which improves the detection rate of general target detection algorithms for small aerial targets.

[0034] First, feature reuse is achieved through bidirectional connections. This allows for a larger receptive field and better extraction of target features, while also providing a richer variety of receptive field scales, all while maintaining the same number of branches and identical convolutional parameters for each branch. Second, K-conv convolutional layers are introduced into the branches to enhance the receptive field and address the contextual information loss issue caused by dilated convolutions. Finally, selective convolutional modules are added to the network, enabling each neuron to adaptively adjust its receptive field size, avoiding a fixed size for each layer. To demonstrate the superiority of this invention, performance comparison tests were conducted using the YOLOv algorithm as a base and various other receptive field modules added. This invention outperforms the other modules in both detection accuracy and speed.

[0035] To further verify the effectiveness and generalization of this invention, it was incorporated into the YOLOx, Faster R-CNN, and YOLOv5 algorithms, respectively. Experimental results show that this invention is effective in improving the detection of small targets in aerial photography. Attached Figure Description

[0036] The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Throughout the drawings, the same reference numerals denote the same parts.

[0037] Figure 1 : RFB structure diagram;

[0038] Figure 2 OSA module diagram;

[0039] Figure 3 : A schematic diagram of dilated convolution with a dilation rate of 4;

[0040] Figure 4 A schematic diagram of a K-conv with r1=4 and r2=3;

[0041] Figure 5 Selective convolution module diagram;

[0042] Figure 6 Overall framework diagram of the adaptive receptive field enhancement network;

[0043] Figure 7 Improved YOLOx network diagram;

[0044] Figure 8 Overall process flowchart;

[0045] Figure 9 : Schematic diagram of the overlap rate between the detection box and the truth box;

[0046] Figure 10 Actual detection images of receptive field enhancement modules with different expansion rates: (a) Original image; (b) No. 1; (c) No. 2; (d) No. 3; (e) No. 4; (f) No. 5; (g) No. 6; (h) No. 7; (i) No. 9;

[0047] Figure 11 : mAP index variation graphs for different receptive field enhancement structures: (a) mAP (IOU = 0.5) variation graph; (b) mAP (IOU = 0.5: 0.95) variation graph;

[0048] Figure 12 : Changes in mAP metrics for each algorithm: (a) mAP (IOU = 0.5) change graph; (b) mAP (IOU = 0.5: 0.95) change graph;

[0049] Figure 13 Comparison of Faster R-CNN algorithm detection: (a1) Original image; (b1) Original algorithm; (c1) Improved algorithm; (a2) Original image; (b2) Original algorithm; (c2) Improved algorithm; (a3) ​​Original image; (b3) Original algorithm; (c3) Improved algorithm; (a4) Original image; (b4) Original algorithm; (c4) Improved algorithm.

[0050] Figure 14 YOLOv5-s algorithm detection comparison images: (a1) Original image; (b1) Original algorithm; (c1) Algorithm improvement; (a2) Original image; (b2) Original algorithm; (c2) Algorithm improvement; (a3) ​​Original image; (b3) Original algorithm; (c3) Algorithm improvement; (a4) Original image; (b4) Original algorithm; (c4) Algorithm improvement.

[0051] Figure 15 YOLOx-s algorithm detection comparison images: (a1) Original image; (b1) Original algorithm; (c1) Algorithm improvement; (a2) Original image; (b2) Original algorithm; (c2) Algorithm improvement; (a3) ​​Original image; (b3) Original algorithm; (c3) Algorithm improvement; (a4) Original image; (b4) Original algorithm; (c4) Algorithm improvement. Detailed Implementation

[0052] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. Furthermore, the technical features involved in the various embodiments of this invention described below can be combined with each other as long as they do not conflict with each other.

[0053] This invention provides an aerial small target detection method based on adaptive receptive field enhancement. The proposed adaptive receptive field enhancement network design is as follows: To obtain receptive fields of multiple scales, the OSA module from VoVNet is used. Feature reuse is implemented in the branches, connecting the output of each branch with the input of subsequent branches, so that the output feature map of the previous branch becomes the input of the next branch. After multiple convolutions in the subsequent branches, the scale and number of receptive field scales of the feature map are increased. To solve the problem of missing contextual information caused by the sparse sampling characteristics of dilated convolution, K-conv is used instead of dilated convolution in the convolutional layer branches of the OSA module to enhance the receptive field of the feature map. To enable neurons to adaptively adjust the receptive field size, a selective convolution module based on a self-attention mechanism is added to the OSA module, aggregating feature maps according to weights to obtain different attention regions in the receptive field. The adaptive receptive field enhancement network proposed in this invention can be added to different general target detection algorithms to improve the detection rate of small targets in aerial photography. The specific implementation steps are as follows:

[0054] Step 1: Creating a small target dataset from aerial images

[0055] To address the need for high-resolution aerial image small target detection, and based on the COCO dataset's classification (targets smaller than 32×32 pixels are classified as small targets, those larger than 96×96 pixels as large targets, and the rest as medium targets), this invention selects three publicly available aerial remote sensing datasets: Wuhan University RSOD, University of Science and Technology of China UCAS-AOD, and Northwestern Polytechnical University NWPU VHR-10. Aircraft, ships, and vehicles are then identified as targets, and an aerial image small target dataset is created. Specific information is shown in Table 1.

[0056] Step 2: Improve the branching structure of the receptive field block

[0057] Receptive field blocks (RFBs) are often used to enhance receptive fields in general algorithms. They are created by designing convolutional kernels of different sizes within convolutional layers, thus forming a parallel multi-branch structure and obtaining receptive fields at various scales, as shown in the attached figure. Figure 1 Therefore, if a small object detection network structure wants to obtain a larger receptive field, it can only be achieved by increasing the branch structure of the convolutional layers and increasing the size of the convolutional kernels, but this approach often leads to an increase in computational cost.

[0058] To address the shortcomings of the receptive field block branching structure, the improved receptive field module network structure, drawing inspiration from the OSA (One-Shot Aggregation) module of vovNet, is shown in the attached figure. Figure 2 The OSA module features multiple receptive fields representing various features. Each convolutional layer in this module contains bidirectional connections: one connects to the next layer as input to subsequent branches, while the other aggregates into the final output feature map. These bidirectional connections allow the output feature map of each layer to be processed again by subsequent convolutional layers. This feature reuse method, with the same number of branches and identical convolutional parameters for each branch, not only achieves a larger receptive field and better extraction of target features, but also provides a richer variety of receptive field scales, better adapting to changes in target scale.

[0059] Step 3: Introduce K-conv convolutional layers

[0060] The Receptive Field Block (RFB) introduces dilated convolutional layers into the branch structure to enhance the receptive field. Compared to ordinary convolution, dilated convolution adds a dilation parameter, which refers to the spacing between each point in the convolutional kernel. The dilation rate is often achieved by padding with zeros, which leads to dilated convolution failing to capture contextual information when the dilation rate is too high, as shown in the attached diagram. Figure 3 When the dilation rate is 4, only 9 of the 81 vectors in the convolutional block are used in the computation.

[0061] Due to the sparse sampling characteristic of dilated convolution leading to a loss of contextual information, a K-conv convolutional layer is introduced in the OSA module. K-conv (Kronecker convolution) is inspired by the Kronecker product in mathematical applications, extending the convolution kernel through the Kronecker product. If A is an m×n matrix, [a 11 ...a mn ] represents the matrix coefficients, and B is an r×s matrix. If the matrix is ​​mr×ns, then the mathematical definition of the Kronecker product is as follows:

[0062]

[0063] For standard convolution, the input feature map is represented as: The output feature map is represented as Where H, W, and C represent the height, width, and number of channels of feature maps A and B, respectively, and C A C B These represent the number of channels in feature maps A and B, respectively. In standard convolution, the convolution kernel is represented as... The bias is represented as In K-conv, a transformation matrix F is introduced, which is set as a fixed r1×r1 matrix. r1 is an intermediate dilation factor controlling the dilation rate of the convolution. The convolution kernel K is expanded by calculating the Kronecker product of the transformation matrix F and the convolution kernel K, so that the convolution kernel in K-conv expands from (2k+1)×(2k+1) to (2k+1)r1×(2k+1)r1. To avoid introducing additional parameters, the matrix F is set as a combination of the identity matrix I and the zero matrix O, where I is an r2×r2 square matrix with all elements being 1. The size of the sub-region is controlled by setting the internal sharing factor r2 to capture the feature vector and the shared filter vector. Therefore, the convolution kernel K' in K-conv is represented as:

[0064]

[0065]

[0066] Where c2∈[1,C B ],c1∈[1,C A The K-conv convolution with r1=4 and r2=3 is shown in the attached figure. Figure 4 .

[0067] Compared to dilated convolution that simply inserts a zero-expansion kernel, K-conv expands the ordinary convolution kernel by performing a Kronecker product between the transformation matrix F and the kernel K, thus avoiding the omission of vectors at zeros. The intermediate dilation factor r1 controls the dilation rate; as r1 increases, the receptive field also expands. Since the transformation matrix F only contains 1 and 0 values, it does not introduce additional parameters. Furthermore, because the transformation matrix F has an r2×r2 identity submatrix, more vectors participate in the computation, allowing K-conv to capture local contextual information that dilated convolution ignores.

[0068] Step 4: Introduce the selective convolution module

[0069] After receptive field enhancement, the various receptive field feature maps output by the convolutional layers are directly aggregated in the last layer. This results in a fixed receptive field size for each neuron in each layer. However, in neuroscience, the receptive field size of neurons in the visual cortex is modulated by stimulation. Therefore, to enable neurons to adaptively adjust the receptive field size based on multi-scale information from the input, a selective convolutional module based on a self-attention mechanism is added to the network to aggregate feature maps with different weights, as shown in the attached diagram. Figure 5 .

[0070] Input feature map F input ∈R C×H×W R i(i∈1,2,3,4) represents the feature maps of multiple receptive fields output after the input feature map passes through the feature extraction convolutional layer (taking a four-branch model as an example). In order for the neuron to adaptively adjust the size of the receptive field according to the stimulus content, it is necessary to calculate the influence of different receptive field feature maps on the next layer, i.e., the weight values.

[0071] First, the feature maps R from multiple branches will be summed and merged:

[0072]

[0073] Then, global information is embedded through global average pooling, which involves summing all pixels in the feature map and averaging the results to obtain a value that represents the corresponding feature map s∈R. C Therefore, after global average pooling, the feature map s of the c-th channel is... c :

[0074]

[0075] Next, a fully connected layer is used to predict the probability vector V∈R of different feature maps. C' :

[0076] V = F fc (s)=δ(B(W s (6)

[0077] δ is the loss function, B represents regularization, and W∈R C' .

[0078] After obtaining the probability vector, a self-attention mechanism is used to adaptively select information at different spatial scales. The softmax function is used to obtain the weight value of the c-th channel.

[0079]

[0080] in, This represents the weight value of the output feature map of the i-th branch in the c-th channel. This represents the probability vector of the output feature map of the i-th branch in the c-th channel.

[0081] Finally, the feature maps R obtained from different branches of the convolutional layer are... i The weighted sum of (i∈1,2,3,4) and the obtained weight values ​​generates the final output feature map F. output F output =[F1,F2,...,F C The output feature map of the c-th channel.

[0082]

[0083]

[0084] Step 5: Add the adaptive receptive field enhancement network to the general object detection algorithm YOLOx and test the expansion factor performance.

[0085] The overall framework of the adaptive receptive field enhancement network is shown in the attached figure. Figure 6 First, the input feature map is passed through a 1×1 convolutional module to reduce the number of channels and parameters. Then, the processed feature map is input into the receptive field block, where K-conv convolutional layers of the concatenated branches of the receptive field block enhance the receptive field. Next, the output feature maps from multiple branches processed by the receptive field block are input into a selective convolutional module. This module calculates the weights of the feature maps from different branches of the receptive field block and sums the weighted feature maps with these weights to obtain the output feature map. The output feature map is then passed through another 1×1 convolution to restore the number of channels to the input size. Finally, it passes through an eSE module with short-circuit connections to further enhance the feature map, resulting in the final output feature map. The adaptive receptive field enhancement network is added to YOLOx as shown in the attached diagram. Figure 7 First, the image is input into the YOLOx network; then, the YOLOx backbone network extracts the image feature map and outputs it; next, the output feature map is enhanced by an adaptive receptive field enhancement network; then, the enhanced receptive field feature map is fused through the neck layer, and finally, target detection is performed in the detection head. The improved YOLOx's detection rate for small aerial targets is affected by the expansion factors r1 and r2. Experiments were designed to select r1 and r2 that optimize performance. Considering the multi-scale target detection characteristics of YOLOx, receptive field modules with different expansion rates were added to the output layers of feature maps for small, medium, and large targets. First, experiments were conducted by increasing the receptive field enhancement module according to the expansion rate; then, experiments were conducted by decreasing the receptive field enhancement module according to the expansion rate; finally, experiments were conducted with the same expansion rate. Based on the combined results of subjective and objective experiments, the expansion factor module that optimizes performance was selected.

[0086] Step Six: Set the optimal expansion factor and add the adaptive receptive field enhancement network to the general object detection algorithm to detect small targets in aerial photography datasets.

[0087] The expansion factor of the adaptive receptive field enhancement network was set to the optimal value obtained experimentally. The network designed in this invention was then incorporated into other general object detection algorithms, YOLOv5 and Faster R-CNN. Optimal weights were obtained through training, and the trained weights and algorithm network structure were loaded during testing. Detection was performed on an aerial small target dataset, and the experimental results were compared with the original algorithm's detection results to verify the effectiveness and generalization of the adaptive receptive field enhancement network. The overall steps are attached. Figure 8First, to address the shortcomings of the receptive field block (RFB), an adaptive receptive field enhancement network is proposed. Next, performance experiments on the expansion factor are conducted using YOLOx as the base algorithm to select the optimal expansion factor. Then, other types of receptive field enhancement networks are added to the YOLOx base algorithm for performance comparison. Finally, the expansion factor of the adaptive receptive field enhancement network is set to the optimal value obtained from the experiments, and it is added to other general object detection algorithms such as YOLOv5 and Faster R-CNN. The optimal weights are obtained through training, and the training weights and algorithm network structure are loaded during testing to detect small aerial targets on a dataset.

[0088] 1. Experimental conditions

[0089] The experimental test environment used Ubuntu 20.04 system: Intel(R) Core(TM) i3-8350 CPU@3.4GHz, 16GB of memory, and NVIDIA GeForce GTX 2080Ti GPU with 11GB of video memory. This paper selects Faster R-CNN (with ResNet50 as the backbone network), YOLOv5 (YOLOv5-s model), and YOLOx (YOLOx-s) as the basic algorithms to verify the effectiveness of the proposed algorithm. Specific experimental parameters are shown in Appendix Table 2.

[0090] To address the need for high-resolution aerial image small target detection, and based on the COCO dataset's classification (targets smaller than 32×32 pixels are classified as small targets, those larger than 96×96 pixels as large targets, and the rest as medium targets), this invention selects three publicly available aerial remote sensing datasets: Wuhan University RSOD, University of Science and Technology of China UCAS-AOD, and Northwestern Polytechnical University NWPU VHR-10. Aircraft, ships, and vehicles are then identified as targets, and an aerial image small target dataset is created. Specific information is shown in Table 1.

[0091] 2. Experiment Content

[0092] This invention draws inspiration from the OSA module in VoVNet. First, it connects the output of each branch to the input of the next branch through bidirectional connections. The output feature map of the preceding branch is then convolved by the following branch to further increase the receptive field of the feature map. Next, it introduces K-conv convolutional layers into the OSA module to replace dilated convolutions to enhance the receptive field, because although dilated convolutions are often used to enhance the receptive field, they lack the ability to capture contextual information. Finally, to enable neurons to adaptively adjust the size of their receptive field, a selective convolution module based on a self-attention mechanism is introduced into the OSA module to aggregate feature maps according to weights, thereby obtaining different attention regions within the receptive field.

[0093] To test the impact of the interval expansion factor in the adaptive receptive field enhancement network module on detection performance, based on the YOLOx algorithm, and considering the multi-scale object detection characteristics of YOLOx, experiments were designed with the following approaches: adding receptive field enhancement modules with progressively increasing expansion rates to the output layers of small, medium, and large object feature maps; adding receptive field enhancement modules with progressively decreasing expansion rates; and adding modules with the same expansion rate. To verify the superiority of the adaptive receptive field enhancement network module, comparative experiments were designed with receptive field modules ASPP, RFB, and RFB-s added to the YOLOx algorithm. To verify the generalization and robustness of the adaptive receptive field enhancement network module, the module was added to the two-stage object detection algorithm Faster R-CNN, the one-stage object detection algorithm YOLOv5, and the anchor-free object detection algorithm YOLOx, and experiments were designed accordingly.

[0094] 3. Classification Evaluation Indicators

[0095] The detection quality evaluation index used in this invention for target detection is mean accuracy (mAP), and the detection speed is represented by the average detection time (FPS) per image. The basic concepts involved in calculating mAP are introduced below.

[0096] (1) Intersection over Union (IoU): In object detection, both the detection box and the ground truth box are rectangles. The intersection of the two boxes divided by their union is the IoU value. The IoU value represents the degree of overlap between the detection box and the ground truth box, and is used to measure the accuracy of localization. When the overlap rate is greater than a set threshold, the localization is considered accurate; otherwise, it is considered incorrect. When the two boxes completely overlap, the IoU reaches its maximum value of 1.

[0097] Appendix Figure 9 This is a diagram illustrating the overlap between the detection bounding box and the ground truth bounding box, where Gt represents the target ground truth bounding box and Dt represents the target detection bounding box. The IOU calculation expression is as follows:

[0098]

[0099] Where Gt∩Dt represents the intersection of the truth box and the detection box, and Gt∪Dt represents the union of the truth box and the detection box.

[0100] (2) True Positive (TP): The number of positive samples that are correctly identified as positive samples, representing those samples that were successfully detected.

[0101] (3) True Negatives (TN): The number of negative samples that are correctly identified as negative samples, indicating that the background is not misclassified as the target.

[0102] (4) False Positives (FP): The number of negative samples that are incorrectly identified as positive samples, indicating that the background is identified as the target, i.e., false detection.

[0103] (5) False Negatives (FN): The number of positive samples that are incorrectly identified as negative samples, indicating the number of samples for which the target is incorrectly identified.

[0104] Errors are attributed to background and not detected, i.e., missed detections.

[0105] (6) Precision: Also known as accuracy. It refers to the proportion of samples that are actually positive out of the positive samples in the test output. The formula is:

[0106]

[0107] (7) Recall: Also known as the percentage of positive instances that are actually identified. It refers to the proportion of positive instances that are correctly identified out of all positive instances. The formula is:

[0108]

[0109] In VOC2010 and later, Average Precision (AP) is the area enclosed by the Precision-Recall (PR) curve and the coordinate axis, used to measure the detection quality of a class of targets. The calculation formula is as follows:

[0110]

[0111] When detecting multiple target classes, averaging the AP across all classes yields the mean average precision (mAP), which measures the detection quality of multi-class targets. The calculation formula is as follows:

[0112]

[0113] M represents the number of categories, i∈(1,M).

[0114] 4. Simulation Testing

[0115] To verify the impact of the interval expansion factor in the adaptive receptive field enhancement network module on detection performance, Table 3 uses YOLOx as the base algorithm. Considering the multi-scale target detection characteristics of YOLOx, receptive field modules with different expansion rates were added to the output layers of feature maps for small, medium, and large targets. First, experiments were designed with increasing expansion rates (numbers 2, 3, and 4); then, experiments were designed with decreasing expansion rates (number 5); finally, experiments were designed with the same expansion rate (numbers 6, 7, and 8). The objective experimental results are shown in Table 3. As can be seen from the table, the performance is improved compared to the original algorithm (number 1). However, considering all detection metrics, especially the detection results for small targets, experiment number 3 yields the best results. The following analysis explains this: Numbers 2, 3, 4, and 5 represent the addition of receptive field enhancement modules with different expansion rates to the output layers of feature maps at different scales. Numbers 2, 3, and 4 all follow the principle of adding receptive field enhancement modules with progressively increasing expansion rates to the output layers of feature maps for small, medium, and large targets, while number 5 is set in the opposite direction. From the detection results of items 2, 3, and 4, item 3 shows the best result. This is because as the expansion rate increases, the receptive field increases, leading to higher target detection accuracy (as seen in items 2 and 3). However, continuously increasing the expansion rate can cause information loss for small and medium-sized targets, resulting in a decrease in detection accuracy (as seen in items 3 and 4). Item 5's overall detection result is worse than item 3 because the addition of a receptive field enhancement module with a large expansion rate to the output layer for small target detection in item 5 causes information loss for small and medium-sized targets, thus reducing the detection performance for these targets. Items 6, 7, and 8, based on item 3, keep the receptive field enhancement modules for different feature map output layers unchanged, but the actual results are worse than item 3. Therefore, considering all objective indicators, the receptive field module setting in item 3 achieves the best detection results.

[0116] Figure 10 Actual detection images of receptive field enhancement modules with different expansion rates are given, among which... Figure 10 (a) is the original image. Figure 10 (b) to Figure 10 (i) Actual detection results of experiments 1 to 8 as set in Table 3. From the actual detection results, the detection result of number 3 is the best. Figure 10 (b) There are two missed detections in the original algorithm detection graph corresponding to serial number 1 (pointed to by the yellow arrows). Figure 10 (c) Figure 10 (d) Figure 10 (e) For sequences 2, 3, and 4, receptive field enhancement modules with increased expansion rates were added. In sequences 2 and 4, missed detections occurred in the detection maps, while sequence (3) did not. This is mainly because excessive increases in expansion rate and receptive field do not improve the detection rate; instead, they can easily lead to missing information on medium and small targets. The expansion rate can only be increased moderately. Sequence 5 has the opposite receptive field module settings to sequence 3, resulting in... Figure 10 (f) Two cases of missed detection occurred. Figure 10 (g) Figure 10 (h) Figure 10 (i) Missed detections also occurred in the experimental detection diagrams corresponding to serial numbers 6, 7, and 8. Considering both subjective and objective detection results, the adaptive receptive field module set in serial number 3 has a better detection effect.

[0117] To verify the superiority of the adaptive receptive field enhancement network module, a comparative experiment was designed based on the YOLOx algorithm, with receptive field modules added respectively: ASPP-D module, RFB module, and RFB-s module. Table 4 shows that the detection performance of all algorithms improved after adding the receptive field enhancement module, with the algorithm showing the greatest performance improvement after adding the adaptive receptive field enhancement network module. Further analysis shows that the original algorithm with the addition of the ASPP-D module mAP... 50 Improved by 0.3%, mAP 50:95 The value increased by 0.8%; the original algorithm added the RFB module mAP 50 Increased by 0.4%, mAP 50:95 The value increased by 0.4%; the original algorithm was improved by adding the RFB-s module mAP. 50 Improved by 0.7%, mAP 50:95 The value was improved by 0.6%; the original algorithm was improved by adding the mAP module proposed in this paper. 50 Increased by 1.5%, mAP 50:95 The value increased by 2.1%. For a more intuitive comparison, Figure 11 The mAP (meta-pass absorption) index of different receptive field enhancement structures was plotted, and the graph shows that the algorithm incorporating the proposed module has the best detection performance. Furthermore, the algorithm with the proposed module is the fastest in terms of detection speed. Therefore, whether comparing detection accuracy or detection speed, the proposed receptive field enhancement module outperforms other receptive field enhancement modules.

[0118] To verify the generalization and robustness of the adaptive receptive field enhancement network module, experiments were designed by adding the module to the two-stage object detection algorithm Faster R-CNN, the one-stage object detection algorithm YOLOv5, and the anchor-free object detection algorithm YOLOx. Table 5 presents the comparative experimental results of these algorithms before and after adding the adaptive receptive field enhancement module on an aerial small object dataset. Furthermore, to more intuitively represent the results, Figure 12 A graph showing the changes in mAP for each algorithm was plotted. The table shows the mAP of Faster R-CNN after adding the module. 50 The value increased by 1.5%, mAP 50:95 The value increased by 0.9%, mainly reflected in the improved detection rate of small and medium-sized targets; after adding the module to YOLOv5-s, the mAP 50The value increased by 1.2%, mAP 50:95 The value increased by 0.7%, mainly reflected in the improved detection rate of small and medium-sized targets; after adding the module to YOLOx-s, the mAP 50 The value increased by 1.5%, mAP 50:95 The value improved by 2.1%. Combining the three object detection algorithms, objective evaluation shows that the detection performance has improved across all categories, especially for the detection of medium and small objects. Faster R-CNN, however, performed the worst overall. To further compare YOLOv5-s and YOLOx-s, the speed of the two algorithms before and after processing was compared. 510 images were randomly tested, and the processing time for each image was calculated, including only the inference time required for each image. This process was repeated three times, and the average value was taken as the final time. Table 6 shows the algorithm test metrics for YOLOv5-s and YOLOx-s. It can be seen that although the detection speed decreased slightly after adding the receptive field enhancement module, the impact on time was relatively small. Furthermore, YOLOx-s has a faster detection speed in terms of the average inference time for a single image.

[0119] Figure 13 Four sets of comparison images of detection by the Faster R-CNN algorithm are given, among which Figure 13 (a) Corresponding to the original image, Figure 13 (b) Corresponding to the detection image of the original algorithm Figure 13 (c) The detection map after adding the adaptive receptive field enhancement module. From Figure 13 It can be seen that the original algorithm has false positives in the first, second, and third groups of images, that is... Figure 13 (b1) Figure 13 (b2) Figure 13 (b3) The red arrow points to the fourth group of images; there are missed detections, with the original algorithm missing four instances, namely... Figure 13 In (b4), the yellow arrow points to two instances where the improved algorithm misses detection. Figure 13 (c4) indicates the location pointed to by the yellow arrow. Therefore, combining subjective and objective evaluation results, Faster R-CNN with the addition of the adaptive receptive field enhancement module has better detection performance.

[0120] Figure 14 Four sets of comparison images of YOLOv5-s algorithm detection are given, among which Figure 14 (a) Corresponding to the original image, Figure 14 (b) Corresponding to the detection image of the original algorithm Figure 14 (c) The detection map after adding the adaptive receptive field enhancement module. From Figure 14 It can be seen that the original algorithm has false positives in the first and second sets of images, that is... Figure 14 (b1) Figure 14(b2) The red arrow points to the area where the original algorithm missed detections in the third and fourth groups of images. Figure 14 (b3) Figure 14 The target is located at the point indicated by the yellow arrow in (b4), even if... Figure 14 (a4) is not marked, but it is still in Figure 14 (c4) was successfully detected. Therefore, YOLOv5-s with the addition of the adaptive receptive field enhancement module has better detection performance.

[0121] Figure 15 Four sets of comparison images of YOLOx-s algorithm detection are given, among which Figure 15 (a) Corresponding to the original image, Figure 15 (b) Corresponding to the detection image of the original algorithm Figure 15 (c) The detection map after adding the adaptive receptive field enhancement module. From Figure 15 It can be seen that the original algorithm has false positives in the first and second sets of images, that is... Figure 15 (b1) Figure 15 In (b2), the red arrow points to an area where a non-target object was mistaken for a target object; in the third and fourth groups of images, the original algorithm had some missed detections, i.e. Figure 15 (b3) Figure 15 (b4) Figure 15 In (c4), as indicated by the yellow arrow, although the improved algorithm also missed some detections in the fourth set of images, the improved algorithm only missed one, while the original algorithm missed two. Therefore, considering both subjective and objective results, YOLOx-s with the addition of the adaptive receptive field enhancement module has better detection performance.

[0122] Table 1. Small Target Dataset from Aerial Images

[0123]

[0124] Table 2 Specific Parameter Design

[0125]

[0126] Table 3 Comparison of Receptive Field Enhancement Modules with Different Expansion Rates

[0127]

[0128]

[0129] Table 4 Comparison of detection performance of different receptive field enhancement structures

[0130]

[0131] Table 5 Comparison of Results from Different Algorithms

[0132]

[0133] Table 6 Speed ​​Test of the Algorithm

[0134]

[0135] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the scope of the technology disclosed in the present invention, and such modifications or substitutions should all be covered within the scope of protection of the present invention.

Claims

1. A method for detecting small targets in aerial photography based on adaptive receptive field enhancement, characterized in that... The steps are as follows: Step 1: Input the image to be detected and extract the image feature map; Step 2: Input the feature map into an adaptive receptive field enhancement network to enhance the receptive field of the feature map; the adaptive receptive field enhancement network: first passes the input feature map through... The convolutional module then inputs the processed feature map into the receptive field block, which adopts a multi-branch concatenated architecture. The branches are cascaded sequentially, and the output of the preceding branch is used as the input of the following branch. K-conv convolutional layers are deployed inside each concatenated branch to replace dilated convolutions. The receptive field is enhanced through the K-conv convolutional layers in each concatenated branch. The output feature maps of the multiple branches after processing by the receptive field block are input into the selective convolutional module. The selective convolutional module calculates the weight values ​​of the feature maps of different branches of the receptive field block, and the feature maps and weight values ​​are weighted and summed to obtain the output feature map. The output feature map is processed again. Convolution restores the number of channels to the input size; finally, the feature map is enhanced by the eSE module and short-circuit connections to obtain the final output feature map. The convolution kernel of the K-conv convolutional layer Represented as: in, F The transformation matrix is... I For identity matrix and O It is a zero matrix. It is the intermediate expansion factor. For internal shared factors, , , , Representing feature maps respectively A , B The number of channels; The selective convolution module is as follows: First, the feature maps of multiple branches will be summed and fused. : Then, global information is embedded through global average pooling, which involves summing all pixels in the feature map and averaging the results to obtain a value that represents the corresponding feature map. Therefore, after global average pooling, the th Feature map of each channel : Next, a fully connected layer is used to predict the probability vectors of different feature maps. : in, It is a loss function. B Indicates regularization, ; After obtaining the probability vector, a self-attention mechanism is used to adaptively select information at different spatial scales; the softmax function is used to obtain the probability vector. Individual channel weight values: in, Indicates the first The first channel The weight values ​​of the feature maps output by each branch. Indicates the first The first channel The probability vector of the output feature map of each branch; Finally, the feature maps obtained from different branches of the convolutional layer are... The weighted sums of the calculated weight values ​​are used to generate the final output feature map. , , No. Output feature map of each channel : Step 3: Input the feature map of the enhanced receptive field into a general object detection method for object detection; When the general object detection method is YOLOx, receptive field modules with different expansion rates are added to the output layer of feature maps for detecting small, medium and large objects.

2. The aerial small target detection method based on adaptive receptive field enhancement according to claim 1, characterized in that... The general object detection method mentioned is the YOLOx, Faster R-CNN, or YOLOv5 algorithm.

Images