A general optimization method for YOLO series target detection in remote sensing images

Abstract

The application relates to a general optimization method for YOLO series target detection in remote sensing images. By integrating a global semantic information extraction (GSIE) module and an adaptive feature fusion (AFF) module into the traditional YOLO algorithm framework, the detection performance of small targets in remote sensing images is significantly improved. The GSIE module expands the receptive field through multi-angle rotation and depth separable convolution to enhance global semantic capture. The AFF module optimizes the fusion of feature maps of different scales through dynamic weight to reduce information loss and improve detail recognition. These improvements not only improve the recognition accuracy of small targets by the model, but also reduce the computational complexity, making the model more suitable for real-time applications with limited resources. The improved algorithm is superior to the traditional method in terms of small target detection accuracy, speed and resource consumption. Especially in the scene of dense small targets in complex background, the improved algorithm effectively reduces false positives and misses, showing a wide application prospect and practical value.

Description

Technical Field

[0001] This invention relates to the field of deep learning object detection technology, specifically to a general optimization method for YOLO series object detection in remote sensing images. Background Technology

[0002] The YOLO algorithm is widely used in various fields, such as video surveillance and autonomous driving, due to its speed and accuracy. While the YOLO family of algorithms performs excellently in general image detection, it still faces many challenges in remote sensing image detection, mainly due to the complex backgrounds and diversity of small targets in these images. In traditional YOLO algorithms, Feature Pyramid Network (FPN) and Path Aggregation Network (PANet) offer some assistance, but they still suffer from information loss and processing redundancy during multi-scale feature fusion. Furthermore, researchers have introduced various solutions that enable the YOLO family of networks to effectively address the challenge of detecting small targets in RSI. However, these solutions only focus on local information, neglecting the importance of global semantic information interaction. Global semantic information plays a crucial role in understanding the relationship between the target and the background, as well as the overall semantic structure of the scene.

[0003] The main technical challenges currently facing target detection in remote sensing images include low detection accuracy due to complex backgrounds and small target sizes, as well as high computational resource consumption. In particular, when applying the traditional YOLO algorithm for remote sensing image processing, the following problems exist: Information loss: Due to the relatively small size of targets and the complex background in remote sensing images, the traditional YOLO algorithm is prone to losing detailed information about small targets during multiple downsampling and pooling processes. This loss of information directly affects the accuracy and reliability of target detection.

[0004] Insufficient feature fusion: Although the introduction of Feature Pyramid Network (FPN) and Path Aggregation Network (PANet) can improve the information fusion problem to some extent, in practical applications, the feature fusion process still faces problems of low efficiency and incomplete fusion. Especially when fusing features at different scales, high-level semantic information and low-level detailed information are difficult to combine effectively, resulting in poor recognition of small-sized targets in practical applications.

[0005] High computational complexity: When processing remote sensing images, traditional YOLO series models have high computational complexity and require a lot of computing resources due to the need to process a large number of small targets and complex backgrounds, which is not conducive to application in resource-constrained environments.

[0006] Patent application CN 109539726 A, entitled "A Single-Stage Remote Sensing Image Target Detection Algorithm," discloses a single-stage remote sensing image target detection algorithm. This algorithm uses Yolo v3 as a benchmark, adding pyramid convolution to the feature extraction network of Yolo v3, replacing the detection network of Yolo v3 with a path aggregation network, and improving the upsampling method of the path aggregation network to transposed convolution. Finally, spatial pyramid pooling is added as an intermediate connection between the feature extraction network and the detection network. While this target detection algorithm improves the detection accuracy of remote sensing images with complex backgrounds to some extent, it still can only detect and identify large targets such as bridges, docks, airplanes, basketball courts, and swimming pools, and cannot effectively identify small targets.

[0007] Therefore, it is necessary to invent a semi-supervised deep learning model for detecting the density distribution of vegetation in remote sensing images to solve the above problems. Summary of the Invention

[0008] The purpose of this invention is to provide a general optimization method for YOLO series target detection in remote sensing images. By integrating the Global Semantic Information Extraction (GSIE) module and the Adaptive Feature Fusion (AFF) module into the traditional YOLO algorithm framework, the detection performance of small targets in remote sensing images is significantly improved, especially when dealing with dense small target scenes in complex backgrounds, effectively reducing false detections and missed detections.

[0009] To achieve the above objectives, the present invention provides the following technical solution: A general optimization method for YOLO target detection in remote sensing images is provided. This general optimization method is implemented using an optimized YOLO algorithm that includes a Global Semantic Information Extraction (GSIE) module and an Adaptive Feature Fusion (AFF) module. The general optimization method includes the following steps: S1. Data preprocessing and feature extraction: After extracting features from the input remote sensing image using the backbone network of the YOLO algorithm (such as Darknet), three different scale feature maps are output: high-resolution feature map, medium-resolution feature map and low-resolution feature map. These feature maps correspond to different levels of semantic information. S2. Global Semantic Information Extraction (GSIE) Module Processing: The Global Semantic Information Extraction (GSIE) module is designed to expand the model's receptive field and enhance its ability to capture global semantic information. Specifically, the GSIE module selects a horizontal line passing through the center point along the channel direction in the feature map as the rotation axis, and performs a 90° rotation on each feature map. ° 180 ° and 270 °The original feature map is rotated to generate a rotated feature map. The original feature map and the rotated feature map are then merged along the channel dimension to form a multi-view feature map with rich semantic information. In this way, the GSIE module can capture target information from multiple angles and significantly improve the ability to recognize small targets in complex backgrounds. S3. Adaptive Feature Fusion (AFF) Module Processing: The Adaptive Feature Fusion (AFF) module optimizes the feature fusion process by dynamically adjusting the fusion weights of feature maps at different scales. Specifically, the adaptive feature fusion module uses 1×1 convolution to adjust the number of channels in low-resolution and medium-resolution feature maps to match the high-resolution feature maps. It then uses upsampling technology to unify the size of these feature maps and introduces three dynamic weights α, β, and δ. These dynamic weights are normalized using the softmax function to adjust the contribution rate of different feature maps. Through this weight adjustment mechanism, the AFF module can effectively reduce information loss while enhancing the expressive power of the feature maps. S4. Target Detection: The feature map processed by the global semantic information extraction module and the adaptive feature fusion module is sent to the detection head (such as the detection head of YOLOv5) to locate and classify the target. This step uses the processed high-quality feature map to improve the accuracy and reliability of small target detection in remote sensing images.

[0010] The beneficial effects of this invention are as follows: This invention significantly improves the detection performance of small targets in remote sensing images by integrating a Global Semantic Information Extraction (GSIE) module and an Adaptive Feature Fusion (AFF) module into the traditional YOLO algorithm framework. The GSIE module expands the receptive field through multi-angle rotation and depthwise separable convolution, enhancing global semantic capture; while the AFF module optimizes the fusion of feature maps at different scales through dynamic weights, reducing information loss and improving detail recognition. These improvements not only enhance the model's accuracy in recognizing small targets but also reduce computational complexity, making the model more suitable for resource-constrained real-time applications. The improved algorithm outperforms traditional methods in terms of accuracy, speed, and resource consumption in small target detection, especially in handling dense small target scenes with complex backgrounds, effectively reducing false positives and false negatives, demonstrating broad application prospects and practical value.

[0011] The above description is merely an overview of the technical solution of the present invention. In order to better understand the technical means of the present invention and to implement it in accordance with the contents of the specification, the preferred embodiments of the present invention are described in detail below with reference to the accompanying drawings. Attached Figure Description

[0012] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, some of the drawings in the following description are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0013] Figure 1 This is an illustration of the overall architecture and module insertion points of the algorithm shown in this invention, where Backbone is the YOLO algorithm backbone network and Neck is the improved neck network. Figure 2 This is the architecture of the Global Semantic Information Extraction (GSIE) module shown in this invention; Figure 3 This is the architecture of the adaptive feature fusion (AFF) module shown in this invention; Figure 4 This is a flowchart illustrating the operation of the method shown in this invention; Figure 5 The following are graphs showing the changes in class activation before and after using the AFF and GSIE modules: (a) original image, (b) image output after processing with the YOLOv7 network, and (c) image output after processing with the method shown in this invention. Detailed Implementation

[0014] The specific embodiments of the present invention will be described in further detail below with reference to the accompanying drawings and examples. The following examples are for illustrative purposes only and are not intended to limit the scope of the invention.

[0015] Unless otherwise specified, the technical solutions described in this invention are all conventional solutions in the field; unless otherwise specified, the reagents or materials described are all from commercial sources.

[0016] Please see Figures 1 to 5 This invention provides a general optimization method for YOLO target detection in remote sensing images. This method integrates a Global Semantic Information Extraction (GSIE) module and an Adaptive Feature Fusion (AFF) module into the traditional YOLO algorithm framework, significantly improving the detection performance of small targets in remote sensing images, especially in handling densely packed small target scenes with complex backgrounds, effectively reducing false positives and false negatives. The method includes the following steps: 1) Input the original image data to be analyzed into the YOLO algorithm backbone network, with a size of 512x512.

[0017] 2) Preprocess the image by normalizing the input image with a size of 512x512.

[0018] 3) Use depthwise convolution as the backbone network (such as YOLO: GoogLeNet, YOLOv2 (YOLO9000): DarkNet-19, YOLOv3 / YOLOX: DarkNet-53, YOLOv4 / YOLOv5 / YOLOv8: CSPDarkNet-53, YOLOv6: EfficientRep, YOLOv7: E-ELAN) to extract features from the preprocessed image.

[0019] 4) The YOLO algorithm is improved by introducing general optimization method modules, including the Global Semantic Information Extraction (GSIE) module and the Adaptive Feature Fusion (AFF) module. The overall architecture of the algorithm after the introduction and the module insertion points are as follows: Figure 1 As shown, the Backbone is the YOLO algorithm backbone network, and the Neck is the improved neck network.

[0020] The GSIE module: Through multi-angle feature rotation, it ensures that features from all perspectives are effectively included, enhancing the expressive power for extremely small targets. The architecture of the GSIE module is as follows: Figure 2 As shown, the arrows along the channel direction follow the rotation axis used for feature rotation. N represents the batch size, C represents the number of channels, H and W represent the height and width of the feature map, respectively, and SE stands for Squeeze-and-Excitation. The GSIE module is designed to expand the model's receptive field and enhance its ability to capture global semantic information. Specifically, the global semantic information extraction module selects the horizontal line passing through the center point along the channel direction in the feature map as the rotation axis, and performs a 90° rotation on each feature map. ° 180 ° and 270 ° The original feature map is rotated to generate a rotated feature map. The original feature map and the rotated feature map are then merged along the channel dimension to form a multi-view feature map with rich semantic information. In this way, the GSIE module can capture target information from multiple angles and significantly improve the ability to recognize small targets in complex backgrounds.

[0021] The AFF module replaces PANet for feature fusion, optimizes the feature propagation path, and reduces information loss, especially for detailed information about small targets. The architecture of the AFF module is as follows: Figure 2As shown, the AFF module optimizes the feature fusion process by dynamically adjusting the fusion weights of feature maps at different scales. Specifically, the adaptive feature fusion module uses 1×1 convolution to adjust the number of channels in low-resolution and medium-resolution feature maps to match the high-resolution feature maps. It then uses upsampling technology to unify the size of these feature maps and introduces three dynamic weights α, β, and δ. These dynamic weights are normalized using the softmax function to adjust the contribution rate of different feature maps. Through this weight adjustment mechanism, the AFF module can effectively reduce information loss while enhancing the expressive power of the feature maps.

[0022] 5) Apply improved feature fusion technology and use a specially designed small detector head for target detection.

[0023] 6) The results are processed using NMS to ensure that the optimal bounding box for each target is selected and plotted on the final output image. For example... Figure 5 The output image shows the correctly labeled locations of the small vehicles. (a) is the original image, (b) is the output after processing by the YOLOv7 network, and (c) is the output after processing by the method proposed in this invention. Figure 5 The comparison of the attention maps shows that (b) not only highlights the target region but also areas where the visual features of the object and the background are relatively similar. In contrast, (c), obtained with the help of AFF and GSIE, focuses more on highlighting the areas where the object and the background are most distinct. The results demonstrate that the method of this invention effectively eliminates attentional distractions in irrelevant areas and concentrates attention on the target discrimination region, thereby contributing to the accurate prediction of densely packed small targets.

[0024] In summary, this invention significantly improves the detection performance of small targets in remote sensing images by integrating the Global Semantic Information Extraction (GSIE) module and the Adaptive Feature Fusion (AFF) module into the traditional YOLO algorithm framework. The GSIE module expands the receptive field through multi-angle rotation and depthwise separable convolution, enhancing global semantic capture; while the AFF module optimizes the fusion of feature maps at different scales through dynamic weights, reducing information loss and improving detail recognition. These improvements not only enhance the model's accuracy in recognizing small targets but also reduce computational complexity, making the model more suitable for resource-constrained real-time applications. The improved algorithm outperforms traditional methods in terms of small target detection accuracy, speed, and resource consumption, especially in handling dense small target scenes with complex backgrounds, effectively reducing false positives and false negatives, demonstrating broad application prospects and practical value.

[0025] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0026] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these all fall within the protection scope of the present invention. Therefore, the protection scope of this invention patent should be determined by the appended claims.

Claims

1. A general optimization method for YOLO series target detection in remote sensing images, characterized in that, The general optimization method is implemented through an optimized YOLO algorithm that includes a global semantic information extraction module and an adaptive feature fusion module. The general optimization method includes the following steps: S1. Data Preprocessing and Feature Extraction: After extracting features from the input remote sensing image through the backbone network of the YOLO algorithm, three different scale feature maps are output: high-resolution feature map, medium-resolution feature map and low-resolution feature map. The feature maps correspond to different levels of semantic information. S2, the global semantic information extraction module processes: the global semantic information extraction module selects a horizontal line passing through the center point in the channel direction in the feature map as a rotation axis, rotates each feature map by 90 ° , 180 ° , and 270 ° , respectively, to generate a rotated feature map, and combines the original feature map and the rotated feature map along the channel dimension to form a multi-view feature map with rich semantic information; S3. Adaptive Feature Fusion Module Processing: The adaptive feature fusion module uses 1×1 convolution to adjust the number of channels in the low-resolution feature map and the medium-resolution feature map to match the high-resolution feature map. It uses upsampling technology to unify the size of the feature map and then introduces three dynamic weights α, β, and δ. The dynamic weights are normalized by the softmax function to adjust the contribution rate of different feature maps. S4. Target Detection: The feature map processed by the global semantic information extraction module and the adaptive feature fusion module is sent to the detection head for target localization and classification.

Images