A remote sensing image small target detection method, equipment, medium and product

Abstract

This application discloses a method, device, medium, and product for detecting small targets in remote sensing images. The method relates to the application field of remote sensing images. The SCW‑YOLO model for detecting small targets in remote sensing images is obtained by improving the YOLOv8 model. The improvements include: removing the large object detection head and adding a small object detection head based on the YOLOv8 model; optimizing the feature fusion network of the YOLOv8 model's neck structure using a bidirectional weighted feature pyramid network structure to fuse feature information directly generated from the backbone network; adding a coordinate attention mechanism to the second and third C2f modules in the YOLOv8 model's backbone network; and optimizing the YOLOv8 model's loss function using the Wise IOU method. This application can improve the detection accuracy of small targets such as cars, airplanes, ships, and oil tanks in remote sensing images.

Description

Technical Field

[0001] The present application relates to the field of remote sensing image applications, and in particular to a remote sensing image small target detection method, equipment, medium and product. Background Art

[0002] With the continuous advancement of remote sensing satellite technology, high-resolution remote sensing images (i.e., remote sensing images) are becoming increasingly informative and contain increasingly detailed information. In recent years, with the development of deep learning, its ability to effectively extract information from massive amounts of data has led to its widespread application in remote sensing image target detection.

[0003] Based on the target detection stage, target detection methods are categorized into one-stage and two-stage detection algorithms. Two-stage detection methods (i.e., two-stage detection algorithms) are based on deep convolutional networks (DCNNs) for candidate regions. They first generate candidate regions containing the target to be detected. Then, they use DCNNs to extract features, perform classification, and correct the features to achieve target detection. Classic algorithms include R-CNN, Fast R-CNN, and Faster R-CNN. While two-stage algorithms (i.e., two-stage detection algorithms) offer high detection accuracy, they are slow and require a large number of parameters, making them unsuitable for application deployment. One-stage detection methods (i.e., one-stage detection algorithms) are based on deep convolutional neural networks with regression and employ end-to-end target detection methods. Typical examples include SDD and the YOLO series. These methods offer high detection speed, low model computational complexity, and rapid detection.

[0004] Currently, the YOLOv8 model is one of the most advanced object detection algorithms, achieving good results in ordinary natural optical images. However, in contrast, when detecting small objects such as cars, ships, airplanes, and oil tanks in remote sensing images, the target scale accounts for a relatively small proportion of the image and is easily affected by complex environmental factors. In addition, objects in remote sensing images are often densely packed, making it difficult to separate the target from the background, increasing the difficulty of feature extraction and resulting in low detection accuracy. Therefore, it is very necessary to further improve the detection effect of deep learning technology in remote sensing images, especially to improve the detection accuracy of small objects such as cars, airplanes, ships, and oil tanks in remote sensing images. Summary of the Invention

[0005] The purpose of this application is to provide a method, equipment, medium and product for detecting small targets in remote sensing images to improve the detection accuracy of small targets such as cars, airplanes, ships and oil tanks in remote sensing images.

[0006] To achieve the above objectives, this application provides the following solutions:

[0007] In a first aspect, the present application provides a method for detecting small targets in remote sensing images, the method comprising:

[0008] Obtain a model training data set; the model training data set includes a plurality of training remote sensing images marked with the locations and categories of small targets; the small targets include small cars, large cars, airplanes, ships, and oil storage tanks;

[0009] Take the YOLOv8 model as the baseline network. On the basis of the YOLOv8 model, remove the large object detection head of the YOLOv8 model, add the small object detection head, change the connection between the second C2f module in the YOLOv8 model backbone network and the YOLOv8 model neck structure to the first C2f module in the YOLOv8 model backbone network and the YOLOv8 model neck structure, change the connection between the third C2f module in the YOLOv8 model backbone network and the YOLOv8 model neck structure to the second C2f module in the YOLOv8 model backbone network and the YOLOv8 model neck structure, remove The fifth convolutional module and the fourth C2f module in the YOLOv8 model backbone network are optimized using a bidirectional weighted feature pyramid network structure to integrate feature information directly generated from the backbone network. A coordinate attention mechanism is added to the second and third C2f modules in the YOLOv8 model backbone network. The WiseIOU method is used to optimize the loss function of the YOLOv8 model to obtain the SCW-YOLO model. The large object detection head is a 20×20 detection head, and the small object detection head is a 160×160 detection head.

[0010] Training and optimizing the SCW-YOLO model using the model training dataset to obtain an optimized SCW-YOLO model;

[0011] The remote sensing image to be detected is input into the optimized SCW-YOLO model, and the optimized SCW-YOLO model is used to obtain the position and category of small targets in the remote sensing image to be detected.

[0012] Optionally, before obtaining the model training data set, the remote sensing image small target detection method further includes:

[0013] Get the DOTA dataset;

[0014] Extracting remote sensing images containing small car targets, remote sensing images containing large car targets, remote sensing images containing aircraft targets, remote sensing images containing ship targets, and remote sensing images containing oil tank targets from the DOTA data set; the area of ​​the small targets in the extracted remote sensing images is less than 1%;

[0015] The extracted remote sensing images are cropped to a uniform size of 1024×1024 to obtain a plurality of training remote sensing images;

[0016] Each of the training remote sensing images is labeled with instances to obtain a plurality of training remote sensing images labeled with positions and categories of small targets;

[0017] A model training dataset is created using multiple training remote sensing images marked with the locations and categories of small targets.

[0018] Optionally, removing the large object detection head of the YOLOv8 model and adding a small object detection head specifically includes:

[0019] Remove the 20×20 detection head in the YOLOv8 model, add a 160×160 detection head to the YOLOv8 model, modify the connection relationship between the YOLOv8 model neck structure and the 80×80 and 40×40 detection heads in the YOLOv8 model, connect the C2f module connected to the 80×80 detection head in the YOLOv8 model neck structure to the 160×160 detection head, connect the C2f module connected to the 40×40 detection head in the YOLOv8 model neck structure to the 80×80 detection head, and connect the C2f module connected to the 20×20 detection head in the YOLOv8 model neck structure to the 40×40 detection head.

[0020] Optionally, removing the fifth convolutional module and the fourth C2f module in the YOLOv8 model backbone network specifically includes:

[0021] Remove the fifth convolution module and the fourth C2f module in the YOLOv8 model backbone network, and connect the third C2f module in the YOLOv8 model backbone network to the spatial pyramid pooling module in the YOLOv8 model backbone network.

[0022] Optionally, the bidirectional weighted feature pyramid network structure is used to optimize the feature fusion network of the neck structure of the YOLOv8 model to fuse feature information directly generated from the backbone network, specifically including:

[0023] A bidirectional weighted feature pyramid network structure is used to optimize the feature fusion network of the neck structure of the YOLOv8 model. A new connection is established using the bidirectional weighted feature pyramid network structure, so that the feature fusion network of the neck structure of the YOLOv8 model fuses feature information directly generated from the backbone network. The new connection is a connection between the second C2f module in the YOLOv8 model backbone network and the third fusion module in the YOLOv8 model neck structure. The third fusion module in the YOLOv8 model neck structure is a fusion module connected to the first convolution module in the YOLOv8 model neck structure.

[0024] Optionally, adding a coordinate attention mechanism to the second C2f module and the third C2f module in the YOLOv8 model backbone network specifically includes:

[0025] In the second C2f module in the YOLOv8 model backbone network, after the feature information is fused through residual convolution, a coordinate attention mechanism is added. At the same time, in the third C2f module in the YOLOv8 model backbone network, after the feature information is fused through residual convolution, a coordinate attention mechanism is added.

[0026] Optionally, the Wise IOU method is used to optimize the loss function of the YOLOv8 model, specifically including:

[0027] The Wise IOU loss function is used to replace the CIOU loss function of the YOLOv8 model.

[0028] In a second aspect, the present application provides a computer device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement any one of the above-described methods for detecting small targets in remote sensing images.

[0029] In a third aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any one of the above-mentioned methods for detecting small targets in remote sensing images.

[0030] In a fourth aspect, the present application provides a computer program product, comprising a computer program, which, when executed by a processor, implements any one of the above-mentioned methods for detecting small targets in remote sensing images.

[0031] According to the specific embodiments provided in this application, this application has the following technical effects:

[0032] The present application provides a method, device, medium and product for detecting small targets in remote sensing images. The SCW-YOLO model with high detection accuracy is obtained by improving the YOLOv8 model. By removing the large object detection head and adding the small object detection head on the basis of the YOLOv8 model, higher-resolution feature extraction is achieved based on the small object detection head, ensuring that local and detail features are not lost, and feature information can be effectively extracted from shallow networks to generate more discriminative features. The feature fusion network of the neck structure of the YOLOv8 model is optimized through the bidirectional weighted feature pyramid network structure, and the feature information (i.e., feature map) directly generated from the backbone network is effectively fused, and the feature information generated in the middle of the backbone network is directly fused into the feature fusion network of the neck structure of the YOLOv8 model, thereby preventing the loss of some information. The coordinate attention mechanism is added to the second C2f module and the third C2f module in the backbone network of the YOLOv8 model, thereby enhancing the local incremental feature extraction capability of the backbone network. The IOU method optimizes the loss function of the YOLOv8 model, solving the problem of poor bounding box fitting ability of the CIOU loss function in the YOLOv8 model. It can better detect occlusions and small objects, and uses the SCW-YOLO model with high detection accuracy to detect small targets in remote sensing images, improving the detection accuracy of small targets such as cars, airplanes, ships, and oil tanks in remote sensing images. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0034] Figure 1 A flowchart of a method for detecting small targets in remote sensing images provided in one embodiment of the present application is provided;

[0035] Figure 2 A flowchart of the steps provided for this application;

[0036] Figure 3 Schematic diagram of the YOLOv8 model provided for this application;

[0037] Figure 4 Schematic diagram of the SCW-YOLO model provided for this application;

[0038] Figure 5 A schematic diagram comparing the model structure improvements provided for this application;

[0039] Figure 6Schematic diagram of the C2f module structure of the model provided in this application;

[0040] Figure 7 The CA-C2f module structure diagram of the model provided for this application;

[0041] Figure 8 The coordinate attention module structure diagram provided for this application;

[0042] Figure 9 Schematic diagram of position regression based on IOU provided for this application;

[0043] Figure 10 Comparison chart of YOLOv8 and SCW-YOLO accuracy provided for this application;

[0044] Figure 11 Comparison chart of YOLOv8 and SCW-YOLO prediction results provided for this application;

[0045] Figure 12 A schematic diagram of the structure of a computer device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0046] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0047] The purpose of this application is to provide a method, equipment, medium and product for detecting small targets in remote sensing images to improve the detection accuracy of small targets such as cars, airplanes, ships and oil tanks in remote sensing images.

[0048] In order to make the above-mentioned purposes, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0049] like Figure 1 As shown, this application provides a remote sensing image small target detection method, including:

[0050] Step 101: Obtain a model training dataset; the model training dataset includes multiple training remote sensing images marked with the locations and categories of small targets; small targets include small cars, large cars, airplanes, ships, and oil tanks.

[0051] Before step 101, the following steps are also included:

[0052] Get the DOTA dataset.

[0053] Remote sensing images containing small car targets, large car targets, aircraft targets, ship targets, and oil tank targets are extracted from the DOTA dataset; the area of ​​small targets in the extracted remote sensing images is less than 1%.

[0054] Each extracted remote sensing image is cropped to a uniform size of 1024×1024 to obtain multiple training remote sensing images.

[0055] Each training remote sensing image is instance-labeled to obtain multiple training remote sensing images labeled with the location and category of small targets.

[0056] A model training dataset is created using multiple training remote sensing images marked with the locations and categories of small targets.

[0057] Step 102: Using the YOLOv8 model as the baseline network, on the basis of the YOLOv8 model, remove the large object detection head of the YOLOv8 model, add the small object detection head, modify the connection between the second C2f module in the YOLOv8 model backbone network and the YOLOv8 model neck structure to the first C2f module in the YOLOv8 model backbone network and the YOLOv8 model neck structure, modify the connection between the third C2f module in the YOLOv8 model backbone network and the YOLOv8 model neck structure to the second C2f module in the YOLOv8 model backbone network and the YOLOv8 model neck structure. The fifth convolutional module and the fourth C2f module in the YOLOv8 model backbone network are connected and removed. The feature fusion network of the YOLOv8 model neck structure is optimized using a bidirectional weighted feature pyramid network structure to fuse the feature information directly generated from the backbone network. The coordinate attention mechanism is added to the second and third C2f modules in the YOLOv8 model backbone network. The WiseIOU method is used to optimize the loss function of the YOLOv8 model to obtain the SCW-YOLO model. Among them, the large object detection head is a detection head of size 20×20; the small object detection head is a detection head of size 160×160.

[0058] In step 102, the large object detection head of the YOLOv8 model is removed, and a small object detection head is added, specifically including: removing the 20×20 detection head in the YOLOv8 model, adding a 160×160 detection head in the YOLOv8 model, modifying the connection relationship between the YOLOv8 model neck structure and the 80×80 detection head and the 40×40 detection head in the YOLOv8 model, connecting the C2f module connected to the 80×80 detection head in the YOLOv8 model neck structure to the 160×160 detection head, connecting the C2f module connected to the 40×40 detection head in the YOLOv8 model neck structure to the 80×80 detection head, and connecting the C2f module connected to the 20×20 detection head in the YOLOv8 model neck structure to the 40×40 detection head.

[0059] In step 102, the fifth convolutional module and the fourth C2f module in the YOLOv8 model backbone network are removed, specifically including: removing the fifth convolutional module and the fourth C2f module in the YOLOv8 model backbone network, and connecting the third C2f module in the YOLOv8 model backbone network to the spatial pyramid pooling module in the YOLOv8 model backbone network.

[0060] In step 102, a bidirectional weighted feature pyramid network structure is used to optimize the feature fusion network of the neck structure of the YOLOv8 model to fuse feature information directly generated from the backbone network. Specifically, the steps include: optimizing the feature fusion network of the neck structure of the YOLOv8 model using the bidirectional weighted feature pyramid network structure, and using a new connection established by the bidirectional weighted feature pyramid network structure to enable the feature fusion network of the neck structure of the YOLOv8 model to fuse the feature information directly generated from the backbone network; the new connection is a connection between the second C2f module in the backbone network of the YOLOv8 model and the third fusion module in the neck structure of the YOLOv8 model; and the third fusion module in the neck structure of the YOLOv8 model is a fusion module connected to the first convolution module in the neck structure of the YOLOv8 model.

[0061] In step 102, a coordinate attention mechanism is added to the second C2f module and the third C2f module in the YOLOv8 model backbone network, specifically including: adding a coordinate attention mechanism after fusing feature information through residual convolution in the second C2f module in the YOLOv8 model backbone network; and adding a coordinate attention mechanism after fusing feature information through residual convolution in the third C2f module in the YOLOv8 model backbone network.

[0062] In step 102, the Wise IOU method is used to optimize the loss function of the YOLOv8 model, specifically including: replacing the CIOU loss function of the YOLOv8 model with the Wise IOU loss function.

[0063] Step 103: Use the model training data set to train and optimize the SCW-YOLO model to obtain an optimized SCW-YOLO model.

[0064] Step 104: Input the remote sensing image to be detected into the optimized SCW-YOLO model, and use the optimized SCW-YOLO model to obtain the position and category of the small target in the remote sensing image to be detected.

[0065] The technical solution of this application is described below with a specific embodiment:

[0066] The remote sensing image small target detection method provided by the present application is a model method for target detection in remote sensing images. Based on the YOLOv8 model, the present application proposes a SCW-YOLO model for detecting small targets such as cars (including small cars and large cars), airplanes, ships and oil tanks in remote sensing images. The SCW-YOLO model provided by the present application is a remote sensing image small target detection model improved based on YOLOv8. The model is not only suitable for marine ship detection, but also for target detection of cars, airplanes and oil tanks on land. It only needs to input a high-resolution remote sensing image into the model, and the model can mark the specific locations of small targets such as cars, airplanes, ships and oil tanks in the image.

[0067] like Figure 2 As shown, the remote sensing image small target detection method provided by this application includes:

[0068] 1. Preparation of dataset.

[0069] The DOTA remote sensing image dataset (DOTA dataset) was obtained. Five typical targets were selected from this dataset: small cars, large cars, airplanes, ships, and storage tanks (i.e., oil tanks). The area of ​​these five typical targets was less than 1%. By extracting examples of these five typical targets from the DOTA dataset, a total of 5,243 remote sensing images were obtained. 4,235 of these images (i.e., remote sensing images) were used for training and 1,008 images for testing. The remote sensing image data was annotated with instances to create the model training dataset.

[0070] This application obtains training remote sensing images from the DOTA dataset. Since the pixel size of the original images of the DOTA dataset is in the range of 800*800 to 4000*4000, it is not suitable for direct training and prediction. Therefore, this application uniformly crops them to a size of 1024*1024.

[0071] 2. Using the YOLOv8 model as the baseline network, a SCW-YOLO model is proposed for small target detection in remote sensing images.

[0072] This application is based on the YOLOv8 model and proposes a SCW-YOLO model for small object detection. Based on the existing YOLOv8, this application removes the large object detection head, adds a higher-resolution feature extractor, ensures that local and detailed features are not lost, and optimizes the network structure. This application introduces an efficient feature pyramid network to guide subsequent network information to perform feature fusion through a top-down path, reducing redundant calculations. In addition, in order to make the backbone network focus on the target and generate more feature information, this application integrates a coordinate attention mechanism, namely the CA (Coordinate Attention) attention mechanism, into the bottleneck structure of the shallow backbone network, focusing on generating the location information of the feature map, better capturing the spatial and channel information in the image, so as to accurately extract the target features in the image. In response to the problem of poor bounding box fitting ability of the CIOU loss function in the YOLOv8 model, in order to improve the bounding box fitting ability, this application proposes a dynamic non-monotonic Wise-IOU Loss as the bounding box regression loss function. By introducing the ratio of the intersection and union of the predicted box and the measured box, it solves the problem of annotation quality imbalance and enhances the convergence and robustness of the model.

[0073] Compared with other mainstream algorithms, YOLOv8 has higher accuracy in detecting small targets, but there are still some problems in detecting small targets in remote sensing images, such as false positives and missed detections. To solve this problem, this application proposes an improved remote sensing small target detection algorithm SCW-YOLO based on YOLOv8. Figure 3 Compared with the original YOLOv8 network structure shown, Figure 4 The improvements made to the SCW-YOLO network structure shown in YOLOv8 include:

[0074] First, this application designs an efficient feature pyramid structure: it removes the feature layer that generates a feature map of size 20×20, and adds a feature layer that generates a feature map of size 160×160, which not only reduces the number of model parameters, but also can effectively extract feature information from the shallow network and generate more discriminative features; inspired by the bidirectional weighted feature pyramid network (BiFPN) structure, its structure is used to optimize the feature fusion network of the neck structure (i.e., the neck network), effectively fusing the feature information from the backbone (i.e., the backbone network). Secondly, after fusing the feature information through residual convolution (residual convolution already exists in the C2f module) in the P4 and P6 parts of the backbone network, a coordinate attention mechanism is added to enhance the local incremental feature extraction capability of the backbone. Finally, this application uses the Wise IOU (WIOU) method to optimize the loss function. This method combines the attention mechanism and the dynamic non-monotonic mechanism to better detect occlusions and small objects.

[0075] In convolutional neural networks, deep features have low resolution, fewer fine-grained features, and more semantic information, while shallow features have high resolution, rich fine-grained features, and less semantic information. In the YOLOv8 model, the backbone network extracts features from the original image by downsampling by 8, 16, and 32 times. Although continuously shrinking the feature map allows the network to obtain a larger receptive field and richer semantic information, for target instances in remote sensing images, since they occupy fewer pixels in the image, the continuous downsampling of the image's location information results in a significant loss of features, making it impossible for the model to locate the target. Furthermore, the semantic loss caused by continuous downsampling has a negative effect on feature learning for detecting targets.

[0076] The YOLOv8 network has a large receptive field for mid-level and deep features, and is mainly focused on representing abstract semantic information. In the feature extraction of small targets, since small targets occupy a relatively small pixel area in the image, reasonably setting the receptive field size and adopting a multi-scale receptive field strategy can capture more detailed information in a limited area. In this application, the input image size is fixed to 640×640, and the prediction head of the model generates three feature maps corresponding to sizes of 80×80, 40×40, and 20×20 respectively. The smaller the resolution, the larger the receptive field accepted by the neurons, the richer the semantic level, and the corresponding loss of local features and detailed features. However, the size of the objects detected in remote sensing images is small, and the detection target may be completely lost in the feature map of size 20×20, affecting subsequent feature fusion. Since the previously lost information cannot be fully restored during the upsampling process, the model loses key semantic information during the detection process, thereby reducing the detection accuracy. This application will remove the 20×20 feature extractor (including the C2f module and the fusion module connected to the C2f module) and the corresponding detection head, which not only simplifies the structure of the model but also solves the problem of information loss. Among them, after the 20×20 detection head is removed, its corresponding 20×20 feature extractor is no longer needed, that is, the connection between the 20×20 detection head and the C2f module connected to the detection head is lost, and the C2f module establishes a new connection with the 40×40 detection head, thereby obtaining a 40×40 feature extractor. Therefore, it can be said that the 20×20 feature extractor has been removed.

[0077] After the above optimization operations, only the 80×80 and 40×40 feature extractors in the model extract target features, and are unable to extract more comprehensive location information and semantic information. In order to expand the perception range of the network and enhance the detection capability of small objects, this application extends a four-fold down-sampling feature extractor in the P2 part of the backbone network, that is, adding a feature extractor of size 160×160 (after adding a detection head of size 160×160, the C2f module connected to the detection head of size 80×80 is connected to the detection head of size 160×160, thereby obtaining a feature extractor of size 160×160, so it can be said that a feature extractor of size 160×160 is added). This improvement helps to capture a wider range of contextual information, thereby improving the model's detection performance for small objects. The new feature extractor focuses on low-level feature information and enhances the feature extraction capability for small targets. An upsampling module is added to the feature extractor that generates an 80×80 feature map (the C2f module connected to a 40×40 detection head is connected to an 80×80 detection head, thereby obtaining an 80×80 feature extractor. The upsampling module connected to the feature extractor changes the number of input channels due to the change in position, so it can be called an added upsampling module) to perform feature fusion with the new feature extractor, deliver richer semantic information and position information, and thus improve the overall detection performance.

[0078] YOLOv8's neck structure uses a path aggregation (PANet) network structure to upsample the features obtained from the backbone network and then perform downsampling and fusion. Each feature is upsampled and then downsampled and fused again, so that each feature contains more information. To ensure that the model has both comprehensive position information and complete semantic information, this application uses the BiFPN structure to improve the feature fusion network of the neck structure.

[0079] BiFPN is an improvement based on the PANet network structure, allowing features to be fused in both top-down and bottom-up directions, thereby more effectively combining features of different scales. BiFPN simplifies the bidirectional network by optimizing the structure and removing nodes with only one input edge. This node contributes very little to the feature network that fuses different features; it adds extra edges between input and output nodes at the same level, and regards each bidirectional path as a feature network layer, fusing more features without increasing too much cost; BiFPN optimizes the feature fusion process by adding weights to each input feature, so that the network can pay more attention to features with more information, thereby achieving differentiated fusion of information at different scales and improving model performance. However, it will also increase the computational complexity and time consumption of the model, and increase the difficulty of training the model. Therefore, this application removes the weight method and retains the network structure of BiFPN, that is, this application removes the calculation weight operation of the BiFPN structure and retains the BiFPN structure part introduced above. As Figure 5 Shown is a comparison of the original neck structure and the improved neck structure. Figure 5 Part (a) shows the original bottleneck structure. Figure 5 Part (b) shows the improved neck structure. Figure 5 The connection method of BiFPN is explained.

[0080] In the YOLOv8 network structure, the P2 part is downsampled 4 times, which will introduce unnecessary noise that affects the backbone network's extraction of feature information. For the target in this application, it occupies fewer pixels in the image, which makes the model unable to accurately notice the target. In order to further improve the detection accuracy of the model under complex backgrounds and enable the model to accurately locate the position of the detected target, the attention mechanism is integrated into the bottleneck in the C2f module of the P2 and P4 parts, so that the module can capture more critical feature information to enhance local features. However, in order to avoid generating a larger amount of computation, this application chooses to introduce a lightweight coordinate attention mechanism. This application refers to the enhanced module as the CA-C2f module and the enhanced bottleneck structure as the CA-Bottleneck structure. The CA-C2f module that combines the coordinate attention mechanism with the C2f module in the YOLOv8 backbone network brings huge advantages to small target detection tasks. This integrated module improves the accuracy of model positioning and target recognition by capturing cross-channel, spatial and position information. Among them, the C2f module structure is as follows Figure 6 As shown, the CA-C2f module structure is as follows Figure 7 shown.

[0081] Coordinate attention is an efficient attention mechanism that divides the attention process into two one-dimensional feature encoding processes, namely coordinate information embedding and coordinate attention generation. Figure 8 As shown in Figure 2. By inputting position information into channel attention, the network can obtain information from a larger area and avoid introducing large overhead. Coordinate attention encodes channel relationships and long-term dependencies through precise position information, which can convert any intermediate tensor into As input, and output an output of the same size Where X represents the input tensor, x C Represents the Cth tensor in the input tensor, Y represents the output tensor, y C represents the Cth tensor in the output tensor, Represents a three-dimensional tensor, C represents the feature dimension, H represents the feature height, and W represents the feature width.

[0082] First, the global pooling method is used to globally encode spatial information in channel attention encoding. Since it compresses global spatial information into the channel descriptor, although the position information of the original data is lost to a certain extent, it also enables the attention module to capture long-range spatial interactions with position information. The global pooling decomposition formula is as follows:

[0083]

[0084] Z c Represents the output result of the cth channel, x c Represents the input data of the cth channel, H represents the feature height, and W represents the feature width. Specifically, a pooling convolution kernel of size (H, 1) or (1, W) is used to encode each channel in the horizontal and vertical directions respectively. This application will obtain the output of the cth channel with a height of h. and the output of the cth channel with width w The formula is as follows:

[0085]

[0086] x c (h,i) represents the input data of the cth channel with height h, x c (j,w) represents the input data with the cth channel width w. The two transformations described above generate features along the vertical and horizontal directions, respectively, resulting in a pair of feature maps with orientation information. This transformation enables the attention module to capture long-term dependencies along one spatial direction and preserve precise position information along the other, helping the network more accurately locate the detection target.

[0087] In order to better utilize the representation with global receptive field and precise position information generated by the above modules, we propose the operation of transforming the coordinate attention generation. First, the two feature maps generated by the previous modules are cascaded using a shared 1×1 convolution transformation function F1 to generate It is an intermediate feature map with horizontal and vertical information, r represents the downsampling ratio, and σ represents the Sigmoid function. It is expressed as follows:

[0088] f=σ(F1([Z h ,Z w ]));

[0089] Among them, Z h is the output of all channels with height h, Z w is the output of all channels with a width of w. Next, f is split into two tensors along the spatial dimension: the tensor along the height direction and the tensor along the width direction Then use two 1×1 convolutions to transform F along the height direction h and the widthwise transformation F w , so that the feature map f h and f w Transform to the same number of channels as the input x, the formula is as follows:

[0090] g h =σ(F h (f h ));

[0091] g w =σ(F w (f w ));

[0092] σ represents the sigmoid function. Finally, along the width direction g w and along the height direction g h As the attention weight, the final output y of the coordinate attention module is obtained c The expression for (i, j) is:

[0093]

[0094] in, represents the attention weight of the c-th channel along the height direction, represents the attention weight of the c-th channel along the width direction.

[0095] In computer vision, the intersection-over-union (IOU) is often used to measure the overlap between the measured box and the model's predicted box. Figure 9 As shown in the figure, this application represents the predicted position, width and height of the bounding box as x, y, w, and h respectively, and the measured position, width and height of the bounding box as x. gt ,y gt , w gt , h gt .w i , hi Represents the width and height of the overlap between bounding boxes, from which the loss function L is derived IoU is defined as:

[0096] L IoU = 1-IoU;

[0097]

[0098] Among them, L IoU Represents the IOU loss function. The larger the IOU value (<1), the greater the overlap between the model's predicted box and the measured box. However, blindly pursuing the low-quality target regression of strengthening the bounding box will jeopardize the performance improvement between models. Unlike the previous YOLO series, the Loss calculation of YOLOv8 includes two parts: classification and regression branches. The classification branch uses the binary cross entropy loss function (BCE Loss); the regression branch uses the complete intersection-over-union loss function (CIOU Loss), and it also needs to be bound to the integral form representation proposed in Distribution Focal Loss. CIOU Loss takes into account the distance, size, overlap and aspect ratio between the centers of the bounding boxes to represent the change in the aspect ratio between the predicted box and the measured box. CIOU loss function L CIoU The calculation formula is as follows:

[0099]

[0100]

[0101] In the formula, α represents the weight function, and v represents the consistency of the aspect ratio. In addition to CIOU Loss, DIOU Loss, SIOULoss, EIOU Loss, etc., the aspect ratio is also defined as a relative value. When the predicted box and the measured box have the same aspect ratio but different widths and heights, the value of the loss function of the above bounding box regression is the same, which limits the convergence of the model. Therefore, this application proposes WIOU as the bounding box regression loss function to enhance the pertinence of the model and accelerate network convergence. This application constructs distance attention based on the residual metric and obtains the loss function L′ with a two-layer attention mechanism WIoU . Loss gradient R WIoU The calculation formula is as follows:

[0102] L' WIoU =R WIoU L IoU ;

[0103]

[0104] Among them, R WIoU ∈[0,1],LIoU ∈[0,1]. In order to ensure R WIoU Convergence of W and H is achieved by separating them from the computational graph. Inspired by Focal E-Loss, a monotonic focusing mechanism is designed. This application defines a non-monotonic focusing coefficient, outlier, to describe the quality of the anchor box. A smaller outlier means a higher quality anchor box, and a small gradient gain is assigned to it, so that the bounding box is focused on an anchor box of average quality. L WIoU The calculation formula is as follows:

[0105] L WIoU =γL' WIoU ;

[0106]

[0107] Among them, L WIoU is the WIOU loss function L′ WIoU Based on the improved, β is the intermediate parameter, δ represents the abnormality coefficient, γ represents the non-monotonic coefficient, represents the monotonic focus coefficient, represents the sliding average. It is dynamic, and the quality division criteria of the anchor boxes are also dynamic, which enables WIOU to make the most appropriate gradient gain allocation strategy at every moment.

[0108] This application builds a training environment and adjusts model parameters, including:

[0109] This application builds a deep learning environment in PyTorch. The testing environment includes an NVIDIA RTX A5000 with 24GB of graphics memory, CUDA version 11.2, cudnn version 8.0.5, and Python 3.7 as the compiler. This application improves upon the YOLOv8 algorithm. To compare the YOLOv8 algorithm with the SCW-YOLO algorithm and ensure its reliability, both training and testing were conducted in the same training environment. Furthermore, the following hyperparameters were set during training: initial learning rate 0.01, momentum 0.937, weight_decay 0.0005, training epochs 200, and batch size 6. The model automatically selects the optimal optimizer during training. Learning rate decay is used to gradually reduce the learning rate during training. The initial learning rate controls the speed at which model parameters are updated, and the final learning rate is obtained by multiplying the initial learning rate by a coefficient. This setting ensures smooth convergence of the model to the optimal solution. After training, the optimal model weights were used for testing and comparison.

[0110] 3. Performance evaluation of the SCW-YOLO model.

[0111] To evaluate the performance of the SCW-YOLO model, we use precision (P), recall (R), average precision (mAP@0.5, mAP@0.5:0.95), detection speed (speed), F1 score, and model parameter count (Params) as evaluation indicators. Accuracy describes the ratio of predicted positive examples to all positive examples, and is calculated as follows:

[0112]

[0113] In the formula, P represents the precision, TP represents the number of correctly predicted positive samples, and FP represents the number of samples in the sample that are predicted as positive. Recall describes the ratio of predicted positive samples to actual positive samples, and is calculated as follows:

[0114]

[0115] In the formula, R represents the recall rate, and FN represents the number of samples in the sample that predict positive examples as negative examples. Subsequently, this application introduces a parameter to integrate the above two parameters, namely mAP, which is defined as the average AP of all objects and is applicable to multi-label image classification and detection. The calculation formula is as follows:

[0116]

[0117] Where N represents the number of classification categories. In addition, this application uses the F1 score to represent the accuracy of the precision and recall rate of the binary classification model. The closer the score is to 1, the more accurate the model is. The calculation formula is as follows:

[0118]

[0119] After testing the improved algorithm in this application on the DOTA small target dataset, the following results were obtained:

[0120] Compared to the original YOLOv8 model, the SCW-YOLO model achieves an improved mAP@0.5 of 84.9%, reduces parameters by 53%, and achieves a detection speed of 37.6ms. The SCW-YOLO model enhances the network's feature extraction capabilities in small target areas, separating useful target information from a large amount of feature data. It also exhibits strong anti-interference capabilities and enables accurate object recognition.

[0121] The improved remote sensing small target detection model based on YOLOv8 provided in this application not only reduces the number of model parameters but also improves the detection accuracy of the model by optimizing the model structure and designing an efficient feature extraction network, making it more conducive to deployment in terminal devices. The SCW-YOLO remote sensing image target detection model proposed in this application can be applied to other remote sensing image detection tasks. Figure 10 As shown in the figure, compared with the baseline model, the network model proposed in this application outperforms the original model in all aspects, and its performance improves rapidly throughout the training process and gradually stabilizes. Figure 10 The blue curve in the middle represents the detection accuracy of the YOLOv8 model. Figure 10 The orange curve in the middle represents the detection accuracy of the SCW-YOLO model. Figure 11 It can be seen that compared with the baseline detection model, the target detection model proposed in this application has stronger anti-interference ability in remote sensing target detection and can better adapt to complex background environments. Figure 11 Part (a) shows the YOLOv8 model detection results. Figure 11 Part (b) shows the detection results of the SCW-YOLO model.

[0122] Compared with the original YOLOv8 model, this application has the following advantages:

[0123] First, since the targets in remote sensing images occupy a very small area of ​​the entire image, the large object detection head of the original YOLOv8 model cannot detect small targets, which will only increase the number of parameters and computational complexity of the model. Moreover, as the model is downsampled, the semantic information becomes less and less, resulting in the inability to subsequently fuse more information, thereby reducing the detection accuracy of the model. Therefore, this application introduces an efficient feature pyramid network based on the existing YOLOv8: removing the large object detection head and adding a higher resolution feature extractor. This network structure is not only used to guide subsequent network information to perform feature fusion through a top-down path, but also to ensure that local and detailed features are not lost. In addition, in the YOLOv8 model, the neck structure adopts the path aggregation network structure (PANet), that is, a bottom-up channel and a top-down channel are established for feature fusion, which is conducive to combining high-level semantic information with low-level semantic information, thereby improving detection accuracy. In order to reduce the computational complexity of the neck structure, this application is inspired by the bidirectional feature pyramid structure (BiFPN) and designs an efficient feature pyramid structure: nodes with only one input edge are deleted, and an additional edge is added between the original input node and the output node. This structure can fuse more feature information without increasing the cost. This application optimizes the model structure, removes redundant structures, improves the efficiency of the network in feature fusion, adds a new detection layer, adopts an efficient feature fusion network to fuse features, enhances the network's ability to learn features from different scales, and improves the model's training efficiency and detection performance.

[0124] Secondly, this application uses the CA-C2f module in the backbone network to embed coordinate attention into the bottleneck structure. This module enables the network to pay more attention to small target features, improves the model's feature extraction ability for small targets, and improves the overall detection performance of the model.

[0125] Finally, this application proposes a new position regression loss function WIOU to accelerate the convergence of the network.

[0126] In an exemplary embodiment, a computer device is provided. The computer device may be a server or a terminal. The internal structure diagram thereof may be as follows: Figure 12 As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, abbreviated as I / O) and a communication interface. The processor, memory and input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and computer program in the non-volatile storage medium. The database of the computer device is used to store remote sensing image small target detection data. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a remote sensing image small target detection method is implemented.

[0127] Those skilled in the art will understand that Figure 12 The structure shown in the figure is merely a block diagram of a portion of the structure related to the solution of the present application and does not constitute a limitation on the computer device to which the solution of the present application is applied. A specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement. In an exemplary embodiment, a computer device is provided, including a memory and a processor. The memory stores a computer program, and the processor implements the steps of the above-mentioned method embodiments when executing the computer program.

[0128] In an exemplary embodiment, a computer-readable storage medium is provided, storing a computer program. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0129] In an exemplary embodiment, a computer program product is provided, including a computer program. When the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0130] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0131] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, database or other media used in the embodiments provided in this application may include at least one of non-volatile and volatile memory. Non-volatile memory may include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory may include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM may be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0132] The databases involved in the various embodiments provided herein may include at least one of a relational database and a non-relational database. Non-relational databases may include, but are not limited to, distributed databases based on blockchains. The processors involved in the various embodiments provided herein may include, but are not limited to, general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic units, data processing logic units based on quantum computing, and the like.

[0133] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0134] This document uses specific examples to illustrate the principles and implementation methods of this application. The description of the above examples is only intended to help understand the method and core concept of this application. At the same time, for those skilled in the art, based on the concept of this application, there may be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting this application.

Claims

1. A method for detecting small targets in remote sensing images, characterized in that: The remote sensing image small target detection method comprises: Obtain a model training data set; the model training data set includes a plurality of training remote sensing images marked with the locations and categories of small targets; the small targets include small cars, large cars, airplanes, ships, and oil storage tanks; Taking the YOLOv8 model as the baseline network, on the basis of the YOLOv8 model, the large object detection head of the YOLOv8 model is removed, and the small object detection head is added. The connection between the second C2f module in the YOLOv8 model backbone network and the YOLOv8 model neck structure is modified to be the connection between the first C2f module in the YOLOv8 model backbone network and the YOLOv8 model neck structure. The connection between the third C2f module in the YOLOv8 model backbone network and the YOLOv8 model neck structure is modified to be the connection between the second C2f module in the YOLOv8 model backbone network and the YOLOv8 model neck structure. The fifth convolution module and the fourth C2f module in the YOLOv8 model backbone network are removed. The bidirectional weighted feature pyramid network structure is used to optimize the feature fusion network of the YOLOv8 model neck structure to fuse the feature information directly generated from the backbone network. The coordinate attention mechanism is added to the second C2f module and the third C2f module in the YOLOv8 model backbone network. The Wise The IOU method optimizes the loss function of the YOLOv8 model to obtain the SCW-YOLO model; wherein the large object detection head is a detection head with a size of 20×20; the small object detection head is a detection head with a size of 160×160; Training and optimizing the SCW-YOLO model using the model training dataset to obtain an optimized SCW-YOLO model; The remote sensing image to be detected is input into the optimized SCW-YOLO model, and the optimized SCW-YOLO model is used to obtain the position and category of small targets in the remote sensing image to be detected.

2. The remote sensing image small target detection method according to claim 1, characterized in that: Before obtaining the model training data set, the remote sensing image small target detection method further includes: Get the DOTA dataset; Extracting remote sensing images containing small car targets, remote sensing images containing large car targets, remote sensing images containing aircraft targets, remote sensing images containing ship targets, and remote sensing images containing oil tank targets from the DOTA data set; the area of ​​the small targets in the extracted remote sensing images is less than 1%; The extracted remote sensing images are cropped to a uniform size of 1024×1024 to obtain a plurality of training remote sensing images; Each of the training remote sensing images is instance-labeled to obtain a plurality of training remote sensing images labeled with the positions and categories of small targets; A model training dataset is created using multiple training remote sensing images marked with the locations and categories of small targets.

3. The remote sensing image small target detection method according to claim 1, characterized in that: The large object detection head of the YOLOv8 model is removed and the small object detection head is added, specifically including: Remove the 20×20 detection head in the YOLOv8 model, add a 160×160 detection head to the YOLOv8 model, modify the connection relationship between the YOLOv8 model neck structure and the 80×80 and 40×40 detection heads in the YOLOv8 model, connect the C2f module connected to the 80×80 detection head in the YOLOv8 model neck structure to the 160×160 detection head, connect the C2f module connected to the 40×40 detection head in the YOLOv8 model neck structure to the 80×80 detection head, and connect the C2f module connected to the 20×20 detection head in the YOLOv8 model neck structure to the 40×40 detection head.

4. The remote sensing image small target detection method according to claim 1, characterized in that: The removal of the fifth convolutional module and the fourth C2f module in the YOLOv8 model backbone network specifically includes: Remove the fifth convolution module and the fourth C2f module in the YOLOv8 model backbone network, and connect the third C2f module in the YOLOv8 model backbone network to the spatial pyramid pooling module in the YOLOv8 model backbone network.

5. The remote sensing image small target detection method according to claim 1, characterized in that: The bidirectional weighted feature pyramid network structure is used to optimize the feature fusion network of the YOLOv8 model neck structure to fuse the feature information directly generated from the backbone network, specifically including: A bidirectional weighted feature pyramid network structure is used to optimize the feature fusion network of the neck structure of the YOLOv8 model. A new connection is established using the bidirectional weighted feature pyramid network structure, so that the feature fusion network of the neck structure of the YOLOv8 model fuses feature information directly generated from the backbone network. The new connection is a connection between the second C2f module in the YOLOv8 model backbone network and the third fusion module in the YOLOv8 model neck structure. The third fusion module in the YOLOv8 model neck structure is a fusion module connected to the first convolution module in the YOLOv8 model neck structure.

6. The remote sensing image small target detection method according to claim 1, characterized in that: The coordinate attention mechanism is added to the second and third C2f modules in the YOLOv8 model backbone network, specifically including: In the second C2f module in the YOLOv8 model backbone network, after the feature information is fused through residual convolution, a coordinate attention mechanism is added. At the same time, in the third C2f module in the YOLOv8 model backbone network, after the feature information is fused through residual convolution, a coordinate attention mechanism is added.

7. The remote sensing image small target detection method according to claim 1, characterized in that: The Wise IOU method is used to optimize the loss function of the YOLOv8 model, specifically including: The Wise IOU loss function is used to replace the CIOU loss function of the YOLOv8 model.

8. A computer device comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the remote sensing image small target detection method according to any one of claims 1 to 7.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the remote sensing image small target detection method according to any one of claims 1 to 7 is implemented.

10. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the remote sensing image small target detection method according to any one of claims 1 to 7 is implemented.

Images