High-resolution optical remote sensing image target detection method and device and medium

Abstract

The invention relates to a high-resolution optical remote sensing image target detection method and device and a medium. The method comprises the steps that construction is carried out on the basis of a YOLOv8s model, an EBiFPN structure is designed to replace a neck network of the YOLOv8s model, a C2f-EB module constructed by an EMA attention mechanism is used to replace a C2f module of the neck network, and a designed SWIoU frame regression loss function is adopted to carry out training; acquiring a remote sensing image, performing annotation and image preprocessing, and constructing a data set; configuring a deep learning environment, and setting parameters of the improved YOLOv8s model; and inputting the high-resolution optical remote sensing image data set into the improved model for training and testing, and identifying a target in the remote sensing image by using the trained model. Compared with the prior art, the method has the advantages of high detection precision, low model parameter quantity, low calculation quantity and the like.

Description

Technical Field

[0001] The invention relates to the technical field of remote sensing image target detection, and in particular to a high-resolution optical remote sensing image target detection method. Background Art

[0002] Optical remote sensing images contain a wealth of information and are widely used in aerial reconnaissance, urban construction, maritime rescue, geological disaster monitoring, and other fields. Although deep learning has a significant effect on target detection, optical remote sensing images are different from natural images. They have complex background information, different target sizes, and many small targets. These characteristics bring many difficulties to target detection.

[0003] At present, deep learning has made great progress in the fields of image segmentation and target detection. Many researchers have applied deep learning methods to remote sensing image recognition tasks. Compared with traditional detection methods, detection methods based on deep learning have the advantages of high detection efficiency, strong generalization ability, and no need for manually designed feature extractors, which can greatly improve detection efficiency. Detection methods based on deep learning can be divided into two-stage detection algorithms and one-stage detection algorithms. Although the recognition accuracy of the one-stage algorithm is slightly lower than that of the two-stage algorithm, since it does not need to generate a large number of candidate boxes and can directly predict the location and category of the target, the detection speed is significantly improved, which is particularly suitable for real-time target recognition and detection tasks in high-resolution optical remote sensing images.

[0004] In the field of one-stage model recognition of high-resolution optical remote sensing images, some literature uses ResNet-50 as the backbone network for target detection tasks to extract richer feature information, and then proposes a feature extraction network that integrates coordinate attention and strip pooling modules to enhance the network's ability to extract strip-shaped objects, but the number of parameters and the amount of calculation are very large. Other literature introduces dilated convolution to design a receptive field expansion module to improve the ability to extract contextual information, and then introduces a feature fusion structure for the complex background of optical remote sensing images, and finally adds angle prediction to improve the loss function to enhance the detection ability of rotating objects. There are also literatures that introduce coordinate attention mechanisms to improve the target positioning ability in view of the complex background of remote sensing images, and propose a context enhancement module to obtain rich context information. Finally, adaptive spatial feature fusion is introduced to improve the detection accuracy of small targets, but the channel dimensionality reduction of the coordinate attention mechanism may cause information loss. The above methods have achieved varying degrees of success in improving the accuracy of target detection in optical remote sensing images, and are of great significance to target detection in optical remote sensing images. However, these methods still face the problems of missed detection and false detection caused by insufficient feature information of small targets and complex backgrounds, and it is difficult to achieve a balance between accuracy and the number of parameters. Summary of the invention

[0005] The purpose of the present invention is to overcome the defects of the above-mentioned prior art and provide a high-resolution optical remote sensing image target detection method, device and medium based on an improved YOLOv8s model.

[0006] The purpose of the present invention can be achieved by the following technical solutions:

[0007] As a first aspect of the present invention, a high-resolution optical remote sensing image target detection method is provided, the method steps comprising: acquiring high-resolution optical remote sensing image data, and detecting targets in the high-resolution optical remote sensing image using a trained improved YOLOv8s model;

[0008] The improved YOLOv8s model is based on the YOLOv8s model, designs an EBiFPN structure to replace the neck network of the YOLOv8s model; and replaces the C2f module of the neck network with a C2f-EB module constructed using an EMA attention mechanism.

[0009] As a preferred technical solution, the EBiFPN structure is optimized and improved on the basis of BiFPN, specifically including: in terms of feature connection, the Concat splicing method is adopted; convolution is added to adjust the number of feature channels, and the shallow features including P2 are input into the detection head.

[0010] As a preferred technical solution, the C2f-EB module adjusts the bottleneck structure inside the C2f module, introduces the EMA efficient multi-head attention mechanism after the second convolution of the C2f module, and constructs the EMABottleneck module to replace the bottleneck structure in the C2f module.

[0011] As a preferred technical solution, the EMA attention mechanism is as follows:

[0012] Divide the input channels into multiple groups, each group is responsible for learning different semantic information;

[0013] Vertical and horizontal average pooling branches: The two parallel branches perform average pooling in the vertical and horizontal directions respectively, then concatenate the resulting feature maps and share a convolution kernel. The two channel attention maps in each group are aggregated through matrix multiplication to achieve cross-dimensional feature interaction;

[0014] Local cross-channel interaction branch: uses convolution kernels to capture cross-channel interaction information within a local range;

[0015] Two sets of feature maps are obtained through global average pooling, and feature aggregation is performed.

[0016] As a preferred technical solution, the improved YOLOv8s model constructs a data set based on annotated and image preprocessed high-resolution optical images, and uses the SWIoU bounding box regression loss function for training and testing.

[0017] As a preferred technical solution, the SWIoUYOLOv8s model loss function combines the non-monotonic focusing coefficient of WIoU and SIoU, and the expression is as follows:

[0018]

[0019]

[0020] Among them, α, δ are hyperparameters, β is the outlier degree; L SIoU is the bounding box regression loss.

[0021] As a preferred technical solution, the frame regression loss function SWIoU includes angle loss Λ, distance loss Δ, shape loss Ω and intersection-over-union loss IoU:

[0022]

[0023]

[0024]

[0025]

[0026]

[0027]

[0028] Among them, ω represents the angle between the line connecting the center point of the predicted box and the real box and the horizontal direction; b(b cx ,b cy )and Represents the center point coordinates of the predicted box and the real box, C w With C h Respectively represent the width and height of the minimum bounding box; w, h and w gt ,h gt Represent the width and height of the real box and the predicted box respectively.

[0029] As a preferred technical solution, when testing the improved YOLOv8s model, the average precision mean, the number of model parameters and the number of billion floating-point operations are used as evaluation indicators.

[0030] As a second aspect of the present invention, a high-resolution optical remote sensing image target detection device is provided, comprising a memory, a processor, and a program stored in the memory, wherein when the processor executes the program, the high-resolution optical remote sensing image target detection method as described above is implemented.

[0031] As a third aspect of the present invention, a storage medium is provided, on which a program is stored, and when the program is executed, the high-resolution optical remote sensing image target detection method as described above is implemented.

[0032] Compared with the prior art, the present invention has the following beneficial effects:

[0033] 1) Based on the original YOLOv8s, the present invention adds an efficient bidirectional weighted feature pyramid EBiFPN, integrates the C2f-EB module of the efficient multi-scale attention mechanism and the SWIoU border regression loss function, and improves the detection accuracy of remote sensing images.

[0034] 2) The present invention uses EMA attention to construct the residual module C2F-EB, which replaces the C2F modules at three scales of 80×80, 40×40 and 20×20 in the YOLOv8s neck network, which can suppress background features and enhance the feature extraction ability of the model.

[0035] 3) The present invention designs an efficient bidirectional weighted feature pyramid EBiFPN to replace the neck network of the original YOLOv8s. By increasing the convolution operation, fine-tuning the number of feature channels, and cleverly integrating shallow feature information, the detection accuracy of small targets can be improved.

[0036] 4) The present invention adopts the SWIoU loss function to improve the positioning accuracy of the predicted box by optimizing the aspect ratio penalty term between the predicted box and the true box. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 This is a flow chart of a high-resolution optical remote sensing image target detection method of the present invention;

[0038] Figure 2 A diagram of the BiFPN and EBiFPN network structures provided for an embodiment of the present invention; 2a) BiFPN network structure, 2b) EBiFPN network structure;

[0039] Figure 3 A schematic diagram of the EMA attention mechanism provided for one embodiment of the present invention;

[0040] Figure 4 A structural diagram of an EMABottleneck module provided for one embodiment of the present invention;

[0041] Figure 5A structural diagram of a C2f-EB module provided for one embodiment of the present invention;

[0042] Figure 6 A schematic diagram of a SWIoU loss function provided by an embodiment of the present invention;

[0043] Figure 7 An improved YOLOv8s model structure diagram provided for one embodiment of the present invention;

[0044] Figure 8 A diagram of detection results provided for an embodiment of the present invention. DETAILED DESCRIPTION

[0045] The present invention is described in detail below in conjunction with the accompanying drawings and specific embodiments. This embodiment is implemented based on the technical solution of the present invention, and provides a detailed implementation method and specific operation process, but the protection scope of the present invention is not limited to the following embodiments.

[0046] Example 1

[0047] like Figure 1 As shown, this embodiment provides a remote sensing image target detection method based on an improved YOLOv8s model, comprising the following steps:

[0048] S1. Build an improved YOLOv8s model.

[0049] The improved YOLOv8s model constructed by the present invention is based on the YOLOv8s model, designs an EBiFPN structure to replace the neck network of the YOLOv8s model, and uses a C2f-EB module constructed by an EMA attention mechanism to replace the C2f module of the neck network, and uses a designed SWIoU border regression loss function for training;

[0050] like Figure 2 As shown in the figure, the EBiFPN structure is further optimized and improved on the basis of BiFPN. The BiFPN structure itself has adopted a more efficient architectural design. By removing nodes that only have one-way connections and adding additional branch structures between the input and output nodes of the same layer, it ensures that information can be fully communicated and integrated in both horizontal and vertical paths, thereby greatly enhancing the information interaction capabilities between features of different scales.

[0051] The EBiFPN structure goes a step further by adding 1×1 convolution to adjust the number of feature channels and incorporating shallow feature information from P2, allowing the model to more accurately capture the geometric features of small targets. In terms of feature connection, EBiFPN uses the Concat splicing method, which ensures that feature information can flow freely between levels, further promoting effective interaction and fusion between features of different scales. EBiFPN inputs shallow features including P2 into the detection head through 1×1 convolution. This design significantly improves the model's perception of the geometric features of small targets, thereby significantly improving the accuracy of small target detection.

[0052] like Figure 3 As shown in the figure, in the EMA attention mechanism, the input channels are divided into multiple groups, each of which is responsible for learning different semantic information. Vertical and horizontal average pooling branches: The two parallel branches perform average pooling in the vertical and horizontal directions respectively, and then concatenate the resulting feature maps and share a 1x1 convolution kernel. Through matrix multiplication, the two channel attention maps in each group are aggregated to achieve cross-dimensional feature interaction. Local cross-channel interaction branch: The third branch uses a 3x3 convolution kernel to capture cross-channel interaction information in a local range. This design aims to expand the feature space and enhance the model's sensitivity to local features in complex backgrounds. Finally, two sets of feature maps are obtained through global average pooling, and feature aggregation is performed.

[0053] like Figure 4 As shown in the figure, the bottleneck structure (Bottleneck) in the C2f module is improved, and EMA attention is introduced after the second convolution (Conv) to construct the EMABottleneck module. Figure 5 The C2f-EB module is obtained by replacing the Bottleneck in the C2f module with the EMABottleneck module. The C2f-EB module further improves the recognition accuracy of small targets by enhancing the attention to spatial and channel information features in complex backgrounds.

[0054] Figure 6 Schematic diagram of the SWIoU loss function. The SWIoU bounding box regression loss function consists of outlier (β), angle loss (Λ), distance loss (Δ), shape loss (Ω) and intersection over union loss (IoU). The calculation formula for Λ is:

[0055]

[0056] Among them, ω represents the angle between the line connecting the center point of the predicted box and the real box and the horizontal direction. During the training process, when ω<π / 4, ω will be minimized, and the predicted box will be close to the real box in the horizontal direction; when ω≥π / 4, π / 2-ω will be minimized, and the predicted box will continue to be close to the real box in the vertical direction. According to the above angle loss, the calculation formula of Δ is defined as:

[0057]

[0058]

[0059] Among them, b(b cx ,b cy )and Represents the center point coordinates of the predicted box and the real box, C w With C h Represent the width and height of the minimum bounding box respectively. The calculation formula of Ω is defined as:

[0060]

[0061]

[0062] Among them, w, h and w gt ,h gt Represents the width and height of the real box and the predicted box respectively. The calculation formula of SIoU is:

[0063]

[0064] WIoU introduces an innovative dynamic non-monotonic focusing mechanism that uses outliers to evaluate the quality of anchor boxes and dynamically adjusts the gradient gain according to the different outliers. This effectively reduces the impact of high-quality and low-quality prediction boxes on the gradient, making the loss regression more focused on prediction boxes of normal quality, thereby improving the effect of the bounding box loss function. The calculation formula of WIoU is as follows:

[0065]

[0066]

[0067] Among them, α and δ are hyperparameters, and β is the outlier degree.

[0068] In order to give full play to the advantages of both, the present invention combines the non-monotonic focusing coefficient of WIoU with SIoU and designs the SWIoU loss function. The expression is as follows:

[0069]

[0070] Figure 7This is a schematic diagram of the overall structure of the improved YOLOv8s model in the present invention.

[0071] S2. Obtain high-resolution optical images and image preprocessing, build a dataset, and divide the dataset into training set, validation set, and test set in a ratio of 8:1:1.

[0072] Specifically, 2806 aerial photos of optical remote sensing images were sorted and 15 categories were annotated, namely small vehicles, large vehicles, aircraft, storage tanks, ships, ports, athletic fields, football fields, tennis courts, swimming pools, baseball fields, roundabouts, basketball courts, bridges and helicopters. Due to the large size of the dataset, this paper uses random cropping and expansion to preprocess, and a total of 21,046 images with a resolution of 1024×1024 are obtained as the dataset. In this way, the dataset is divided into training set, validation set and test set in a ratio of 8:1:1.

[0073] S3. Configure the deep learning environment and set the improved YOLOv8s model parameters.

[0074] In this embodiment, the operating system used in the experiment is Ubuntu, the graphics card is NVIDIA GeForce RTX 3090, the GPU size is 32G, the Python version used in the experiment is 3.8, the CUDA version is 11.7, and the deep learning framework is PyTorch 2.0.1. During training, the input images will be uniformly adjusted to 640×640 pixels, and Mosaic data enhancement will be turned on. The model training cycle is set to 200 rounds, the batch size is set to 32, the number of multi-threads is 8, and mixed half-precision training is turned on. The model uses a stochastic gradient descent optimizer with an initial learning rate of 0.01 and a momentum of 0.937. The model uses a weight decay strategy to prevent overfitting, and the decay value is set to 5×10 -4 .

[0075] S4. Input the optical remote sensing image training set into the improved model for training. Use the validation set for verification during the training process, and use the test set to test the trained model.

[0076] This embodiment uses mean average precision (mAP), model parameters (Params) and giga floating point operations (GFLOPs) as evaluation indicators. The calculation method of mean average precision is:

[0077]

[0078]

[0079]

[0080]

[0081] Where R is the recall rate, P is the precision, TP is the number of correctly predicted positive samples, FN is the number of incorrectly detected positive samples, FP is the number of incorrectly predicted negative samples, AP is the area of ​​the precision and recall curve, and N is the number of detection categories.

[0082] S5. Use the trained improved YOLOv8s model to detect targets in remote sensing images.

[0083] Figure 8 This is a detection result diagram obtained by using the method of the present invention in an embodiment. In this implementation, the trained improved model is evaluated, and it can be found that the mAP of the present invention reaches 73.0% on the DATAv1.0 public dataset, the model parameter amount is 12.3M, and the calculation amount is 32.1GFLOPs, which effectively improves the target detection accuracy of optical remote sensing images and occupies less computing resources.

[0084] Example 2

[0085] This embodiment provides a high-resolution optical remote sensing image target detection device of an improved YOLOv8s model, including a memory, a processor, and a program stored in the memory, and when the processor executes the program, the method described in the above embodiment 1 is implemented.

[0086] In a preferred embodiment, the device comprises:

[0087] A model building module is used to build an improved YOLOv8s model. The improved model backbone network includes a CBS module, a C2F module, and an SPPF module. The EBiFPN structure is designed to replace the neck network of the YOLOv8s model. The C2f-EB module built using the EMA attention mechanism replaces the C2f module of the neck network. The designed SWIoU border regression loss function is used for training.

[0088] The dataset construction module is used to acquire remote sensing images, preprocess images, and construct datasets;

[0089] Environment and parameter configuration module, used to configure the deep learning environment and set the improved YOLOv8s model parameters;

[0090] Model training and testing module, used to input high-resolution optical remote sensing image datasets into the improved YOLOv8s model for training and testing;

[0091] The detection module is used to detect remote sensing images using the trained improved model.

[0092] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the described module can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0093] Example 3

[0094] This embodiment provides a storage medium on which a program is stored. When the program is executed, the method described in the above embodiment 1 is implemented.

[0095] Storage media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include temporary computer-readable media (transitory media), such as modulated data signals and carrier waves.

[0096] The preferred specific embodiments of the present invention are described in detail above. It should be understood that a person skilled in the art can make many modifications and changes based on the concept of the present invention without creative work. Therefore, any technical solution that can be obtained by a person skilled in the art through logical analysis, reasoning or limited experiments based on the concept of the present invention on the basis of the prior art should be within the scope of protection determined by the claims.

Claims

1. A high-resolution optical remote sensing image target detection method, characterized in that: The method comprises the following steps: obtaining high-resolution optical remote sensing image data, and detecting targets in the high-resolution optical remote sensing image using a trained improved YOLOv8s model; The improved YOLOv8s model is based on the YOLOv8s model, designs an EBiFPN structure to replace the neck network of the YOLOv8s model; and replaces the C2f module of the neck network with a C2f-EB module constructed using an EMA attention mechanism.

2. The high-resolution optical remote sensing image target detection method according to claim 1, characterized in that: The EBiFPN structure is optimized and improved on the basis of BiFPN, specifically including: in terms of feature connection, the Concat splicing method is adopted; convolution is added to adjust the number of feature channels, and the shallow features including P2 are input into the detection head.

3. The high-resolution optical remote sensing image target detection method according to claim 1, characterized in that: The C2f-EB module adjusts the bottleneck structure inside the C2f module, introduces the EMA efficient multi-head attention mechanism after the second convolution of the C2f module, and constructs the EMA Bottleneck module to replace the bottleneck structure in the C2f module.

4. The high-resolution optical remote sensing image target detection method according to claim 3, characterized in that: The EMA attention mechanism is as follows: Divide the input channels into multiple groups, each group is responsible for learning different semantic information; Vertical and horizontal average pooling branches: The two parallel branches perform average pooling in the vertical and horizontal directions respectively, then concatenate the resulting feature maps and share a convolution kernel. The two channel attention maps in each group are aggregated through matrix multiplication to achieve cross-dimensional feature interaction; Local cross-channel interaction branch: uses convolution kernels to capture cross-channel interaction information within a local range; Two sets of feature maps are obtained through global average pooling, and feature aggregation is performed.

5. The high-resolution optical remote sensing image target detection method according to claim 1, characterized in that: The improved YOLOv8s model constructs a dataset based on annotated and image preprocessed high-resolution optical images, and uses the SWIoU bounding box regression loss function for training and testing.

6. A high-resolution optical remote sensing image target detection method according to claim 5, characterized in that: The SWIoUYOLOv8s model loss function combines the non-monotonic focusing coefficient of WIoU and SIoU, and the expression is as follows: Among them, α, δ are hyperparameters, β is the outlier degree; L SIoU is the bounding box regression loss.

7. A high-resolution optical remote sensing image target detection method according to claim 6, characterized in that: The frame regression loss function SWIoU includes angle loss Λ, distance loss Δ, shape loss Ω and intersection-over-union loss IoU: Among them, ω represents the angle between the line connecting the center point of the predicted box and the real box and the horizontal direction; b(b cx ,b cy )and Represents the center point coordinates of the predicted box and the real box, C w With C h Respectively represent the width and height of the minimum bounding box; w, h and w gt ,h gt Represent the width and height of the real box and the predicted box respectively.

8. The high-resolution optical remote sensing image target detection method according to claim 5, characterized in that: When testing the improved YOLOv8s model, the average precision mean, the number of model parameters, and the number of billion floating-point operations are used as evaluation indicators.

9. A high-resolution optical remote sensing image target detection device, comprising a memory, a processor, and a program stored in the memory, characterized in that: When the processor executes the program, the high-resolution optical remote sensing image target detection method as described in any one of claims 1-8 is implemented.

10. A storage medium having a program stored thereon, characterized in that: When the program is executed, the high-resolution optical remote sensing image target detection method as described in any one of claims 1-8 is implemented.

Images