Remote sensing image aircraft target detection method based on YOLOv8 improvement

Abstract

The invention discloses an improved remote sensing image aircraft target detection method based on YOLOv8. The method comprises the steps of operating an environment and an image data set, introducing a C2f-DCNv2 module, adding an additional small target detection layer, introducing an improved attention mechanism, and introducing a scale sequence fusion mechanism SF. A deformable convolution C2f-DCNv2 module is introduced into a backbone network of a YOLOv8 network, and a small target detection layer is added to a neck network, so that the feature extraction capability of the backbone network is improved, the feature loss of the small target is reduced, and the recognition capability of the network on the small aircraft target is enhanced; an improved VCK-CBAM attention module is introduced to the neck of the YOLOv8 network, the key aircraft target area features are further strengthened, the model is helped to effectively focus on the target, and the recognition ability of the model to the aircraft target in the complex background is improved; a scale sequence fusion mechanism SF is introduced into the neck of an original YOLOv8 network, so that information from a multi-scale feature map is fused in a network feature map, and the detection precision of aircraft targets with different scales is enhanced. According to the improved YOLOv8 network model provided by the invention, the mean average precision value (mAP 0.5) can be improved by 2.53%.

Description

Technical Field

[0001] The invention relates to a remote sensing image aircraft target detection method based on improved YOLOv8, and belongs to the field of remote sensing image target detection. Background Art

[0002] With the rapid development of remote sensing satellite technology, remote sensing technology has become an advanced and practical earth observation and space exploration technology, which has a profound impact on the development of social economy. Object detection is an important research field of computer vision, which aims to find all the objects of interest in the image by extracting the features of the target objects, and determine their types, locations and other useful information. Aircraft targets are special creations of human civilization activities. They are not only important means of transportation for daily travel, but also an indispensable strategic resource in national defense. This makes the identification and positioning of aircraft a hot research direction in the current tasks in this field.

[0003] Although science and technology are becoming increasingly prosperous, the number of cars is increasing, and high-speed railways are constantly opening up, people's travel has become more convenient, but airplanes still occupy an extremely important position in modern transportation due to their fast, convenient, and wide-ranging characteristics. By interpreting the aircraft information contained in remote sensing images in a targeted manner, people can apply it in various scenarios such as searching for crashed aircraft, scheduling and dispatching flights at airport dispatch desks, and other fields.

[0004] In remote sensing aircraft images, aircraft targets are small in size, the image background is complex, the aircraft targets are unevenly distributed, and the aircraft targets are of different sizes. It is difficult to achieve ideal results when using current target detection algorithms to process remote sensing aircraft images.

[0005] Most traditional aircraft detection methods require manually designed features. The algorithm detection speed is slow when processing massive data, and the accuracy and robustness are greatly reduced when faced with the complex background of remote sensing images.

[0006] The development of artificial intelligence technology has brought new changes to the ability to interpret remote sensing images, which can significantly shorten the remote sensing image interpretation cycle and improve the accuracy of interpretation. Compared with traditional target detection methods, the use of convolutional neural networks for image interpretation and target detection has huge advantages in accuracy. The emergence of convolutional neural networks integrates the target feature extraction, feature selection and feature classification functions into a single network, which can automatically learn different types of features of images at all levels, with faster detection speed, stronger feature expression ability, and richer semantic information. Therefore, there is still a lot of room for research on the use of convolutional neural network frameworks for aircraft target detection. Summary of the invention

[0007] 1. Technical issues to be resolved

[0008] In view of the shortcomings of the prior art, the present invention provides an improved remote sensing image aircraft target detection method based on YOLOv8, which solves the problems raised in the above background technology.

[0009] (II) Technical solution

[0010] In order to achieve the above-mentioned purpose, the present invention specifically adopts the following technical solutions:

[0011] A remote sensing image aircraft target detection method based on improved YOLOv8 includes an operating environment, an image data set, and solutions to the problems raised in the above background technology.

[0012] The operating environment includes an Ubuntu 20.04 operating system, an Intel i7-11700 eight-core CPU, and an NVIDIA GeFORCE RTX 3090Ti for building a training platform, a deep learning framework of Pytorch 1.9.0, a Python programming language, and a CUDA 11.4 computing platform.

[0013] The image dataset of the remote sensing image aircraft target detection method improved based on YOLOv8 adopts the DOTA dataset, which has 16 categories, contains 2806 pictures, and a total of 188282 remote sensing target instances, and the picture resolution ranges from 800×800 to 4000×4000.

[0014] Preferably, images containing aircraft targets in the DOTA data set are extracted, images with a resolution greater than 1024×1024 in the data set are cropped to 1024×1024, the portion less than 1024 after cropping is padded, 70% of the final image data set is divided into a training set, 15% into a test set, and 15% into a validation set, and the DOTA data labels are converted into data labels in YOLO format for subsequent training, validation, and testing.

[0015] The improved remote sensing image aircraft target detection method based on YOLOv8 aims to solve the problem that the size of aircraft targets in remote sensing aircraft images is small, resulting in poor effect on small target detection and easy missed detection. In order to further improve the feature extraction capability of the backbone network, the present invention introduces a deformable convolution DCNv2 module into the C2f module in the backbone network of YOLOv8, and effectively improves the feature extraction capability of the network by expanding the receptive field.

[0016] Preferably, in order to further enhance the network's ability to recognize small aircraft targets in remote sensing images, the present invention adds an extra detection layer in YOLOv8 without changing the size of other feature maps, that is, a small target detection layer of 160×160 pixels is added, and the size of the detected aircraft target is larger than 4×4 pixels, thereby reducing the feature loss of small targets and enhancing the network's ability to recognize small aircraft targets.

[0017] The improved remote sensing image aircraft target detection method based on YOLOv8 aims at the problem that remote sensing aircraft images have complex aircraft target backgrounds. The present invention proposes a VCK-CBAM attention mechanism based on the CBAM attention mechanism, introduces a variable convolution kernel in the spatial attention module of the CBAM attention mechanism, and continuously adjusts the size of the convolution kernel according to the size of the input feature map to change the size of the receptive field. The VCK-CBAM attention mechanism is introduced into the neck network to further strengthen the key aircraft target area features, help the model effectively focus on the target, and improve the model's ability to recognize aircraft targets in complex backgrounds.

[0018] The improved remote sensing image aircraft target detection method based on YOLOv8 aims at the problem that remote sensing aircraft images have aircraft targets presenting multiple scales. The present invention introduces a scale sequence fusion mechanism SF into the neck of the original YOLOv8, introduces a triple feature encoder TFE (Triple Feature Encoder) module and a scale sequence feature fusion SSFF (Scalesequence feature fusion) module, redesigns the bottom-up path of the neck of the original network model, realizes the fusion of multi-scale aircraft target features, and enhances the network's ability to extract aircraft target features of different scales.

[0019] (III) Beneficial effects

[0020] Compared with the prior art, the present invention provides an improved remote sensing image aircraft target detection method based on YOLOv8, which has the following beneficial effects:

[0021] The present invention improves the YOLOv8 target detection algorithm and introduces a deformable convolution DCNv2 module into the C2f module in the backbone network to further improve the feature extraction capability of the backbone network. An additional small target detection layer is added to YOLOv8 without changing the size of other feature maps to enhance the network's detection capability of small aircraft targets in remote sensing images. By introducing an improved VCK-CBAM attention mechanism module into the neck network, the variable convolution kernel helps to more accurately distinguish between aircraft targets and backgrounds in complex backgrounds. By introducing a scale sequence fusion mechanism, the network's ability to extract features of aircraft targets of different scales is enhanced. Experimental verification shows that the improved YOLOv8 network model of the present invention can improve the mean average precision (mAP) by 1.1%. 0.5 ) increased by 2.53 percentage points. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 A schematic diagram of the overall network structure provided by the present invention;

[0023] Figure 2 A schematic diagram of a C2f-DCNv2 module provided by the present invention;

[0024] Figure 3 A schematic diagram of the scale sequence fusion mechanism SF provided by the present invention;

[0025] Figure 4 This is a schematic diagram of the VCK-CBAM module provided by the present invention.

[0026] Figure 5 This is a diagram of the detection and recognition results provided by the present invention. DETAILED DESCRIPTION

[0027] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0028] Example

[0029] The overall structure of the remote sensing image aircraft target detection in the embodiment provided by the present invention is as follows: Figure 1 As shown, the remote sensing image aircraft target detection method in this example includes an operating environment, an image dataset, the introduction of a C2f-DCNv2 module and the addition of an additional small target detection layer, the introduction of an improved attention mechanism, and the introduction of a scale sequence fusion mechanism SF.

[0030] Based on the above system, this embodiment provides a remote sensing image aircraft target detection method based on YOLOv8 improvement. The following operations are performed:

[0031] The specific implementation step S1, the operating environment includes the operating system Ubuntu 20.04, the training platform is built with Intel i7-11700 eight-core CPU and NVIDIA GeFORCE RTX 3090Ti, the deep learning framework is Pytorch1.9.0, the programming language is Python, and the computing platform is CUDA11.4.

[0032] Specifically, step S2 is implemented as follows: the data set uses the DOTA data set, extracts the pictures and corresponding labels containing the aircraft targets in the data set, and crops the images with a resolution greater than 1024×1024 in the data set according to 1024×1024, and fills the part less than 1024 after cropping. However, the cropping operation may cause the loss of edge aircraft target information. For this reason, the present invention retains an overlapping area of ​​512 pixels during cropping. At the same time, the DOTA data labels are converted into data labels in the YOLO format, and 70% of the images are divided into a training set, 15% of the images are divided into a test set, and 15% of the images are divided into a verification set for subsequent training, verification, and testing.

[0033] Specifically, in step S3, the remote sensing images of the training set are input into the backbone network to extract features. The convolutional layer is used and the C2f-DCNv2 module is introduced to expand the receptive field to effectively improve the feature extraction capability of the network. Feature maps of sizes 160×160, 80×80, 40×40, and 20×20 are obtained in the 3rd, 5th, 7th, and 9th layers of the backbone network, respectively.

[0034] Specifically, in step S4, the feature map is fused through the SPPF module, the input is passed through the CBS convolution layer, and then through three maximum pooling layers, the feature map without maximum pooling and the feature map obtained after each additional maximum pooling are concat spliced ​​to achieve feature fusion.

[0035] Specifically, in step S5, the high-level semantic information of the three feature maps of different sizes output by the backbone network is further fused using the introduced SSFF module. The output feature map fuses the information from the multi-scale feature maps and retains the details and semantic content more comprehensively.

[0036] Specifically, in step S6, a newly added three-feature encoder TFE module is used to fuse feature maps of three different sizes to obtain richer semantic and spatial information.

[0037] Specifically, step S7 is implemented to reduce feature loss by using the newly added small target detection layer. Adding the small target detection layer helps to improve the network's perception of small aircraft targets. The high-resolution feature map can retain more small target locations and detail information, and further extract aircraft target features.

[0038] Specifically, step S8 is implemented by introducing the improved VCK-CBAM attention mechanism to strengthen the key aircraft target area features, help the model effectively focus on the aircraft target, suppress interference noise, and improve the model's ability to identify areas of interest.

[0039] Specifically, in step S9, the top-down convolution operation of the neck network is used to reduce the size of the feature map, and the feature maps of the same size from the bottom-up are fused, and the processing of the C2f and VCK-CBAM modules is combined to finally generate the detection result of the aircraft target, such as Figure 5 The detection and recognition results are shown in the figure.

[0040] like Figure 2 As shown, in some embodiments, the C2f-DCNv2 module is introduced in S3 to expand the receptive field and effectively improve the feature extraction capability of the network. The specific process is: considering the relationship between the efficiency and performance of the model, the second 3×3 standard convolution in the Bottleneck of the C2f module is replaced by DCNv2 to obtain C2f-DCNv2, and the C2f-DCNv2 module is used to replace all C2f modules in the backbone network. C2f-DCNv2 adopts multi-branch and residual structure to more effectively capture the detailed features of the aircraft target, so that the network can capture more contextual information.

[0041] like Figure 3 As shown, in some embodiments, the scale sequence fusion mechanism SF is used in S5 and S6 to combine multi-scale feature maps, which not only contains multi-scale global semantic information, but also retains sufficient spatial details, so that the bottom layer information is easier to pass to the top of the high-level layer. The specific process is: the input of the TFE module of the 12th layer comes from feature maps of sizes 80×80, 40×40 and 20×20, and after fusing feature maps of different scales, a feature map of size 40×40 is output; the TFE module of the 15th layer receives feature maps of sizes 160×160, 80×80 and 40×40, and after fusing feature maps of different scales, a feature map of size 80×80 is output; the SSFF module receives feature maps of size 160×160 from the 3rd layer, 80×80 from the 5th layer, and 40×40 from the 7th layer, respectively. Compared with the direct input method of the original network model, the output feature map fuses the information from the multi-scale feature map.

[0042] like Figure 4As shown, in some embodiments, the VCK-CBAM module introduced in S8 further strengthens the key aircraft target area features, helps the model to effectively focus on the target, suppresses interference noise, and thus improves the detection capability of the target. In addition, the attention mechanism not only improves the distinguishability of the features, but also provides more distinguishing input features for the detection head, making the classification and positioning of the detection head more accurate. Specifically, the basic idea of ​​the variable convolution kernel is to continuously adjust the size of the convolution kernel according to the size of the input feature map and change the size of the receptive field. That is, when processing 160×160 and 80×80 feature maps, these larger feature maps usually contain richer information and broader context. In order to capture more context information, a 7×7 convolution kernel is used; when processing a 40×40 feature map, in order to capture the necessary context information and pay attention to enough details, a 5×5 convolution kernel is used; when processing a 20×20 feature map, the information of each pixel is more concentrated, and the overall context information that can be provided is limited, and a 3×3 convolution kernel is used.

[0043] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art can still modify the technical solutions described in the aforementioned embodiments or replace some of the technical features therein by equivalents. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A remote sensing image aircraft target detection method based on improved YOLOv8 includes an operating environment, an image dataset, the introduction of a C2f-DCNv2 module and the addition of an additional small target detection layer, the introduction of an improved attention mechanism, and the introduction of a scale sequence fusion mechanism SF.

2. According to the remote sensing image aircraft target detection method described in claim 1, the operating environment is characterized in that: The operating system is Ubuntu 20.04, and the training platform is built with Intel i7-11700 eight-core CPU and NVIDIA GeFORCE RTX 3090Ti. The deep learning framework is Pytorch1.9.0, the programming language is Python, and the computing platform is CUDA11.

4.

3. According to the remote sensing image aircraft target detection method described in claim 1, the data set is characterized in that: Using the public DOTA dataset, the pictures and corresponding labels containing aircraft targets in the dataset were extracted, and the images with a resolution greater than 1024×1024 in the dataset were cropped to 1024×1024. The part less than 1024 after cropping was padded, and the overlapping area of ​​512 pixels was retained during cropping. At the same time, the DOTA data labels were converted into data labels in YOLO format, and 70% of the images were divided into training sets, 15% of the images into test sets, and 15% of the images into validation sets.

4. According to the remote sensing image aircraft target detection method described in claim 1, the introduction of the C2f-DCNv2 module and the introduction of an additional small target detection layer are characterized in that: The second 3×3 standard convolution in the Bottleneck of the C2f module is replaced by DCNv2 to obtain C2f-DCNv2. The C2f-DCNv2 module is used to replace all C2f modules in the backbone network. The multi-branch and residual structure are adopted to more effectively capture the detailed features of the aircraft target. A 160×160 pixel small target detection layer is added to YOLOv8 without changing the size of other feature maps. The detected target size is larger than 4×4 pixels, which reduces the feature loss of small targets and enhances the network's recognition ability of small aircraft targets.

5. According to the remote sensing image aircraft target detection method described in claim 1, the improved attention mechanism is introduced, characterized in that: A variable convolution kernel is introduced into the spatial attention module of the CBAM attention mechanism to obtain VCK-CBAM. The VCK-CBAM attention mechanism continuously adjusts the size of the convolution kernel according to the size of the input feature map and changes the size of the receptive field. The VCK-CBAM attention mechanism is introduced into the neck network to further strengthen the characteristics of the key aircraft target area, help the model effectively focus on the target, and improve the model's ability to recognize aircraft targets in complex backgrounds.

6. According to the remote sensing image aircraft target detection method described in claim 1, the scale sequence fusion mechanism SF is characterized in that: The scale sequence fusion mechanism SF is introduced into the neck of the original YOLOv8, the triple feature encoder TFE (Triple Feature Encoder) module and the scale sequence feature fusion SSFF (Scale sequence feature fusion) module are introduced, and the bottom-up path of the original neck network model is redesigned to realize the fusion of multi-scale aircraft target features and enhance the network's ability to extract aircraft target features of different scales.

Images