Improved YOLOv8 model for satellite image ship detection
By improving the YOLOv8 model, adding small object detection branches and feature extraction modules, optimizing the loss function, and using enhancement means in training, the problem of low ship detection accuracy in satellite images is solved, and higher detection accuracy and image quality adaptation are achieved sex.
Patent Information
- Application Number
- CN202411176132.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-26
- Publication Date
- 2024-12-17
AI Technical Summary
The existing YOLO series models face small target recognition difficulties and image quality problems when detecting ships in satellite images, resulting in low detection accuracy, especially under the influence of complex sea surface textures and occlusion factors, making it difficult to accurately extract ship features.
Improve the YOLOv8 model by adding small object detection branches and feature extraction modules, combining feature fusion modules and detection output modules, optimizing the loss function to enhance the detection capability of small object, and using motion blur and cloud mask enhancement methods during the training process to improve the model. Great.
It significantly improves the detection accuracy of small and medium-sized satellite images and the adaptability of models to image quality, improves detection efficiency and accuracy, and meets the needs of marine monitoring.
Smart Images
Figure CN119152385A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of remote sensing image processing, and in particular to an improved YOLOv8 model for satellite image ship detection. Background Art
[0002] In recent years, with the rapid development of remote sensing technology, satellite images have become a powerful tool for obtaining surface information. These images are widely used in many fields such as agricultural monitoring, disaster assessment, urban planning, and marine environmental research. In marine monitoring, satellite images provide all-weather, large-scale observation capabilities, and play an irreplaceable role in monitoring maritime traffic flow and other aspects. Among the detections of many marine targets, ship detection has attracted much attention due to its complexity and importance. Traditional ship detection methods, such as manual visual analysis, are not only time-consuming and labor-intensive, but also inefficient and easily affected by subjective factors. Therefore, deep learning-based target detection models, such as the YOLO series, have been widely used in the field of ship detection due to their advantages such as high degree of automation, fast detection speed, and better accuracy.
[0003] As a new version of the YOLO series, YOLOv8 has become one of the popular algorithms in the current target detection field with its excellent real-time performance and detection accuracy. However, when performing the task of detecting ships in satellite images, the model faces two major challenges: First, in satellite images, due to the long shooting distance and wide viewing angle, ships usually appear as small targets at the pixel level, mixed with complex sea surface textures, waves, shadows, etc., resulting in huge challenges for traditional target detection models in identifying these small targets. Problems such as low signal-to-noise ratio and blurred boundaries of small targets make it difficult for the model to accurately extract their features, which in turn affects the detection accuracy. Second, satellite images are often affected by a variety of factors, including atmospheric scattering, cloud occlusion, sea surface reflection, motion blur, etc. These factors not only reduce the clarity of the image, but also increase the complexity of the detection algorithm. For example, cloud occlusion may completely cover the target, while motion blur will make the target outline unclear, all of which put higher requirements on the detection algorithm.
[0004] Although the existing ship detection models based on the YOLO series have improved the detection performance to a certain extent, they still have limitations in the face of the above challenges, especially in the detection ability of small targets. Small targets are easily ignored in the feature extraction process due to their limited pixel information, resulting in low detection rates. In addition, the standard loss function design often focuses on balancing the detection of various categories and large and small targets, and fails to fully optimize the recognition of small targets, resulting in insufficient capture capabilities of small targets in practical applications. Therefore, there is an urgent need to optimize the performance of the existing YOLO model in satellite image ship detection, especially to improve the recognition accuracy of small targets, while effectively addressing the image quality issues unique to satellite images to meet the growing needs of ocean monitoring and management. Summary of the invention
[0005] In view of this, the present invention proposes an improved YOLOv8 model for satellite image ship detection, which can effectively detect small target ships with high detection accuracy. At the same time, it can effectively deal with the image quality problems unique to satellite images and adapt to different application scenarios.
[0006] An improved YOLOv8 model for satellite image ship detection includes the following steps:
[0007] (1) Data preparation: input remote sensing satellite images, perform truncation and stretching data preprocessing on the images; then mark the ship targets in the images, cut the slice data using sliding windows, and divide the slice data into training set and validation set;
[0008] (2) Model improvement: The improved YOLOv8 model consists of a feature extraction module, a feature fusion module, and a detection output module; the feature extraction module outputs image features of four different scales, which outputs an image feature with a smaller scale than the original YOLOv8 model; the feature fusion module fuses image features of different scales, using image features with a smaller scale; the detection output module consists of four target detection branch networks, which adds a detection branch network to the original YOLOv8 model, and can extract targets with a smaller scale;
[0009] (3) Model training: When training the improved YOLOv8 model, motion blur and cloud masking are used to enhance the input slice data to improve the robustness of the model. In terms of loss function, the improved bounding box loss function contains four parts, one more part than the original bounding box loss function, and the added part corresponds to the output of the small object detection branch.
[0010] (4) Model reasoning: The satellite image that has undergone data preprocessing is subjected to sliding cropping to obtain the slice to be tested. The slice to be tested is sequentially passed through the feature extraction module, feature fusion module, and detection output module of the improved YOLOv8 model to obtain four target detection branch outputs. The target detection results are obtained through splicing and non-maximum suppression operations.
[0011] Preferably, the feature extraction module of step (2) is specifically as follows:
[0012] Taking satellite image slice data as the input unit, the feature extraction module is composed of multiple convolutional networks connected in series to extract features from the slice data. By setting different convolution kernel step sizes, image features from shallow to deep can be obtained; the feature extraction module is composed of 5 convolutional layers, 4 C2f convolutional structures and 1 spatial pyramid pooling layer; the kernel size of the 5 convolutional layers is 3×3, the step size is 2, and 2x downsampling operation will be performed; each C2f convolutional structure includes 2 convolutional layers with a kernel size of 1×1 and a Bottleneck structure with residual; after the input unit passes through the feature extraction module, feature map outputs with downsampling of 4 times, 8 times, 16 times and 32 times will be obtained.
[0013] Preferably, the feature fusion module of step (2) is specifically as follows:
[0014] Taking the feature map output by the feature extraction module as the input unit, the feature fusion module uses a channel aggregation network to fuse image features of different scales; the feature fusion module consists of 3 convolutional layers, 6 C2f convolutional structures, 6 splicing layers and 3 2x upsampling layers; the kernel size of the 3 convolutional layers is 3×3 and the step size is 2; each C2f convolutional structure includes 2 convolutional layers with a kernel size of 1×1 and 1 Bottleneck structure without residual; after the input unit passes through the feature fusion module, it will also obtain feature map outputs that are downsampled 4 times, 8 times, 16 times and 32 times; different from the output of the feature extraction module, the output of the feature fusion module fuses the spatial information of the low-level feature map and the semantic information of the high-level feature map.
[0015] Preferably, the detection output module of step (2) is specifically as follows:
[0016] The feature map output by the feature fusion module is used as the input unit. The detection output module is composed of 4 target detection branch networks. Each target detection branch network contains 3 convolution blocks, each convolution block contains 3 convolution layers. The kernel size of the first two convolution layers is 3×3 and the stride is 1, and the kernel size of the last convolution layer is 1×1 and the stride is 1. The input unit passes through the detection output module to obtain a multidimensional matrix output with target category, position and angle information.
[0017] Preferably, the improved bounding box loss function of step (3) is as follows:
[0018] The original loss function is defined as:
[0019] Loss=Loss iou +Loss cls +Loss dfl
[0020] In the formula, Loss iou Represents the bounding box loss, Loss cls Represents category loss, Loss dfl represents the distribution focal loss;
[0021] Loss iou It can also be expressed as:
[0022] Loss iou =Loss P5_iou +Loss P4_iou +Loss P3_iou
[0023] Among them, Loss P5_iou 、Loss P4_iou 、Loss P3_iou They correspond to the bounding box losses of the original three target detection branches respectively;
[0024] Loss P2_iou For the added small target detection branch’s bounding box loss, the improved bounding box loss function is expressed as:
[0025] Loss iou =Loss P5_iou +Loss P4_iou +Loss P3_iou +Loss P2_iou
[0026] Loss P2_iou =1.0-sigmoid((iou P2 -0.5)×4)
[0027] In the formula, iou P2 It represents the area intersection-union ratio of the small target detection branch. The sigmoid function is defined as follows:
[0028]
[0029] Compared with the prior art, the present invention has the following beneficial effects:
[0030] The present invention improves the model's ability to detect small targets by adding a small target detection branch to the original YOLOv8 model. At the same time, in order to prevent the small target detection branch from experiencing gradient vanishing during the training of the improved YOLOv8 model, which results in a decrease in model performance, the present invention improves the loss function, enhances the model's ability to optimize the gradient of the small target detection branch, and is conducive to obtaining the optimal model parameters. In addition, in view of the data characteristics of satellite images, the present invention adopts enhancement measures such as motion blur and cloud masking on the input data when improving the training of the YOLOv8 model to adapt to different application scenarios. Thanks to the above-mentioned improvement measures, the present invention can effectively improve the detection accuracy of satellite image ships and further promote the widespread application of ship detection based on deep learning methods. BRIEF DESCRIPTION OF THE DRAWINGS
[0031] Figure 1 is a flow chart of the present invention;
[0032] Figure 2 This is the structural diagram of the original YOLOv8 model;
[0033] Figure 3 The structural diagram of the improved YOLOv8 model of the present invention;
[0034] Figure 4 It is a detection effect diagram of the present invention. DETAILED DESCRIPTION
[0035] In order to better understand the technical content of the present invention, specific embodiments will be provided below in conjunction with the drawings in the present invention, and the present invention will be further described in conjunction with the drawings.
[0036] An improved YOLOv8 model for ship detection in satellite images, such as Figure 1 As shown, the following steps are included:
[0037] (1) Data preparation: Remote sensing satellite images are input and preprocessed by truncating and stretching the images. Then, ship targets are marked in the images and sliced data is obtained by sliding window cropping. The sliced data is divided into a training set and a validation set.
[0038] (2) Model improvement: The original YOLOv8 model is improved. The original YOLOv8 model structure is shown in the following figure. Figure 2 The improved YOLOv8 model structure is shown in Figure 3As shown in the figure, it consists of a feature extraction module, a feature fusion module and a detection output module; the feature extraction module will output image features of 4 different scales, and output an image feature with a smaller scale than the original YOLOv8 model; the feature fusion module will fuse image features of different scales, and use image features with a smaller scale during the process; the detection output module consists of 4 target detection branch networks, and adds a detection branch network compared to the original YOLOv8 model, which can extract targets with a smaller scale. The three modules are as follows:
[0039] Taking satellite image slice data as the input unit, the feature extraction module is composed of multiple convolutional networks connected in series to extract features from the slice data. By setting different convolution kernel step sizes, image features from shallow to deep can be obtained; the feature extraction module is composed of 5 convolutional layers, 4 C2f convolutional structures and 1 spatial pyramid pooling layer; the kernel size of the 5 convolutional layers is 3×3, the step size is 2, and 2x downsampling operation will be performed; each C2f convolutional structure includes 2 convolutional layers with a kernel size of 1×1 and a Bottleneck structure with residual; after the input unit passes through the feature extraction module, feature map outputs with downsampling of 4 times, 8 times, 16 times and 32 times will be obtained.
[0040] Taking the feature map output by the feature extraction module as the input unit, the feature fusion module uses a channel aggregation network to fuse image features of different scales; the feature fusion module consists of 3 convolutional layers, 6 C2f convolutional structures, 6 splicing layers and 3 2x upsampling layers; the kernel size of the 3 convolutional layers is 3×3 and the step size is 2; each C2f convolutional structure includes 2 convolutional layers with a kernel size of 1×1 and 1 Bottleneck structure without residual; after the input unit passes through the feature fusion module, it will also obtain feature map outputs that are downsampled 4 times, 8 times, 16 times and 32 times; different from the output of the feature extraction module, the output of the feature fusion module fuses the spatial information of the low-level feature map and the semantic information of the high-level feature map.
[0041] The feature map output by the feature fusion module is used as the input unit. The detection output module is composed of 4 target detection branch networks. Each target detection branch network contains 3 convolution blocks, each convolution block contains 3 convolution layers. The kernel size of the first two convolution layers is 3×3 and the stride is 1, and the kernel size of the last convolution layer is 1×1 and the stride is 1. The input unit passes through the detection output module to obtain a multidimensional matrix output with target category, position and angle information.
[0042] (3) Model training: Improve the training strategy: When training the improved YOLOv8 model, motion blur and cloud masking are used to enhance the input slice data to improve the robustness of the model; in terms of loss function, the improved bounding box loss function contains 4 parts, one more part than the original bounding box loss function, and the added part corresponds to the output of the small target detection branch. The improved bounding box loss function is as follows:
[0043] During the model training process, the optimizer continuously adjusts the model parameters through the function calculation loss to obtain the best effect. The original loss function is defined as:
[0044] Loss=Loss iou +Loss cls +Loss dfl
[0045] In the formula, Loss iou Represents the bounding box loss, Loss cls Represents category loss, Loss dfl represents the distribution focal loss;
[0046] Loss iou It can also be expressed as:
[0047] Loss iou =Loss P5_iou +Loss P4_iou +Loss P3_iou
[0048] Among them, Loss P5_iou 、Loss P4_iou 、Loss P3_iou They correspond to the bounding box losses of the original three target detection branches respectively;
[0049] The present invention improves the loss function iou , the bounding box loss of the small target detection branch is increased, denoted as Loss P2_iou , then the improved bounding box loss function is expressed as:
[0050] Loss iou =Loss P5_iou +Loss P4_iou +Loss P3_iou +Loss P2_iou
[0051] Loss P2_iou =1.0-sigmoid((iou P2 -0.5)×4)
[0052] In the formula, iouP2 It represents the area intersection-union ratio of the small target detection branch. The sigmoid function is defined as follows:
[0053]
[0054] (4) Model reasoning: The satellite image that has undergone data preprocessing is subjected to sliding cropping to obtain the slice to be tested. The slice to be tested is sequentially subjected to the feature extraction module, feature fusion module, and detection output module of the improved YOLOv8 model to obtain four target detection branch outputs. The target detection results are obtained through splicing and non-maximum suppression operations. After manual correction, the detection results can be used as a performance indicator to evaluate the model detection accuracy, and can also be used for the subsequent target sample set production.
[0055] In this embodiment, a sample set is made using remote sensing satellite images, and the sample set is divided into a training set and a validation set according to a ratio of 9:1, specifically 609 training set samples and 68 validation set samples. These samples are used for YOLOv5, original YOLOv8, S2ANet, R3Det model and the improved YOLOv8 model of the present invention to train the optimal model weight. The training cycle (round) of all models is 100, and the training size is 800. The S2ANet and R3Det model training adopts the SGD optimization method, the initial learning rate is set to 0.01, and the training strategy of multi-step adjustment of the learning rate is adopted, the coefficient is 0.1, and it is adjusted in the 50th and 90th cycles respectively. YOLOv5, original YOLOv8 and the improved YOLOv8 of the present invention are trained using the Adam optimization method, the learning rate is set to 0.01, and the training strategy of degraded learning rate is adopted.
[0056] In the inference test phase, 100 slices to be tested are used as the test set, and the size of each slice to be tested is 800 pixels × 800 pixels. YOLOv5, original YOLOv8, S2ANet, R3Det model and the improved YOLOv8 model of the present invention are used for the inference test of the test set, and the inference results of each model are compared with the annotated true results. The accuracy (Precision), recall (Recall) and mean average precision (mean Average Precision, mAP) are used as performance evaluation indicators. The results are shown in the following table:
[0057] Reasoning test results of each model
[0058] Model Precision Recall mAP50 Improving YOLOv8 0.939 0.924 0.963 Original YOLOv8 0.916 0.859 0.947 YOLOv5 0.865 0.868 0.942 S2ANet — 0.875 0.810 R3D — 0.806 0.802
[0059] It can be seen from the above table that the accuracy, recall rate and mean average precision of the improved YOLOv8 model of the present invention for detecting ships in remote sensing satellite images are higher than those of other models, that is, the detection accuracy of the present invention is higher. Figure 4 In the figure, the left picture is the original satellite image, and the right picture is the detection result obtained by using the improved YOLOv8 model. It can also be seen from the figure that the present invention can effectively detect ship targets.
[0060] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For ordinary technicians in the relevant field, several improvements and modifications can be made to the present invention without departing from the technical principles of the present invention, and these improvements and modifications also fall within the scope of protection of the claims of the present invention.
Claims
1. An improved YOLOv8 model for satellite image ship detection, characterized in that: The following steps are involved: (1) Data preparation: input remote sensing satellite images, perform truncation and stretching data preprocessing on the images; then mark the ship targets in the images, cut the slice data using sliding windows, and divide the slice data into training set and validation set; (2) Model improvement: The improved YOLOv8 model consists of a feature extraction module, a feature fusion module and a detection output module; the feature extraction module outputs image features of four different scales, including an image feature of a smaller scale; the feature fusion module fuses image features of different scales, during which image features of a smaller scale are used; the detection output module consists of four target detection branch networks, which can extract targets of a smaller scale; (3) Model training: When training the improved YOLOv8 model, motion blur and cloud masking are used to enhance the input slice data to improve the robustness of the model. In terms of loss function, the improved bounding box loss function contains four parts, including the output part corresponding to the small object detection branch; (4) Model reasoning: The satellite image that has undergone data preprocessing is subjected to sliding cropping to obtain the slice to be tested. The slice to be tested is sequentially passed through the feature extraction module, feature fusion module, and detection output module of the improved YOLOv8 model to obtain four target detection branch outputs. The target detection results are obtained through splicing and non-maximum suppression operations.
2. The improved YOLOv8 model according to claim 1, characterized in that The feature extraction module of step (2) is specifically as follows: Taking satellite image slice data as the input unit, the feature extraction module is composed of multiple convolutional networks connected in series to extract features from the slice data. By setting different convolution kernel step sizes, image features from shallow to deep can be obtained; the feature extraction module is composed of 5 convolutional layers, 4 C2f convolutional structures and 1 spatial pyramid pooling layer; the kernel size of the 5 convolutional layers is 3×3, the step size is 2, and 2x downsampling operation will be performed; each C2f convolutional structure includes 2 convolutional layers with a kernel size of 1×1 and a Bottleneck structure with residual; after the input unit passes through the feature extraction module, feature map outputs with downsampling of 4 times, 8 times, 16 times and 32 times will be obtained.
3. The improved YOLOv8 model according to claim 2, characterized in that The feature fusion module of step (2) is specifically as follows: Taking the feature map output by the feature extraction module as the input unit, the feature fusion module uses a channel aggregation network to fuse image features of different scales; the feature fusion module consists of 3 convolutional layers, 6 C2f convolutional structures, 6 splicing layers and 3 2x upsampling layers; the kernel size of the 3 convolutional layers is 3×3 and the step size is 2; each C2f convolutional structure includes 2 convolutional layers with a kernel size of 1×1 and 1 Bottleneck structure without residual; after the input unit passes through the feature fusion module, it will also obtain feature map outputs that are downsampled 4 times, 8 times, 16 times and 32 times; different from the output of the feature extraction module, the output of the feature fusion module fuses the spatial information of the low-level feature map and the semantic information of the high-level feature map.
4. The improved YOLOv8 model according to claim 3, characterized in that The detection output module of step (2) is specifically as follows: The feature map output by the feature fusion module is used as the input unit. The detection output module is composed of 4 target detection branch networks. Each target detection branch network contains 3 convolution blocks, each convolution block contains 3 convolution layers. The kernel size of the first two convolution layers is 3×3 and the stride is 1, and the kernel size of the last convolution layer is 1×1 and the stride is 1. The input unit passes through the detection output module to obtain a multidimensional matrix output with target category, position and angle information.
5. The improved YOLOv8 model according to claim 1, characterized in that The improved bounding box loss function of step (3) is as follows: Loss P2_iou is the bounding box loss of the small target detection branch, then the improved bounding box loss function is expressed as: Loss iou =Loss P5_iou +Loss P4_iou +Loss P3_iou +Loss P2_iou Loss P2_iou =1.0-sigmoid((iou P2 -0.5)×4) In the formula, iou P2 It represents the area intersection-union ratio of the small target detection branch. The sigmoid function is defined as follows: