Ship detection method and device for optical remote sensing image based on YOLOv8

Abstract

The application provides a YOLOv8-based optical remote sensing image ship detection method and device, and relates to the technical field of remote sensing image recognition. The method comprises the following steps: constructing an intelligent ship detection model, pre-processing a first data set and a second data set respectively to obtain a first ship image and a second ship image with labels; inputting the first ship image and the second ship image into the intelligent ship detection model in sequence, training the intelligent ship detection model, and obtaining a trained intelligent ship detection model; inputting a prediction image into the trained intelligent ship detection model, detecting prediction ship information; and post-processing the prediction ship information to obtain a detection image with labels. The Backbone module of the intelligent ship detection model comprises an LSK block and a C2f module with a DWR module introduced, so that the feature representation of the feature map is comprehensive, and the Neck module with a SimAM introduced, so that the detection accuracy of small and dense ship targets is relatively high.

Description

Technical Field

[0001] This invention relates to the field of remote sensing image recognition technology, and in particular to a method and apparatus for ship detection based on optical remote sensing images using YOLOv8. Background Technology

[0002] Remote sensing technology is a scientific method that uses long-distance observation and measurement. Utilizing various sensors such as optical, microwave, and infrared sensors, and transmitted via satellites, aircraft, or drones, it acquires information about targets on the Earth's surface, atmosphere, and water bodies, and transforms this information into identifiable remote sensing images or data. Remote sensing images include remote sensing optical ship images, and the use of these images for ship target detection is highly valued in the field of marine remote sensing. Ship target detection allows for the timely and effective discovery and extraction of target ship information.

[0003] Currently, remote sensing ship target detection based on convolutional neural networks (CNNs) requires collecting and labeling remote sensing image data of both ships and non-ships; selecting a CNN architecture and training it on a training set, monitoring model performance to prevent overfitting; evaluating model performance using a test set and adjusting hyperparameters to optimize performance during the model evaluation phase; and finally deploying the trained model to a real-world application environment, adjusting thresholds and post-processing as needed to obtain ship images. However, CNN-based remote sensing ship target detection suffers from significant performance degradation, particularly in detecting small, densely packed ship targets. This is due to the vast differences in the length and width of ship targets, the relatively high density of near-shore ships, and the small size of ship targets relative to the background. Summary of the Invention

[0004] The purpose of this invention is to provide a ship target detection method and device based on YOLOv8, which solves the problem of low detection accuracy for small and dense ship targets.

[0005] To address the aforementioned technical problems, the embodiments of the present invention provide the following technical solutions:

[0006] The first aspect of this invention provides a method for ship detection based on YOLOv8 optical remote sensing images, the method comprising:

[0007] A smart ship detection model is constructed, which includes the YOLOv8 Backbone module and Neck module. The Backbone module includes a large split convolution kernel module and a channel-to-pixel module containing an expanded residual sub-module. The Neck module includes a parameterless attention module.

[0008] The first and second datasets are preprocessed to obtain multiple labeled first ship images and multiple labeled second ship images. The labeled target format is suitable for intelligent ship detection models.

[0009] Multiple first ship images and multiple second ship images are sequentially input into the intelligent ship detection model to train the intelligent ship detection model and obtain a trained intelligent ship detection model.

[0010] The predicted image is input into the trained intelligent ship detection model to detect the predicted ship information;

[0011] The predicted ship information is post-processed to obtain an labeled detection image.

[0012] The second aspect of this application provides a ship detection device based on YOLOv8 optical remote sensing images, the device comprising:

[0013] The building module is used to build an intelligent ship detection model. The intelligent ship detection model includes the YOLOv8 Backbone module and Neck module. The Backbone module includes a large separating convolution kernel module and a channel-to-pixel module containing an expanded residual sub-module. The Neck module includes a parameterless attention module.

[0014] The preprocessing module is used to preprocess the first dataset and the second dataset respectively to obtain multiple first ship images with annotations and multiple second ship images with annotations. The target format of the annotation is suitable for the intelligent ship detection model.

[0015] The training module is used to input multiple first ship images and multiple second ship images into the intelligent ship detection model in sequence to train the intelligent ship detection model and obtain a trained intelligent ship detection model.

[0016] The detection module is used to input the predicted image into the trained intelligent ship detection model to detect the predicted ship information.

[0017] The post-processing module is used to post-process the predicted ship information to obtain an labeled detection image.

[0018] Compared to existing technologies, the YOLOv8-based optical remote sensing image ship detection method and apparatus provided in this invention constructs an intelligent ship detection model. It preprocesses a first dataset and a second dataset to obtain multiple labeled first ship images and multiple labeled second ship images. These first and second ship images are then sequentially input into the intelligent ship detection model for training, resulting in a trained model. Predicted images are then input into the trained model to detect predicted ship information. Post-processing of the predicted ship information yields labeled detection images. This approach leverages the fact that the intelligent ship detection model's Backbone module includes LSKblock and a C2f module incorporating a DWR submodule, allowing for dynamic adjustment of the network's receptive field and enhanced information extraction capabilities at different scales. This results in more comprehensive feature representation of the feature maps. Furthermore, the introduction of the SimAM Neck module enhances the detection accuracy for small, densely packed ship targets. Attached Figure Description

[0019] The above and other objects, features, and advantages of exemplary embodiments of the present invention will become readily apparent upon reading the following detailed description with reference to the accompanying drawings. In the drawings, several embodiments of the invention are illustrated by way of example and not limitation, with the same or corresponding reference numerals denoteing the same or corresponding parts, wherein:

[0020] Figure 1 The flowchart of the ship detection method based on optical remote sensing images using YOLOv8 is illustrated schematically. Figure 1 ;

[0021] Figure 2 The flowchart of the ship detection method based on optical remote sensing images using YOLOv8 is illustrated schematically. Figure 2 ;

[0022] Figure 3 The network structure diagram of the Backbone module is shown schematically.

[0023] Figure 4 A schematic diagram of the network structure of a channel-to-pixel module including an extended residual submodule is shown.

[0024] Figure 5 The network structure diagram of the Neck module is shown schematically.

[0025] Figure 6 The network structure diagram of the intelligent ship detection model is shown schematically.

[0026] Figure 7 The diagram illustrates a comparison of training accuracy for the first dataset.

[0027] Figure 8 A schematic diagram of a ship detection device based on YOLOv8 optical remote sensing images is shown. Detailed Implementation

[0028] Exemplary embodiments of the invention will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the invention are shown in the drawings, it should be understood that the invention can be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided to enable a more thorough understanding of the invention and to fully convey the scope of the invention to those skilled in the art.

[0029] It should be noted that, unless otherwise stated, the technical or scientific terms used in this invention should have the ordinary meaning as understood by those skilled in the art.

[0030] The methods described in the embodiments of the present invention will be explained in detail below.

[0031] Figure 1 A flowchart illustrating a ship detection method for YOLOv8-based optical remote sensing images in an embodiment of the present invention is shown schematically. See [link to flowchart illustration]. Figure 1 As shown, the method may include:

[0032] S101. Construct an intelligent ship detection model.

[0033] The intelligent ship detection model includes YOLOv8's Backbone module and Neck module. The Backbone module includes a large separating convolution kernel module and a channel-to-pixel module containing an expanded residual submodule. The Neck module includes a parameterless attention module.

[0034] YOLOv8's backbone module includes a large split convolution kernel module (LSKblock) and a channel-to-pixel (C2f) module containing a dilation wise residual (DWR) submodule. A C2f module containing DWR can be represented as C2f_DWR.

[0035] LSKblock can dynamically adjust the receptive field of the network, improving the detection accuracy of small ship targets. The C2f module, which includes DWR, can reduce the increase in network depth brought about by LSKblock and can fuse multi-scale receptive fields to generate feature maps, making the feature maps more comprehensive.

[0036] The Neck module contains a Simple Parameter-Free Attention Module (SimAM), which captures the saliency and importance of features in the feature map, giving higher weights to important features and thus enhancing the representational power of features. This helps improve the performance of tasks such as ship target detection and classification.

[0037] S102. Preprocess the first dataset and the second dataset respectively to obtain multiple labeled first ship images and multiple labeled second ship images.

[0038] The target format for annotation is suitable for intelligent ship detection models.

[0039] The first dataset can be the HRSC2016 dataset, and the second dataset can be the FGSD dataset. Various dataset types are possible, and no specific restrictions are placed here. The annotation formats for the first and second datasets differ. Both datasets contain optical remote sensing images of ship targets.

[0040] Preprocessing may include image slicing, annotation format conversion, and augmentation processing. Augmentation processing includes horizontal and vertical flipping, cropping and stitching, adding noise, and randomizing the stitching.

[0041] The preprocessed image containing the ship target was selected as the training sample.

[0042] The order of steps S101 and S102 is not specifically limited here. Step S101 can be executed first and then step S102, or step S102 can be executed first and then step S101.

[0043] S103. Input multiple first ship images and multiple second ship images into the intelligent ship detection model in sequence to train the intelligent ship detection model and obtain the trained intelligent ship detection model.

[0044] All labeled first ship images in the preprocessed first dataset and all second ship images in the preprocessed second dataset are sequentially input into the intelligent ship detection model to train the intelligent ship detection model until the model reaches the preset number of training rounds or the loss value no longer decreases and the evaluation meets the standard, thus obtaining the trained intelligent ship detection model.

[0045] S104. Input the predicted image into the trained intelligent ship detection model to detect the predicted ship information.

[0046] The predicted ship information includes rotation angle, preset bounding box, preset category confidence, and target confidence.

[0047] S105. Post-process the predicted ship information to obtain an labeled detection image.

[0048] Post-processing is performed on the rotation angle, preset bounding box, preset category confidence, and target confidence to obtain an labeled detection image.

[0049] Post-processing includes nonmaximum suppression, bounding box decoding, class confidence thresholding, coordinate decoding, and annotation.

[0050] Based on the above Figure 1 As can be seen from the implementation method, the embodiments of the present invention construct an intelligent ship detection model, preprocess the first dataset and the second dataset respectively to obtain multiple labeled first ship images and multiple labeled second ship images; input the multiple first ship images and multiple second ship images sequentially into the intelligent ship detection model to train the intelligent ship detection model, and obtain a trained intelligent ship detection model; input the predicted image into the trained intelligent ship detection model to detect the predicted ship information; postprocess the predicted ship information to obtain a labeled detection image. In this way, since the backbone module of the intelligent ship detection model includes LSKblock and the C2f module with the introduction of DWR submodule, the receptive field of the network can be dynamically adjusted, enhancing the ability to extract information at different scales, making the feature map feature representation more comprehensive, and the introduction of SimAM's Neck module can achieve high detection accuracy for small and dense ship targets.

[0051] As a refinement and extension of the above embodiments, Figure 2 The following is a flowchart of a ship detection method based on YOLOv8 optical remote sensing images in an embodiment of the present invention. Figure 2 See Figure 2 As shown in the figure, an embodiment of the present invention provides a ship detection method based on YOLOv8 optical remote sensing images, which may include:

[0052] S201. Construct an intelligent ship detection model.

[0053] The intelligent ship detection model includes YOLOv8's Backbone module, Neck module, and Head module. The Head module includes one convolutional group module and one ordinary convolutional module.

[0054] The Backbone module consists of two convolutional module groups (ConvModule), one channel-to-pixel module (i.e., a C2f module containing DWR), one convolutional module group, one channel-to-pixel module (i.e., a C2f module containing DWR), one convolutional module group, one channel-to-pixel module (i.e., a C2f module containing DWR), one convolutional module group, one channel-to-pixel module (i.e., a C2f module containing DWR), one large separable convolutional kernel module, and one Spatial Pyramid Pooling Fast (SPPF) module.

[0055] Figure 3 The network structure of the Backbone module is illustrated schematically. The Backbone module incorporating the LSKblock, from top to bottom, includes one ConvModul, one ConvModul, one C2f (i.e., a C2f module containing DWR), one ConvModul, one C2f (i.e., a C2f module containing DWR), one ConvModul, one C2f (i.e., a C2f module containing DWR), one ConvModul, one C2f (i.e., a C2f module containing DWR), one LSKblock, and one SPPF. The second C2f (i.e., a C2f module containing DWR), the third C2f (i.e., a C2f module containing DWR), and the SPPF are used as outputs.

[0056] The channel-to-pixel module consists of one convolutional group module, one splitting layer, n dilated residual sub-modules, one concatenation layer, and one convolutional group module.

[0057] Figure 4 The diagram schematically illustrates the network structure of a channel-to-pixel module containing an extended residual submodule. The C2f module with DWR introduced includes, from top to bottom, one ConvModul, one Split, n DWRs, one Concat, and one ConvModul.

[0058] The Neck module containing SimAM consists of, in sequence, one upsampling layer, one stitching layer, one channel-to-pixel module, one upsampling layer, one stitching layer, one C2f module (i.e., a C2f module containing DWR), one parameterless attention module, one convolutional group module, one stitching layer, one channel-to-pixel module, one parameterless attention module, one convolutional group module, one stitching layer, one channel-to-pixel module, and one parameterless attention module.

[0059] Figure 5The network structure of the Neck module is illustrated schematically. The Neck module incorporating SimAM consists of, in sequence: 1 Upample, 1 Concat, 1 C2f (i.e., a C2f module containing DWR), 1 Upample, 1 Concat, 1 C2f (i.e., a C2f module containing DWR), 1 SimAM, 1 ConvModul, 1 Concat, 1 C2f (i.e., a C2f module containing DWR), 1 SimAM, 1 ConvModul, 1 Concat, 1 C2f (i.e., a C2f module containing DWR), and 1 SimAM. The first, second, and fourth Concat are taken as inputs, and the first, second, and third SimAMs are taken as outputs.

[0060] Figure 6 The network structure diagram of the intelligent ship detection model is schematically shown. The intelligent ship detection model includes, in sequence, one ConvModul, one ConvModul, one C2f_DWR (i.e., a C2f module containing DWR), one ConvModul, one C2f_DWR, one ConvModul, one C2f_DWR, one ConvModul, one C2f_DWR, one LSKblock, one SPPF, one Upample, one Concat, one C2f_DWR, one Upample, one Concat, one C2f_DWR, one SimAM, one ConvModul, one Concat, one C2f_DWR, one SimAM, one ConvModul, one Concat, one C2f_DWR, one SimAM, one ConvModul, and one ordinary convolutional module (Conv2d).

[0061] The following steps S202-S205 are specific operations for preprocessing the first dataset and the second dataset respectively to obtain multiple labeled first ship images and multiple labeled second ship images.

[0062] S202. According to the preset overlap rate, the first dataset and the second dataset are cut respectively to obtain multiple first cut images and multiple second cut images.

[0063] Ships are present in both the first and second cut images.

[0064] The first dataset can be the HRSC2016 dataset, and the second dataset can be the FGSD dataset. Various dataset types are possible, and no specific restrictions are placed here. The annotation formats for the first and second datasets differ. Both datasets contain optical remote sensing images of ship targets.

[0065] Because the remote sensing images in each dataset have excessively high resolution and the size of the ships in these images occupies a relatively small proportion of the entire image, compressing the images to the corresponding resolution using the resizing method might result in the loss of key target information, leading to model training failure. Therefore, the excessively high-resolution remote sensing images are cut into smaller blocks according to a preset overlap rate, ultimately resulting in multiple first-cut images corresponding to the first dataset and multiple second-cut images corresponding to the second dataset.

[0066] The preset overlap rate can be 50%, and the resolution of the multiple first-cut images and multiple second-cut images is 1024×1024. The multiple first-cut images and multiple second-cut images after cutting are labeled images.

[0067] S203. Convert the annotation format corresponding to each first cut image and each second cut image into the target format to obtain multiple first converted images and multiple second converted images.

[0068] The target format includes the ship category. The labeled target format is applicable to intelligent ship detection models.

[0069] Because the annotation formats of the various datasets (i.e., multiple first-segment images and multiple second-segment images) are not uniform, it is necessary to convert the annotation formats corresponding to each first-segment image and each second-segment image into the annotation format that conforms to the intelligent ship detection model, i.e., the target format, to obtain multiple first-transformed images and multiple second-transformed images. The resulting multiple first-transformed images and multiple second-transformed images are all images with the target format.

[0070] The target format includes the ship category and the coordinates of the four vertices of the preset bounding box. The target format is (class, x1, y1, x2, y2, x3, y3, x4, y4), where class is the ship category and (xi, yi) is the coordinates of the four vertices of the rotated preset bounding box relative to the whole image.

[0071] S204. When the number of ship categories is greater than the preset value, the corresponding multiple first converted images and multiple second converted images are respectively determined as multiple first ship images and multiple second ship images.

[0072] After step S203, it is necessary to determine the ship categories of multiple first ship images and multiple second ship images. If the ship category is greater than a preset value, the corresponding multiple first converted images and multiple second converted images are respectively determined as multiple first ship images and multiple second ship images. If the ship category is not greater than the preset value, the following step S205 is executed.

[0073] The default value can be 10% of the maximum number of ship categories.

[0074] S205. When the number of ship types is not greater than a preset value, the corresponding multiple first conversion images and multiple second conversion images are expanded to obtain multiple first ship images and multiple second ship images.

[0075] When the number of ship types is no greater than a preset value, the corresponding multiple first-transformed images and multiple second-transformed images are expanded. The expansion can be horizontal and vertical flipping, which makes the target's posture more diverse; it can also be cropping and stitching, which, in order to increase the number of targets on a single image, crops the parts of different images containing targets and stitches them into a new image; it can also be noise perturbation, which improves the robustness of the model by randomly adding Gaussian noise; or it can be random processing stitching, which can stitch multiple images together in a random scaling, random cropping, and random arrangement manner.

[0076] The above expansion can increase the number of small and dense target samples, making the training data distribution more uniform.

[0077] S206. Input multiple first ship images and multiple second ship images into the intelligent ship detection model in sequence to train the intelligent ship detection model and obtain the trained intelligent ship detection model.

[0078] All labeled first ship images in the preprocessed first dataset and all second ship images in the preprocessed second dataset are sequentially input into the intelligent ship detection model to train the intelligent ship detection model until the model reaches the preset number of training rounds or the loss value no longer decreases and the evaluation meets the standard, thus obtaining the trained intelligent ship detection model.

[0079] Specifically, the parameter settings during training were as follows: the HRSC2016 dataset (the first dataset) had a total of 900 training epochs, and the FGSD dataset (the second dataset) had a total of 300 training epochs. The SGD optimizer was used to optimize the network, with an initial learning rate of 0.01, a momentum parameter of 0.937, an optimizer weight decay coefficient of 0.0005, a target bounding box loss weight of 12.0, a class loss weight of 0.5, and a DFL loss weight of 1.5. No pre-trained parameters were used in the experiments. The training data (i.e., multiple images of the first and second ships) was fed into the model. In forward propagation, the input data was processed by the intelligent ship detection model to obtain predicted values. Then, the loss between the predicted and true values ​​was calculated. Next, the gradient was calculated using the backpropagation algorithm, and the optimizer was used to update the weights and biases of the intelligent ship detection model. Finally, this process was repeated until the intelligent ship detection model reached the preset number of training epochs or the loss value stopped decreasing. After training the intelligent ship detection model, it needs to be evaluated using the test set allocated in step one to test its generalization ability, i.e., its performance on unseen data. Evaluation metrics can include accuracy, precision, and recall. Once training is complete, the intelligent ship detection model can be saved as a file for future use. When needed for detection, the model can be loaded from the file and used directly.

[0080] S207. Input the predicted image into the trained intelligent ship detection model to detect the predicted ship information.

[0081] Specifically, the predicted image is input into the trained intelligent ship detection model to detect the predicted ship information, including:

[0082] Step A1: Input the predicted image into the Backbone module, and output the first feature map P3, the second feature map P4, and the third feature map P5.

[0083] The resolution of the first feature map is greater than that of the second feature map, and the resolution of the second feature map is greater than that of the third feature map.

[0084] Specifically, the predicted image is input into the Backbone module, which outputs a first feature map, a second feature map, and a third feature map, including:

[0085] Step A11: Input the predicted image into two convolutional group modules, one channel-to-pixel module, one convolutional group module, and one channel-to-pixel module in sequence, and output the first feature map.

[0086] Step A12: Input the first feature map into one convolutional group module and one channel into the pixel module in sequence, and output the second feature map.

[0087] Step A13: Input the second feature map into one convolutional group module, one channel-to-pixel module, one large separating convolutional kernel module, and one fast spatial pyramid pooling module in sequence, and output the third feature map.

[0088] Step A2: Input the first feature map, the second feature map, and the third feature map into the Neck module, and output the fourth feature map T1, the fifth feature map T2, and the sixth feature map T3.

[0089] Specifically, the first, second, and third feature maps are input into the Neck module, and the fourth, fifth, and sixth feature maps are output, including:

[0090] Step A21: Input the third feature map into one upsampling layer, output the first sampled feature map, and input the first sampled feature map and the second feature map into one stitching layer and one channel into the pixel module in sequence, and output the first intermediate feature map F1.

[0091] Step A22: Input the first intermediate feature map into one upsampling layer, output the second sampled feature map, and input the second sampled feature map and the first feature map into one concatenation layer, one channel to the pixel module and one parameterless attention module in sequence, and output the fourth feature map.

[0092] Step A23: Input the fourth feature map into a convolutional group module, output the second intermediate feature map, and input the second intermediate feature map and the first intermediate feature map into a concatenation layer, a channel-to-pixel module, and a parameterless attention module in sequence, and output the fifth feature map.

[0093] Step A24: Input the fifth feature map into a convolutional group module, output the third intermediate feature map, and input the third intermediate feature map and the third feature map into a concatenation layer, a channel-to-pixel module, and a parameterless attention module in sequence, outputting the sixth feature map.

[0094] Step A3: Input the output fourth feature map, fifth feature map and sixth feature map into one convolutional group module and one ordinary convolutional module in sequence to output the predicted ship information.

[0095] The predicted ship information includes rotation angle, preset bounding box, preset category confidence, and target confidence.

[0096] S208. Post-process the predicted ship information to obtain an labeled detection image.

[0097] Post-processing includes nonmaximum suppression, bounding box decoding, class confidence thresholding, coordinate decoding, and annotation.

[0098] Specifically, the predicted ship information includes rotation angle, preset bounding box, preset category confidence, and target confidence. Post-processing of the predicted ship information yields an labeled detection image, including:

[0099] Step B1: Based on nonmaximum suppression, filter the preset bounding boxes to obtain the target bounding boxes, which include the original coordinates.

[0100] Non-maximum suppression (NMS) iterates through all predefined bounding boxes and compares the overlap (e.g., intersection-over-union ratio) between pairs of predefined bounding boxes. A threshold of 0.5 is set; if the overlap between two predefined bounding boxes exceeds this threshold, the lower-confidence bounding box is suppressed by the higher-confidence bounding box and thus removed from the final detection result. By suppressing and eliminating redundant, highly overlapping predefined bounding boxes, only those most likely to be the real target are retained, thereby improving the accuracy and efficiency of target detection.

[0101] Step B2: Filter the confidence scores of the preset categories according to the preset threshold to obtain the confidence scores of the target categories.

[0102] The confidence scores of a preset category are compared with a preset threshold. Confidence scores above the threshold are retained, while those below the threshold are discarded, resulting in a set containing only high-confidence preset category scores. These high-confidence sets are more likely to represent real targets, thus improving the accuracy and reliability of ship target detection.

[0103] The preset threshold can be set according to the specific scenario, and there is no specific limitation.

[0104] Step B3: Decode the original coordinates into the target coordinates.

[0105] The original coordinates are coordinates in the range of 0 to 1, and the target coordinates are the coordinates of the actual position of the bounding box.

[0106] Within the filtered target bounding box, predicted values ​​associated with that bounding box are obtained. These predicted values ​​include the center point coordinate offset, width and height scaling factors, and confidence scores. Then, the center point coordinate offset is applied to the feature map grid through decoding to obtain the center point position of the bounding box on the feature map. Similarly, the width and height scaling factors are used to calculate the actual width and height of the bounding box on the feature map. Next, since the feature map is obtained through downsampling, the decoded bounding box coordinates need to be mapped back from the feature map space to the original image space. Finally, the decoded bounding boxes are adjusted as needed to prevent predicting bounding boxes that are too large or too small. The output of bounding box decoding is the transformed and calculated actual bounding box coordinates, i.e., the target coordinates.

[0107] Step B4: Label the rotation angle, target bounding box, target category confidence, and target confidence in the prediction image to obtain a labeled detection image.

[0108] The embodiments of the present invention can be further illustrated by the following experiments:

[0109] The high-performance computer system used in the experiment was configured with: an Intel Core i9-13900KF CPU (5.8GHz), 32GB of RAM, and an NVIDIA RTX 4090 GPU. The operating system was Ubuntu 20.04.164-bit. The programming environment consisted of Python 3.8.18, PyTorch 1.12.0, and cudatoolkit 11.6.

[0110] Simulation results:

[0111] Figure 7 A schematic diagram illustrates the comparison of training accuracy on the first dataset. The horizontal axis represents the number of training epochs, and the vertical axis represents training accuracy, recall, mAP_0.5, mAP_0.5:0.95, bounding box loss (train / box_loss), DFL loss (train / dfl_loss), and class loss (train / als_loss). The blue line represents the original YOLOv8 model, the orange line represents the original YOLOv8 model with the addition of C2F and DWR modules, the green line represents the original YOLOv8 model with the addition of C2F and SIMAM modules including DWR, and the red line represents the method of this invention. Based on the visualization results of the HRSC2016 dataset training, it is clear that the accuracy and recall of this invention are higher than the original YOLOv8 model throughout the training process, while the bounding box loss, class loss, and DFL loss are all lower than those of the original YOLOv8 model. While improving performance, the model also converges faster than the original model. Therefore, this proves that the improvement in this paper achieves comprehensive optimization and enhancement based on the original YOLOv8 model.

[0112] From the comparative experiments below, Tables 1 and 2 define mAP50 as the model's average precision when the IoU between the predicted and ground truth boxes is greater than or equal to 0.5. mAP75 represents the model's average precision when the IoU is greater than or equal to 0.75. mAP50:95 indicates that the IoU threshold starts at 0.5 and increases in increments of 0.05 to 0.95. This metric calculates the average of all mAP values ​​within this IoU threshold range. VOC07 uses the 11-point interpolation method to calculate Average Precision (AP). In this method, precision is interpolated at 11 fixed points (0.0, 0.1, 0.2, ..., 1.0) from 0 to 1 for recall. VOC12 uses a more precise interpolation method, namely the all-point interpolation method. This method uses every possible recall value to calculate precision, instead of using 11 fixed points like VOC07. This method is more refined and can provide more accurate AP calculations.

[0113] Tables 1 and 2 below fully demonstrate that the detection performance of the model of this invention is greatly improved for small and dense targets. In the three datasets, the evaluation metrics AP75 and AP50-95 are far superior to the current R3Det, ​​S2ANet, KLD, Gliding Vertex, ROI TransFormer and KFIOU models, proving the effectiveness of the improvement of this paper for small and dense targets.

[0114] Table 1 Comparative trials on the HRSC2016 dataset

[0115]

[0116] Table 2 Comparative trials on the FGSD dataset

[0117]

[0118] To verify the effectiveness of adding the LSKblock module to Backbone, improving C2f, and adding the SimAM attention mechanism to Neck, ablation experiments were conducted on the HRSC and FGSD datasets to compare detection performance, as shown in Tables 3 and 4:

[0119] Table 3 Ablation trials on the HRSC2016 dataset

[0120]

[0121] Table 4 Ablation trials on the FGSD dataset

[0122]

[0123] Analysis of the data in Tables 3 and 4 shows improvements across all evaluation metrics on the HRSC2016 and FGSD datasets. Furthermore, the introduction of the SimAM attention mechanism in the Neck effectively improves the model's detection accuracy for dense targets, extremely small targets, and difficult targets, with varying degrees of improvement in mAP75 and mAP50-95 across the three datasets. Finally, the introduction of the LSKblock module in the Backbone further enhances the model's detection accuracy. Ultimately, the improved YOLOv8-based model demonstrates significant accuracy improvements compared to the original YOLOv8 model on the HRSC2016, FGSD, and SDR9 datasets, while reducing the model size from 139.6MB to 131.8MB, a reduction of 5.59%. This improvement in both detection speed and accuracy is achieved while simultaneously reducing the computational overhead during model inference.

[0124] Based on the same inventive concept, as an implementation of the above-mentioned ship detection method based on YOLOv8 optical remote sensing images, this embodiment of the invention also provides a ship detection device based on YOLOv8 optical remote sensing images. Figure 8 This is a structural diagram of the device in an embodiment of the present invention. See also: Figure 8 As shown, the device may include:

[0125] Module 801 is used to build an intelligent ship detection model. The intelligent ship detection model includes the YOLOv8 Backbone module and Neck module. The Backbone module includes a large split convolution kernel module and a channel-to-pixel module containing an expanded residual sub-module. The Neck module includes a parameterless attention module.

[0126] The preprocessing module 802 is used to preprocess the first dataset and the second dataset respectively to obtain multiple first ship images with annotations and multiple second ship images with annotations. The target format of the annotation is suitable for the intelligent ship detection model.

[0127] The training module 803 is used to input multiple first ship images and multiple second ship images preprocessed by the preprocessing module 802 into the intelligent ship detection model constructed by the construction module 801 in sequence to train the intelligent ship detection model and obtain the trained intelligent ship detection model.

[0128] The detection module 804 is used to input the predicted image into the intelligent ship detection model trained by the training module 803 to detect the predicted ship information.

[0129] The post-processing module 805 is used to post-process the predicted ship information detected by the detection module 804 to obtain a labeled detection image.

[0130] The Backbone module in the intelligent ship detection model, which is constructed in module 801, includes, in sequence, two convolutional group modules, one channel-to-pixel module, one convolutional group module, one channel-to-pixel module, one convolutional group module, one channel-to-pixel module, one convolutional group module, one channel-to-pixel module, one large separable convolutional kernel module, and one fast spatial pyramid pooling module.

[0131] The construction module 801, the channel-to-pixel module, includes one convolutional group module, one segmentation layer, n dilated residual sub-modules, one stitching layer and one convolutional group module in sequence.

[0132] The Neck module, consisting of module 801, comprises, in sequence, one upsampling layer, one stitching layer, one channel-to-pixel module, one upsampling layer, one stitching layer, one channel-to-pixel module, one parameterless attention module, one convolutional group module, one stitching layer, one channel-to-pixel module, one parameterless attention module, one convolutional group module, one stitching layer, one channel-to-pixel module, and one parameterless attention module.

[0133] The detection module 804 is specifically used to input the predicted image into the Backbone module and output a first feature map, a second feature map, and a third feature map. The resolution of the first feature map is greater than that of the second feature map, and the resolution of the second feature map is greater than that of the third feature map. The first feature map, the second feature map, and the third feature map are input into the Neck module and output a fourth feature map, a fifth feature map, and a sixth feature map. The output fourth feature map, the fifth feature map, and the sixth feature map are then input into one convolutional group module and one ordinary convolutional module in sequence to output the predicted ship information. The intelligent ship detection model also includes a YOLOv8 Head module, which includes one convolutional group module and one ordinary convolutional module in sequence.

[0134] The detection module 804 inputs the predicted image into the Backbone module and outputs a first feature map, a second feature map, and a third feature map. This includes: sequentially inputting the predicted image into two convolutional group modules, one channel-to-pixel module, one convolutional group module, and one channel-to-pixel module to output the first feature map; sequentially inputting the first feature map into one convolutional group module and one channel-to-pixel module to output the second feature map; and sequentially inputting the second feature map into one convolutional group module, one channel-to-pixel module, one large separating convolutional kernel module, and one fast spatial pyramid pooling module to output the third feature map.

[0135] The detection module 804 inputs the first, second, and third feature maps into the Neck module and outputs the fourth, fifth, and sixth feature maps. This includes: inputting the third feature map into an upsampling layer to output a first sampled feature map; sequentially inputting the first and second sampled feature maps into a stitching layer with one channel into the pixel module to output a first intermediate feature map; inputting the first intermediate feature map into an upsampling layer to output a second sampled feature map; and sequentially inputting the second sampled feature map and the first feature map into a stitching layer with one channel into the pixel module to output a first intermediate feature map. The fourth feature map is output from the pixel module and a parameterless attention module. The fourth feature map is input into a convolutional group module to output the second intermediate feature map. The second intermediate feature map and the first intermediate feature map are then input into a concatenation layer, a channel-to-pixel module, and a parameterless attention module in sequence to output the fifth feature map. The fifth feature map is input into a convolutional group module to output the third intermediate feature map. The third intermediate feature map and the third feature map are then input into a concatenation layer, a channel-to-pixel module, and a parameterless attention module in sequence to output the sixth feature map.

[0136] The preprocessing module 802 is specifically used to segment the first dataset and the second dataset according to a preset overlap rate to obtain multiple first segmented images and multiple second segmented images, each containing ships; to convert the annotation format corresponding to each first segmented image and each second segmented image into a target format to obtain multiple first transformed images and multiple second transformed images, the target format including ship categories; when the number of ship categories is greater than a preset value, the multiple first transformed images and multiple second transformed images are respectively determined as multiple first ship images and multiple second ship images; when the number of ship categories is not greater than a preset value, the multiple first transformed images and multiple second transformed images are expanded to obtain multiple first ship images and multiple second ship images.

[0137] The post-processing module 805 is specifically used to post-process the predicted ship information to obtain an annotated detection image, including: filtering preset bounding boxes according to non-maximum suppression to obtain target bounding boxes, the target bounding boxes including the original coordinates; filtering preset category confidence scores according to preset thresholds to obtain target category confidence scores; decoding the original coordinates into target coordinates; and annotating the rotation angle, target bounding box, target category confidence score, and target confidence score in the predicted image to obtain an annotated detection image; the predicted ship information includes the rotation angle, preset bounding box, preset category confidence score, and target confidence score.

[0138] It should be noted that the above description of the YOLOv8-based optical remote sensing image ship detection device embodiment is similar to the description of the above method embodiment, and has similar beneficial effects. For technical details not disclosed in the embodiments of the YOLOv8-based optical remote sensing image ship detection device of the present invention, please refer to the description of the method embodiment of the present invention for understanding.

[0139] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A ship detection method based on YOLOv8 optical remote sensing images, characterized in that, Ship target detection methods include: A smart ship detection model is constructed, which includes a YOLOv8 Backbone module and a Neck module. The Backbone module includes a large separating convolution kernel module and a channel-to-pixel module containing an expanded residual submodule. The Neck module includes a parameterless attention module. The first dataset and the second dataset are preprocessed respectively to obtain multiple first ship images with annotations and multiple second ship images with annotations. The target format of the annotations is suitable for the intelligent ship detection model. The plurality of first ship images and the plurality of second ship images are sequentially input into the intelligent ship detection model to train the intelligent ship detection model and obtain a trained intelligent ship detection model. The predicted image is input into the trained intelligent ship detection model to detect the predicted ship information; The predicted ship information is post-processed to obtain an labeled detection image; The Backbone module sequentially includes two convolutional group modules, one channel-to-pixel module, one convolutional group module, one channel-to-pixel module, one convolutional group module, one channel-to-pixel module, one convolutional group module, one channel-to-pixel module, one large split convolutional kernel module, and one fast spatial pyramid pooling module. The Neck module sequentially includes one upsampling layer, one stitching layer, one channel-to-pixel module, one upsampling layer, one stitching layer, one channel-to-pixel module, one parameterless attention module, one convolutional group module, one stitching layer, one channel-to-pixel module, one parameterless attention module, one convolutional group module, one stitching layer, one channel-to-pixel module, and one parameterless attention module.

2. The ship detection method using optical remote sensing images according to claim 1, characterized in that, The channel-to-pixel module sequentially includes one convolutional group module, one segmentation layer, n dilated residual sub-modules, one stitching layer, and one convolutional group module.

3. The ship detection method using optical remote sensing images according to claim 1, characterized in that, The intelligent ship detection model also includes a YOLOv8 Head module, which comprises one convolutional group module and one ordinary convolutional module. The process of inputting the predicted image into the trained intelligent ship detection model to detect the predicted ship information includes: The predicted image is input into the Backbone module, which outputs a first feature map, a second feature map, and a third feature map. The resolution of the first feature map is greater than that of the second feature map, and the resolution of the second feature map is greater than that of the third feature map. The first feature map, the second feature map, and the third feature map are input into the Neck module, and the fourth feature map, the fifth feature map, and the sixth feature map are output. The output fourth feature map, the fifth feature map, and the sixth feature map are sequentially input into one convolutional group module and one ordinary convolutional module to output the predicted ship information.

4. The ship detection method using optical remote sensing images according to claim 3, characterized in that, The step of inputting the predicted image into the Backbone module and outputting a first feature map, a second feature map, and a third feature map includes: The predicted image is sequentially input into two convolutional group modules, one channel-to-pixel module, one convolutional group module, and one channel-to-pixel module to output the first feature map; The first feature map is sequentially input into one of the convolutional group modules and one of the channels into the pixel module, and the second feature map is output. The second feature map is sequentially input into one of the convolutional group modules, one of the channel-to-pixel modules, one of the large split convolutional kernel modules, and one of the fast spatial pyramid pooling modules, and the third feature map is output.

5. The ship detection method using optical remote sensing images according to claim 3, characterized in that, The step of inputting the first feature map, the second feature map, and the third feature map into the Neck module and outputting the fourth feature map, the fifth feature map, and the sixth feature map includes: The third feature map is input into one upsampling layer to output a first sampled feature map. The first sampled feature map and the second feature map are then sequentially input into one channel of one stitching layer to a pixel module to output a first intermediate feature map. The first intermediate feature map is input into one of the upsampling layers, and a second sampled feature map is output. The second sampled feature map and the first feature map are then sequentially input into one of the stitching layers, one of the channel-to-pixel modules, and one of the parameterless attention modules, and the fourth feature map is output. The fourth feature map is input into a convolutional group module, which outputs a second intermediate feature map. The second intermediate feature map and the first intermediate feature map are then sequentially input into a concatenation layer, a channel-to-pixel module, and a parameterless attention module, which outputs the fifth feature map. The fifth feature map is input into a convolutional group module, which outputs a third intermediate feature map. The third intermediate feature map and the third feature map are then sequentially input into a concatenation layer, a channel-to-pixel module, and a parameterless attention module, which outputs the sixth feature map.

6. The ship detection method based on optical remote sensing images according to claim 1, characterized in that, The preprocessing of the first and second datasets to obtain corresponding labeled first ship images and labeled second ship images includes: According to a preset overlap rate, the first dataset and the second dataset are cut into segments respectively to obtain multiple first segmented images and multiple second segmented images, each of which contains ships. The annotation formats corresponding to each of the first cut images and each of the second cut images are converted into the target format to obtain a plurality of first converted images and a plurality of second converted images, wherein the target format includes ship categories; When the number of ship categories is greater than a preset value, the corresponding plurality of first converted images and plurality of second converted images are respectively determined as the plurality of first ship images and the plurality of second ship images; When the number of ship categories is not greater than a preset value, the corresponding plurality of first converted images and plurality of second converted images are expanded to obtain the plurality of first ship images and the plurality of second ship images.

7. The method for ship detection using optical remote sensing images according to claim 1, characterized in that, The predicted ship information includes rotation angle, preset bounding box, preset category confidence score, and target confidence score. The post-processing of the predicted ship information to obtain an labeled detection image includes: Based on nonmaximum suppression, the preset bounding box is filtered to obtain the target bounding box, which includes the original coordinates; The confidence scores of the preset categories are filtered according to a preset threshold to obtain the confidence scores of the target categories; Decode the original coordinates into target coordinates; The rotation angle, the target bounding box, the target category confidence score, and the target confidence score are all labeled in the prediction image to obtain the labeled detection image.

8. A ship detection device based on YOLOv8 optical remote sensing images, characterized in that, The optical remote sensing image ship detection device includes: The construction module is used to build an intelligent ship detection model. The intelligent ship detection model sequentially includes a YOLOv8 Backbone module and a Neck module. The Backbone module includes a large separating convolutional kernel module and a channel-to-pixel module containing an expanded residual submodule. The Neck module includes a parameterless attention module. The Backbone module sequentially includes two convolutional group modules, one channel-to-pixel module, one convolutional group module, one channel-to-pixel module, one convolutional group module, one channel-to-pixel module, and one convolutional group module. The module comprises: one channel-to-pixel module, one large split convolutional kernel module, and one fast spatial pyramid pooling module; the Neck module sequentially includes one upsampling layer, one stitching layer, one channel-to-pixel module, one upsampling layer, one stitching layer, one channel-to-pixel module, one parameterless attention module, one convolutional group module, one stitching layer, one channel-to-pixel module, one parameterless attention module, one convolutional group module, one stitching layer, one channel-to-pixel module, and one parameterless attention module; The preprocessing module is used to preprocess the first dataset and the second dataset respectively to obtain multiple first ship images with annotations and multiple second ship images with annotations. The target format of the annotations is suitable for the intelligent ship detection model. The training module is used to sequentially input the plurality of first ship images and the plurality of second ship images into the intelligent ship detection model to train the intelligent ship detection model and obtain a trained intelligent ship detection model. The detection module is used to input the predicted image into the trained intelligent ship detection model and detect the predicted ship information. The post-processing module is used to post-process the predicted ship information to obtain a labeled detection image.

Images