SPD-YOLOv8-based remote sensing image ship detection method

Abstract

The invention discloses a remote sensing image ship detection method based on SPD-YOLOv8, which belongs to the technical field of satellite remote sensing, is used for ship detection, and comprises the following steps: acquiring a satellite remote sensing image ship data set in an experimental sea area, cutting a remote sensing image, and performing radiation correction and geometric correction; dividing the data set into a training set, a test set and a verification set; performing image enhancement on the data set; an SPD-YOLOv8 ship target detection model is constructed; performing ship target detection by using the ship target detection model; parameter tuning and result evaluation are carried out; deploying the ship target detection model to a server; and abnormal behaviors and illegal activities are detected and early warned. Compared with the prior art, the method has the advantages that the detection effect of low-resolution images and small targets is improved, the accuracy, real-time performance and response speed of the detection model are improved, ship targets of different sizes and types can be learned and generalized more effectively, and the robustness and practicability of a detection algorithm are improved.

Description

Technical Field

[0001] The invention discloses a remote sensing image ship detection method based on SPD-YOLOv8, and belongs to the technical field of satellite remote sensing. Background Art

[0002] Satellite remote sensing technology has been widely used in the field of marine monitoring due to its wide coverage, strong monitoring capability and high real-time performance. In terms of ship detection, high-resolution optical images and synthetic aperture radar (SAR) images each have unique advantages. High-resolution optical images can quickly obtain and update information about ships in navigation through timely monitoring and response capabilities, and have high timeliness and accurate target recognition capabilities. Its wide data coverage enables it to cover a large area of ​​sea, support the monitoring and analysis of long-term changes, and is particularly suitable for application scenarios that require refined observations and high details. In contrast, synthetic aperture radar (SAR) images can accurately monitor ships in all-weather and bad weather conditions through their unique three-dimensional imaging capabilities and high sensitivity. SAR images are not restricted by weather and lighting conditions, have the advantages of penetrating clouds and haze, and can provide high-resolution images and multi-polarization information, effectively detecting small ships and hidden targets. Taking all factors into consideration, high-resolution optical images can provide accurate target recognition and detailed feature display under good lighting conditions, while SAR images have more advantages in all-weather monitoring and applications in complex environments. The combined use of the two can improve the accuracy and robustness of ship detection, provide a more comprehensive and reliable ship monitoring and safety management solution, effectively respond to various maritime monitoring needs, and thus achieve a higher level of maritime safety and management capabilities.

[0003] Remote sensing images are difficult to process using traditional image processing methods due to their large data volume, fast-changing speed, multiple types, low resolution, and complex ground object types. With the improvement of remote sensing satellite imaging resolution and the performance of target detection algorithms, deep learning-based ship target detection algorithms have become a trend, which are mainly divided into two categories: regional detection (Two-stage) and regression detection (One-stage). Regional detection algorithms such as R-CNN and FastR-CNN first extract features, then make predictions and screen candidate areas, and finally identify target categories and locations.

[0004] Although researchers have made a lot of optimizations to deep learning target detection algorithms, the following major problems still exist due to the complexity of the marine environment and the special needs of the experimental sea area: (1) Difficulty in detecting small targets: The network is not sensitive to small targets, details are lost after dimensionality reduction, the small number of small targets makes it difficult to learn generalized features, and small targets participate less in training. (2) Complex background and noise interference: Complex background, noise and cloud interference can easily lead to overfitting, weaken target features, and cause missed detection and false detection. (3) High complexity of the detection model: Poor real-time monitoring stability, long inference time and difficult to detect in real time due to environmental changes. The ship remote sensing image dataset has a small sample size and a single category, resulting in insufficient training, poor robustness and generalization.

[0005] Aiming at the problems of poor detection effect, weak anti-interference ability and slow detection speed of small ship targets, the present invention proposes a remote sensing image ship detection method based on SPD-YOLOv8. Based on YOLOv8, this method combines the SPD-Conv fine-grained convolution module and the CBAM attention mechanism module, which significantly improves the detection effect of low-resolution images and small targets. At the same time, the feature pyramid network (GFPN) is introduced to further improve the accuracy, real-time and response speed of the detection model by increasing the cross-scale connection between adjacent levels and the cross-layer connection under the same scale. In addition, this method is also specially optimized for the problems of uneven distribution of large, medium and small targets, limited sample size and single category in the self-made data set. By making full use of the multi-sample information in the data set, the model can more effectively learn and generalize ship targets of different sizes and types, thereby significantly improving the robustness and practicality of the detection algorithm. This comprehensive and innovative method not only effectively solves the technical challenges in ship detection, but also combines the time series information of AIS (Automatic Identification System) ship targets, and can perform comprehensive position, heading, speed and motion trajectory analysis of detected ships. By monitoring the position and status of ships in real time, this method can more accurately understand the laws and characteristics of ship activities, thereby achieving rapid detection and early warning of abnormal behaviors and illegal activities. This comprehensive method of multi-source data fusion and spatiotemporal information analysis provides strong technical support for ship detection and maritime safety management in the experimental waters. Summary of the invention

[0006] The purpose of the present invention is to provide a remote sensing image ship detection method based on SPD-YOLOv8 to solve the problems of poor detection effect of small ship targets, weak anti-interference ability and slow detection speed in the prior art.

[0007] The remote sensing image ship detection method based on SPD-YOLOv8 includes:

[0008] S1. Obtain the satellite remote sensing image ship dataset of the experimental sea area, crop the high-resolution satellite remote sensing image according to the scope of the experimental sea area, and perform radiation correction and geometric correction on the high-resolution satellite remote sensing image;

[0009] S2. Divide the acquired experimental sea area satellite remote sensing image ship dataset into a training set, a test set and a validation set;

[0010] S3. Perform image enhancement on the acquired satellite remote sensing image ship dataset of the experimental sea area to highlight the characteristics of the ship;

[0011] S4. Based on the YOLOv8 architecture, the SPD-YOLOv8 ship target detection model is constructed by combining the SPD-Conv fine-grained convolution module, the CBAM attention mechanism module and the feature pyramid network GFPN;

[0012] S5. Use the constructed SPD-YOLOv8 ship target detection model to perform ship target detection and obtain the ship detection result;

[0013] S6. Perform parameter tuning and result evaluation;

[0014] S7. Deploy the constructed SPD-YOLOv8 ship target detection model to the server;

[0015] S8. Combine the time series information of AIS ship targets to perform motion trajectory analysis, detect and warn abnormal behaviors and activities.

[0016] S1 includes selecting SAR satellites and optical remote sensing satellites, accessing the data providing platform corresponding to the satellites, applying for and downloading high-resolution SAR satellite images and optical remote sensing satellite images according to the scope of the experimental sea area, fusing the two images, and creating a comprehensive data set to utilize the advantages of the two satellites for ship target detection; decompressing the downloaded satellite images using a decompression tool, unpacking the multi-band image data, performing radiation correction on the multi-band image data, adjusting the brightness value of the image to reflect the real ground reflection characteristics in the image, and converting the original DN value into a reflectivity value; performing geometric correction on the multi-band image data, loading the radiation-corrected multi-band image data, applying geographic reference points GCPs to calibrate the geographic location of the image to ensure that the multi-band image data matches the actual geographic coordinates and is aligned with the map projection system; using the image processing tool ArcGIS to load the geometrically corrected multi-band image data, defining the clipping area as a polygonal or rectangular bounding box according to the geographic coordinates of the experimental sea area, and clipping out the area of ​​interest AOI.

[0017] S2 includes collecting and organizing the experimental sea area satellite remote sensing image ship dataset, ensuring that each sample has the annotation information of the ship target, and randomizing the entire experimental sea area satellite remote sensing image ship dataset to ensure the randomness and representativeness of the data; dividing the experimental sea area satellite remote sensing image ship dataset into training set, test set and validation set in the ratio of 7:1.5:1.5; verifying and adjusting the divided experimental sea area satellite remote sensing image ship dataset.

[0018] S3 includes preprocessing the satellite remote sensing image ship data set of the experimental sea area, including denoising and normalization; applying histogram equalization or adaptive histogram equalization to enhance the contrast of satellite images, histogram equalization enhances the global contrast of satellite images, and adaptive histogram equalization enhances the contrast while limiting the enhancement range; using edge detection algorithm to detect edge information in satellite images and highlight the outline of ships; converting satellite images to HSV color space, adjusting saturation and value, and adjusting the color balance and color saturation of optical remote sensing images.

[0019] SPD-YOLOv8 includes backbone network, connection part and task head;

[0020] The backbone network includes a 3×3 convolution kernel with a stride of 2, an alternating module, and a fast spatial pyramid layer. The alternating module includes four groups of fine-grained convolution modules and C2f modules connected in series. The four C2f modules contain 3, 6, 6, and 3 bottleneck models respectively. The pooling kernel size of the fast spatial pyramid layer is 5.

[0021] The connection part includes three serially connected upward processing models and three serially connected downward processing modules. The upward processing module includes an upsampling layer, a feature fusion layer and a C2f module. The downward processing module includes a C2f module, a fine-grained convolution module and a feature fusion layer. The fast spatial pyramid layer is connected to the first upsampling layer of the upward processing module. The last C2f module of the upward processing module is the first C2f module of the downward processing module. The downward processing module is finally connected to a C2f module with a channel number of 1024.

[0022] The task head includes four convolutional layers with sizes of 160×160×45, 80×80×45, 40×40×45, and 20×20×45, respectively. The four C2f modules of the downward processing module are connected to the four convolutional layers respectively.

[0023] The second C2f module of the backbone network is connected to the second feature fusion layer of the upward processing module, the third C2f module of the backbone network is connected to the first feature fusion layer of the upward processing module, the first C2f module of the upward processing module is connected to the second feature fusion layer of the downward processing module, the second C2f module of the upward processing module is connected to the first feature fusion layer of the downward processing module, and the fast spatial pyramid layer is also connected to a C2f module with 1024 channels.

[0024] The fine-grained convolution module includes a 3×3 convolution kernel with a stride of 2 and an SPD module.

[0025] S5 includes inputting the aerial photography or satellite remote sensing image to be detected, and after preprocessing and image enhancement, using the SPD-YOLOv8 ship target detection model to detect the target, and outputting the ship's location information, category prediction and confidence score;

[0026] S6 includes adjusting the model's hyperparameters and training strategies. Hyperparameters include learning rate, batch size, and training cycle; evaluating the generalization ability and stability of the model on different data sets through validation sets or cross-validation; quantifying the performance of the model using evaluation indicators, including accuracy, recall rate, F1 score, and mean average precision (mAP), and performing continuous iterations and improvements to improve the robustness and accuracy of the model.

[0027] S7 includes converting the trained model into the model format for inference and setting up the server environment to support the efficient operation and calling of the model; establishing an API interface or integrating it into the data processing pipeline, obtaining input from the image data stream, and outputting the detection results to the application or database to automatically and in real time monitor and manage ships.

[0028] S8 includes using the SPD-YOLOv8 ship target detection model to detect the position and type of ships from aerial or satellite images. The detected ship position is mapped to the geospatial coordinate system through the image pixel coordinates to obtain the coordinates of the ship in the geographic location. The ship time series information provided by the AIS system includes the position, speed, heading, and ship name of the ship. The ship target identified in each image is matched and associated with the AIS data through the ship's unique identifier or ship name. The movement trajectory of the ship is analyzed by analyzing and integrating image detection data and AIS time series data. Based on continuous detection results and AIS data, the actual position and movement trajectory of the ship are drawn, the starting point, end point, speed change and channel selection information of the ship are confirmed, and the ship's driving trajectory is analyzed. By analyzing the time and frequency of the ship's stay at a specific location, the ship's common stopover points or activity areas are identified, and the ship's operating mode and activity characteristics are analyzed. Based on historical trajectory data, the ship's selected routes and navigation paths are evaluated, and route planning is optimized to improve shipping efficiency and save fuel costs. Detect and track ships that have not turned on AIS or deliberately turned off AIS, and monitor activities in the sea area.

[0029] Compared with the prior art, the present invention has the following beneficial effects: it improves the detection effect of low-resolution images and small targets, improves the accuracy, real-time and response speed of the detection model, can more effectively learn and generalize ship targets of different sizes and types, and improves the robustness and practicality of the detection algorithm. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 It is an overall flow chart of the satellite remote sensing image ship detection method based on SPD-YOLOv8 in the example of the present invention;

[0031] Figure 2 Schematic diagram of the structure of the ship detection model based on SPD-YOLOv8 in the example of the present invention;

[0032] Figure 3 It is a schematic diagram of the effect of PANet feature fusion in the example of the present invention;

[0033] Figure 4 It is a schematic diagram of the effect of GFPN feature fusion in an example of the present invention;

[0034] Figure 5 Schematic diagram of the effect of the adaptive feature fusion process in the example of the present invention;

[0035] Figure 6 Schematic diagram of the structure of the convolution downsampling module in the example of the present invention;

[0036] Figure 7It is a structural schematic diagram of the improved satellite remote sensing image ship detection model based on SPD-YOLOv8 in the example of the present invention;

[0037] Figure 8 This is the structure diagram of the SPD-Conv fine-grained convolution module. DETAILED DESCRIPTION

[0038] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention is described clearly and completely below. Obviously, the described embodiments are part of the embodiments of the present invention, but not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0039] The remote sensing image ship detection method based on SPD-YOLOv8 includes:

[0040] S1. Obtain the satellite remote sensing image ship dataset of the experimental sea area, crop the high-resolution satellite remote sensing image according to the scope of the experimental sea area, and perform radiation correction and geometric correction on the high-resolution satellite remote sensing image;

[0041] S2. Divide the acquired experimental sea area satellite remote sensing image ship dataset into a training set, a test set and a validation set;

[0042] S3. Perform image enhancement on the acquired satellite remote sensing image ship dataset of the experimental sea area to highlight the characteristics of the ship;

[0043] S4. Based on the YOLOv8 architecture, the SPD-YOLOv8 ship target detection model is constructed by combining the SPD-Conv fine-grained convolution module, the CBAM attention mechanism module and the feature pyramid network GFPN;

[0044] S5. Use the constructed SPD-YOLOv8 ship target detection model to perform ship target detection and obtain the ship detection result;

[0045] S6. Perform parameter tuning and result evaluation;

[0046] S7. Deploy the constructed SPD-YOLOv8 ship target detection model to the server;

[0047] S8. Combine the time series information of AIS ship targets to perform motion trajectory analysis, detect and warn abnormal behaviors and activities.

[0048] S1 includes selecting SAR satellites and optical remote sensing satellites, accessing the data providing platform corresponding to the satellites, applying for and downloading high-resolution SAR satellite images and optical remote sensing satellite images according to the scope of the experimental sea area, fusing the two images, and creating a comprehensive data set to utilize the advantages of the two satellites for ship target detection; decompressing the downloaded satellite images using a decompression tool, unpacking the multi-band image data, performing radiation correction on the multi-band image data, adjusting the brightness value of the image to reflect the real ground reflection characteristics in the image, and converting the original DN value into a reflectivity value; performing geometric correction on the multi-band image data, loading the radiation-corrected multi-band image data, applying geographic reference points GCPs to calibrate the geographic location of the image to ensure that the multi-band image data matches the actual geographic coordinates and is aligned with the map projection system; using the image processing tool ArcGIS to load the geometrically corrected multi-band image data, defining the clipping area as a polygonal or rectangular bounding box according to the geographic coordinates of the experimental sea area, and clipping out the area of ​​interest AOI.

[0049] S2 includes collecting and organizing the experimental sea area satellite remote sensing image ship dataset, ensuring that each sample has the annotation information of the ship target, and randomizing the entire experimental sea area satellite remote sensing image ship dataset to ensure the randomness and representativeness of the data; dividing the experimental sea area satellite remote sensing image ship dataset into training set, test set and validation set in the ratio of 7:1.5:1.5; verifying and adjusting the divided experimental sea area satellite remote sensing image ship dataset.

[0050] S3 includes preprocessing the satellite remote sensing image ship data set of the experimental sea area, including denoising and normalization; applying histogram equalization or adaptive histogram equalization to enhance the contrast of satellite images, histogram equalization enhances the global contrast of satellite images, and adaptive histogram equalization enhances the contrast while limiting the enhancement range; using edge detection algorithm to detect edge information in satellite images and highlight the outline of ships; converting satellite images to HSV color space, adjusting saturation and value, and adjusting the color balance and color saturation of optical remote sensing images.

[0051] SPD-YOLOv8 includes backbone network, connection part and task head;

[0052] The backbone network includes a 3×3 convolution kernel with a stride of 2, an alternating module, and a fast spatial pyramid layer. The alternating module includes four groups of fine-grained convolution modules and C2f modules connected in series. The four C2f modules contain 3, 6, 6, and 3 bottleneck models respectively. The pooling kernel size of the fast spatial pyramid layer is 5.

[0053] The connection part includes three serially connected upward processing models and three serially connected downward processing modules. The upward processing module includes an upsampling layer, a feature fusion layer and a C2f module. The downward processing module includes a C2f module, a fine-grained convolution module and a feature fusion layer. The fast spatial pyramid layer is connected to the first upsampling layer of the upward processing module. The last C2f module of the upward processing module is the first C2f module of the downward processing module. The downward processing module is finally connected to a C2f module with a channel number of 1024.

[0054] The task head includes four convolutional layers with sizes of 160×160×45, 80×80×45, 40×40×45, and 20×20×45, respectively. The four C2f modules of the downward processing module are connected to the four convolutional layers respectively.

[0055] The second C2f module of the backbone network is connected to the second feature fusion layer of the upward processing module, the third C2f module of the backbone network is connected to the first feature fusion layer of the upward processing module, the first C2f module of the upward processing module is connected to the second feature fusion layer of the downward processing module, the second C2f module of the upward processing module is connected to the first feature fusion layer of the downward processing module, and the fast spatial pyramid layer is also connected to a C2f module with 1024 channels.

[0056] The fine-grained convolution module includes a 3×3 convolution kernel with a stride of 2 and an SPD module.

[0057] S5 includes inputting the aerial photography or satellite remote sensing image to be detected, and after preprocessing and image enhancement, using the SPD-YOLOv8 ship target detection model to detect the target, and outputting the ship's location information, category prediction and confidence score;

[0058] S6 includes adjusting the model's hyperparameters and training strategies. Hyperparameters include learning rate, batch size, and training cycle; evaluating the generalization ability and stability of the model on different data sets through validation sets or cross-validation; quantifying the performance of the model using evaluation indicators, including accuracy, recall rate, F1 score, and mean average precision (mAP), and performing continuous iterations and improvements to improve the robustness and accuracy of the model.

[0059] S7 includes converting the trained model into the model format for inference and setting up the server environment to support the efficient operation and calling of the model; establishing an API interface or integrating it into the data processing pipeline, obtaining input from the image data stream, and outputting the detection results to the application or database to automatically and in real time monitor and manage ships.

[0060] S8 includes using the SPD-YOLOv8 ship target detection model to detect the position and type of ships from aerial or satellite images. The detected ship position is mapped to the geospatial coordinate system through the image pixel coordinates to obtain the coordinates of the ship in the geographic location. The ship time series information provided by the AIS system includes the position, speed, heading, and ship name of the ship. The ship target identified in each image is matched and associated with the AIS data through the ship's unique identifier or ship name. The movement trajectory of the ship is analyzed by analyzing and integrating image detection data and AIS time series data. Based on continuous detection results and AIS data, the actual position and movement trajectory of the ship are drawn, the starting point, end point, speed change and channel selection information of the ship are confirmed, and the ship's driving trajectory is analyzed. By analyzing the time and frequency of the ship's stay at a specific location, the ship's common stopover points or activity areas are identified, and the ship's operating mode and activity characteristics are analyzed. Based on historical trajectory data, the ship's selected routes and navigation paths are evaluated, and route planning is optimized to improve shipping efficiency and save fuel costs. Detect and track ships that have not turned on AIS or deliberately turned off AIS, and monitor activities in the sea area.

[0061] In the present invention, Backbone is responsible for extracting features from the input image and converting the image into a feature representation with rich semantic information. Neck is an intermediate layer used to fuse features from Backbone to improve the performance of the model. Head is the last layer of the model, and its structure varies depending on the task. For example, in image classification tasks, a softmax classifier is usually used as the Head, while in object detection tasks, a bounding box regressor and a classifier may be used as the Head.

[0062] The optimization steps of the converted model include: in the training phase, the model may adopt a specific format for the purpose of facilitating algorithm optimization, storing intermediate states, etc. In the inference phase, factors such as operating efficiency, memory usage, and compatibility with specific inference frameworks or hardware need to be considered. The converted model format may be optimized in some ways, such as removing auxiliary structures used only for training, quantizing parameters to reduce storage requirements and speed up calculations, and adjusting the model's storage layout to better adapt to the memory access mode of specific hardware. This allows the model to process input data more quickly and give accurate detection results in actual applications.

[0063] The advantages of high-resolution SAR and optical remote sensing image data are that SAR images have all-weather and all-day imaging capabilities, can penetrate clouds and rain and fog, and can still provide clear surface information under low-light conditions, which is very suitable for sea area monitoring under complex meteorological conditions, while optical remote sensing images have higher spatial resolution and color information, can provide rich ground object details and realistic visual effects, and help to accurately identify and classify ship targets; according to the pre-calculated ratio, the training set, test set and validation set are proportionally divided from the randomized experimental sea area satellite remote sensing image ship data set, and it is necessary to ensure that each sub The number and quality of concentrated samples can represent the characteristics and distribution of the entire dataset. To verify and adjust the divided dataset, it is necessary to ensure that the sample ratio and data quality of each subset meet expectations, providing a reliable foundation for subsequent model training and evaluation based on this dataset. The feature pyramid network GFPN is introduced to establish cross-scale and cross-level connections between different levels, integrate multi-scale semantic information, improve the model's detection ability for multi-scale targets, and enhance the fusion and utilization of features of different scales. The CBAM attention mechanism module is integrated to improve the representation ability and detection accuracy of feature maps through channel and spatial attention mechanisms.

[0064] There are often a large number of similar objects in the area of ​​remote sensing scenes, and the background contains a lot of noise information, which weakens the target features and easily leads to missed detections and false detections. Semantic segmentation is combined with target detection, dilated convolution to increase the receptive field, and channel attention mechanism to weaken background information. The purpose of denoising is to reduce noise interference in the image. Common methods include Gaussian filtering or median filtering. Normalization can adjust the dynamic range of the image to ensure that subsequent processing steps can be better applied to the entire image; the Canny algorithm detects edges by finding the strongest grayscale changes in the image, with high accuracy and good noise resistance. In addition to edge detection, Laplacian filtering is also often used to enhance image details. Laplacian filtering can extract high-frequency information in the image, enhance the details and texture in the image, and help to more clearly display the structural characteristics of the ship; in the parameter tuning and result evaluation stage, the purpose of carefully adjusting the model's hyperparameters and training strategies is to optimize detection accuracy and performance.

[0065] The idea of ​​the present invention is to use remote sensing data from a variety of different sources, such as high-resolution optical images and synthetic aperture radar (SAR) images for fusion processing to improve the accuracy and robustness of ship target detection. In view of the characteristics and needs of the experimental sea area, the present invention designs an efficient and real-time ship target detection algorithm, which adopts YOLOv8 combined with SPD-Conv fine-grained convolution module and CBAM attention mechanism module to improve the detection effect of low-resolution images and small targets. The feature pyramid network (GFPN) is introduced into the model to increase the cross-scale connection between adjacent levels and the cross-layer connection under the same scale, so that the ship target detection model has higher accuracy, real-time and response speed. The present invention also combines the timing information of AIS ship targets to analyze the location information, heading, speed and motion trajectory of ship targets, and monitors the position and status of ships in real time to better understand the laws and characteristics of ship activities, and realize the detection and early warning of abnormal behavior and illegal activities.

[0066] Figure 1 The overall flow chart of the satellite remote sensing image ship detection method based on SPD-YOLOv8 in the example of the present invention is shown, including obtaining data sets, cropping and correcting images, dividing data sets, image enhancement, building SPD-YOLOv8 models, ship target detection, parameter tuning and result evaluation, model deployment, motion trajectory analysis, abnormal behavior and illegal activity detection and early warning. The main framework of the YOLOv8 algorithm is an advanced model in the field of target detection, with efficient real-time detection capabilities and high accuracy. The YOLOv8 algorithm consists of multiple modules, each of which is responsible for different functions, including feature extraction, feature fusion and target detection. The specific structure of the YOLOv8 algorithm is as follows: Figure 2As shown in the figure, the input end receives the input image, the Backbone network is the backbone network used to extract features, and the Neck network is placed between the Backbone and the Head to further improve the diversity and robustness of the features; the Head network is used to predict the location and category of the target, and the Bottleneck is the bottleneck, which usually refers to the different dimensions of the network input and output data. Usually, the output dimension is much smaller than the input dimension. The output layer outputs the bounding box and category of the detected target. The Conv module is a standard convolution module. The Conv module consists of a Conv2d, a BatchNorm2d, and an activation function. Among them, Conv2d is an important operation in convolutional neural networks, which performs convolution on a two-dimensional image to extract image features. BatchNorm2d is a function in PyTorch that batch normalizes the input 4D tensor, which helps stabilize the training process and improve model performance. k (kernel) is the number of filters or kernels, representing the depth of the output volume. Each filter is responsible for detecting different features in the input. s (stride) is the stride, which refers to the step length of the filter / kernel sliding on the input. A larger stride will reduce the spatial dimension of the output volume. p (padding) is padding, which refers to the additional zero border added on each side of the input, which helps to maintain spatial information and can be used to control the spatial dimension of the output volume. The C2f (Channel-to-Pixel) module is an upsampling module and a key module for feature fusion. It adopts the "Concat" feature fusion method, that is, the feature maps from different levels are spliced ​​according to the channel dimension, stacked to form a deeper feature map, thereby retaining more spatial information and semantic information. The Concat module represents the connection operation. It is used to splice multiple feature maps together in the channel dimension. The feature map after the attention mechanism is calculated and passed as a parameter to the Concat layer to achieve the splicing of feature maps. The CBS module (Cross-Stage-Partial-connections) consists of cross-stage connections and partial connections. Cross-stage connections promote information flow and improve model accuracy, while partial connections reduce the amount of calculation and accelerate model training. Split refers to the training of instance segmentation. Instance segmentation is a computer vision task that aims to assign each pixel in an image to the object instance to which it belongs and segment each object instance. The Botleneck module is a commonly used convolution block, usually composed of 1x1 convolution, 3x3 convolution and 1x1 convolution. The Botleneck module can process inputs with a large number of channels with a smaller number of parameters, reduce the number of parameters by using 1x1 convolution layers, and reduce the amount of calculation in deep networks. SiLU is a (Sigmoid-weighted Linear Unit) activation function that is calculated by multiplying the input value with the result of the input value processed by the sigmoid function, that is.It has the advantages of no upper bound to avoid overfitting, lower bound to produce regularization effect, smoothness and easy training, and non-monotonicity when the input value is less than zero, which is an important difference from activation functions such as ReLU. Detect module The detection block is responsible for detecting objects. Anchor-free detection reduces the number of box predictions and speeds up the complex post-processing steps of screening candidate detection results after inference. The detection block contains two tracks. The first track is for bounding box prediction and the second track is for category prediction. Both tracks contain two convolution blocks followed by a separate Conv2d layer, which gives the bounding box loss and category loss respectively. The MaxPool2d module is a pooling layer used to downsample the spatial dimension of the input volume, reduce the computational complexity of the network and extract the main features. Max pooling is a specific pooling operation that retains only the maximum value for each region of the input tensor and discards the other values. The Conv2d module is a mathematical operation that involves sliding a small matrix (called a kernel or filter) over the input data, performing element-wise multiplication, and summing the results to generate a feature map. The "2D" in Conv2d means that the convolution is applied to two spatial dimensions, usually height and width. The BatchNorm2d module is batch normalization, a technique used in deep neural networks to improve training stability and convergence speed. In convolutional neural networks (CNNs), the BatchNorm2d layer specifically batch normalizes 2D inputs, usually the output of convolutional layers. It does this by normalizing the features in each mini-batch of data so that the mean of each feature in the mini-batch is close to 0 and the variance is close to 1, ensuring that the data passing through the network is not too large or too small, which helps prevent problems during training. The SPPF module (Spatial PyramidPooling-Fast) is a fast block designed to efficiently capture multi-scale information, and it uses a simplified version of spatial pyramid pooling. This block allows the network to handle features of different scales, which is particularly useful in object detection tasks because objects may appear at different sizes in an image.

[0067] The original YOLOv8 network uses the path aggregation network structure PANet to fuse features, which to some extent alleviates the loss of feature information caused by the network being too deep. Figure 3 , the network structure only considers the influence of the same scale between adjacent levels and adjacent scales between the same level, resulting in more information loss in the feature fusion process, especially the features of small targets. The introduction of the generalized feature pyramid network (GFPN) increases the cross-scale connections between adjacent levels and the cross-layer connections at the same scale, so that the deep and shallow layers of the network, large and small scales of information are fully exchanged, which is conducive to the transmission of small target feature information.

[0068] like Figure 4As shown in Figure 1, the improved GFPN adds cross-scale connections between adjacent layers and cross-layer connections between the same scales based on PANet. In the process of feature fusion, the size of the feature maps must be consistent. However, in the process of cross-scale fusion between adjacent layers, the sizes of the feature maps of each scale are inconsistent, and scale adaptation is required. Adaptive feature fusion is shown in Figure 1. Figure 5 As shown in the figure, it includes P2 layer, P3 layer, P4 layer and P5 layer. The large-size feature map of P3 layer must be downsampled through maximum pooling to match the size of the feature map of P4 layer. The small-size feature map of P5 layer can be upsampled through bilinear interpolation, and the cross-scale connection of adjacent layers can be met through adaptive feature fusion to enhance the transmission of information.

[0069] There are design flaws in the CNN design system in YOLOv8, namely the use of convolution strides or pooling layers, which can lead to fine-grained information loss or inefficient feature learning. The current model design suffers from the loss of fine-grained information when processing small objects and low-resolution images, and cannot fully learn the features of small objects. Fine-grained convolution (SPD-Conv) is introduced to replace the convolution downsampling module with a stride of 2 in the YOLOv8 model to achieve fine-grained learning features and prevent small feature loss. The convolution downsampling module is as follows: Figure 6 As shown in the figure, the BatchNorm2d function plays an important role in deep neural network training. Normalizing the data before model training can make the data distribution consistent. Deep neural network training usually uses batches rather than all data. Different batches have different distributions, which will cause internal covariate shift problems. That is, the change in data distribution during training makes it difficult for the next layer of network to learn. The BatchNorm2d function forcibly pulls the data back to a normal distribution with a mean of 0 and a variance of 1, making the data distribution consistent, making it easier for the model to learn features, avoiding gradient disappearance, and improving the stability and efficiency of training.

[0070] SPD-Conv consists of a spatial-to-depth (SPD) layer and a non-strided convolutional layer, which will be described in detail in this invention. SPD downsamples the feature maps within and throughout the CNN. In general, given any (original) feature map X, the submaps consist of all feature maps to form a feature map X, and each submap downsamples X by a scaling factor. Consider an arbitrary intermediate feature map X of size S×S×C1, slice the sequence of subfeature maps as follows: The diagram of SPD-Conv when scale=2 obtains four submaps and downsamples X by a factor of 2. These subfeature maps are concatenated along the channel dimension to obtain a feature map X' whose spatial dimension is reduced by a scaling factor and whose channel dimension is increased by a scaling factor of 2.

[0071] Based on the improvement of the above YOLOv8 model, an improved satellite remote sensing image ship detection model based on SPD-YOLOv8 is proposed for the detection of ships in satellite remote sensing images. The model structure is as follows: Figure 7 As shown in the figure, SPD-YOLOv8 includes a backbone network, a connection part and a task head; the backbone network includes a 3×3 convolution kernel with a step size of 2, an alternating module and a fast spatial pyramid layer, wherein the alternating module includes 4 groups of fine-grained convolution modules and C2f modules connected in series, and the 4 C2f modules contain 3, 6, 6, and 3 bottleneck models respectively, and the pooling kernel size of the fast spatial pyramid layer is 5; the connection part includes three series of upward processing models and three series of downward processing modules, the upward processing module includes an upsampling layer, a feature fusion layer and a C2f module, and the downward processing The processing module includes a C2f module, a fine-grained convolution module and a feature fusion layer. The fast spatial pyramid layer is connected to the first upsampling layer of the upward processing module. The last C2f module of the upward processing module is the first C2f module of the downward processing module. The downward processing module is finally connected to a C2f module with a channel number of 1024. The task head includes four convolutional layers with sizes of 160×160×45, 80×80×45, 40×40×45, and 20×20×45. The four C2f modules of the downward processing module are connected to four convolutional layers respectively. The second C2f module of the backbone network is connected to the second feature fusion layer of the upward processing module, the third C2f module of the backbone network is connected to the first feature fusion layer of the upward processing module, the first C2f module of the upward processing module is connected to the second feature fusion layer of the downward processing module, the second C2f module of the upward processing module is connected to the first feature fusion layer of the downward processing module, and the fast spatial pyramid layer is also connected to a C2f module with a channel number of 1024.

[0072] The SPD-Conv fine-grained convolution module structure is as follows Figure 8 As shown, the fine-grained convolution module includes a 3×3 convolution kernel with a stride of 2 and an SPD (space to depth) module.

[0073] The constructed and trained SPD-YOLOv8 ship target detection model is used to detect actual ship images. The image data containing ship targets are collected and preprocessed, and normalized to the input size required by the model. These image data are input into the SPD-YOLOv8 ship target detection model, and the model quickly identifies and locates the ship targets in the image through its efficient feature extraction and detection modules. The introduction of the SPD module and the CBAM attention mechanism enables the model to more accurately identify small target ships in low-resolution images, especially in complex marine environments. Finally, the model outputs the bounding box, confidence, and category label of the ship target, and visualizes these detection results for further analysis and evaluation. After obtaining the preliminary ship detection results, the model parameters are tuned and the results are evaluated. First, appropriate evaluation indicators such as precision, recall, F1 score, and mean average precision (mAP) are selected to comprehensively evaluate the detection performance of the model. According to the evaluation results, the model's hyperparameters, including learning rate, batch size, number of network layers, etc., are adjusted to optimize the model performance. In addition, cross-validation can be performed to use different training and validation sets to ensure the generalization ability of the model. Through continuous iterative tuning and evaluation, the optimal model parameter configuration is found to improve the accuracy and robustness of ship target detection. Finally, the tuned model is tested to ensure its reliability and stability in actual application scenarios.

[0074] After the model has been fully trained and tuned, deploy the SPD-YOLOv8 ship target detection model to the server to achieve real-time detection capabilities. Select a suitable server environment and ensure that it has sufficient computing power and storage space to support the efficient operation of the model. Upload the model file and related dependencies to the server, and perform environment configuration and dependency installation. Next, write and configure the model service interface, and open the model detection function to the outside world in the form of web services or APIs. Ensure the stable operation of the model on the server through regular maintenance and monitoring, and update and optimize as needed to adapt to new detection requirements and environmental changes.

[0075] Finally, the ship detection results are combined with the time series information of the AIS (Automatic Identification System) ship target to perform motion trajectory analysis. AIS provides real-time position information, heading, speed and other data of the ship. By combining it with the SPD-YOLOv8 detection results, a ship motion trajectory model is constructed. By analyzing these trajectory data, normal and abnormal ship activity patterns can be identified. For example, through abnormal behaviors such as trajectory deviation from the scheduled route, frequent changes in speed and direction, illegal activities such as illegal fishing, smuggling and intrusion into sensitive waters can be detected and warned. Combined with the real-time monitoring system, early warning information is sent to relevant departments in a timely manner to ensure the safety and order of maritime activities. This comprehensive analysis method not only improves the accuracy of ship target detection, but also enhances the ability of maritime safety management.

[0076] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some or all of the technical features may be replaced by equivalents, and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A remote sensing image ship detection method based on SPD-YOLOv8, characterized in that: include: S1. Obtain the satellite remote sensing image ship dataset of the experimental sea area, crop the high-resolution satellite remote sensing image according to the scope of the experimental sea area, and perform radiation correction and geometric correction on the high-resolution satellite remote sensing image; S2. Divide the acquired experimental sea area satellite remote sensing image ship dataset into a training set, a test set and a validation set; S3. Perform image enhancement on the acquired satellite remote sensing image ship dataset of the experimental sea area to highlight the characteristics of the ship; S4. Based on the YOLOv8 architecture, the SPD-YOLOv8 ship target detection model is constructed by combining the SPD-Conv fine-grained convolution module, the CBAM attention mechanism module and the feature pyramid network GFPN; S5. Use the constructed SPD-YOLOv8 ship target detection model to perform ship target detection and obtain the ship detection result; S6. Perform parameter tuning and result evaluation; S7. Deploy the constructed SPD-YOLOv8 ship target detection model to the server; S8. Combine the time series information of AIS ship targets to perform motion trajectory analysis, detect and warn abnormal behaviors and activities.

2. The remote sensing image ship detection method based on SPD-YOLOv8 according to claim 1, characterized in that: S1 includes selecting SAR satellites and optical remote sensing satellites, accessing the data providing platform corresponding to the satellites, applying for and downloading high-resolution SAR satellite images and optical remote sensing satellite images according to the scope of the experimental sea area, fusing the two images, and creating a comprehensive data set to utilize the advantages of the two satellites for ship target detection; decompressing the downloaded satellite images using a decompression tool, unpacking the multi-band image data, performing radiation correction on the multi-band image data, adjusting the brightness value of the image to reflect the real ground reflection characteristics in the image, and converting the original DN value into a reflectivity value; performing geometric correction on the multi-band image data, loading the radiation-corrected multi-band image data, applying geographic reference points GCPs to calibrate the geographic location of the image to ensure that the multi-band image data matches the actual geographic coordinates and is aligned with the map projection system; using the image processing tool ArcGIS to load the geometrically corrected multi-band image data, defining the clipping area as a polygonal or rectangular bounding box according to the geographic coordinates of the experimental sea area, and clipping out the area of ​​interest AOI.

3. The remote sensing image ship detection method based on SPD-YOLOv8 according to claim 2, characterized in that: S2 includes collecting and organizing the satellite remote sensing image ship dataset of the experimental sea area, ensuring that each sample has the annotation information of the ship target, and performing random processing on the satellite remote sensing image ship dataset of the entire experimental sea area to ensure the randomness and representativeness of the data; The experimental sea area satellite remote sensing image ship dataset is divided into training set, test set and validation set in the ratio of 7:1.5:1.5; the divided experimental sea area satellite remote sensing image ship dataset is verified and adjusted.

4. The remote sensing image ship detection method based on SPD-YOLOv8 according to claim 3, characterized in that: S3 includes preprocessing the satellite remote sensing image ship data set of the experimental sea area, including denoising and normalization; applying histogram equalization or adaptive histogram equalization to enhance the contrast of the satellite image, histogram equalization enhances the global contrast of the satellite image, and adaptive histogram equalization enhances the contrast while limiting the enhancement range; The edge detection algorithm is used to detect the edge information in the satellite image and highlight the outline of the ship; the satellite image is converted to the HSV color space, the saturation and value are adjusted, and the color balance and color saturation of the optical remote sensing image are adjusted.

5. The remote sensing image ship detection method based on SPD-YOLOv8 according to claim 4, characterized in that: SPD-YOLOv8 includes backbone network, connection part and task head; The backbone network includes a 3×3 convolution kernel with a stride of 2, an alternating module, and a fast spatial pyramid layer. The alternating module includes four groups of fine-grained convolution modules and C2f modules connected in series. The four C2f modules contain 3, 6, 6, and 3 bottleneck models respectively. The pooling kernel size of the fast spatial pyramid layer is 5. The connection part includes three serially connected upward processing models and three serially connected downward processing modules. The upward processing module includes an upsampling layer, a feature fusion layer and a C2f module. The downward processing module includes a C2f module, a fine-grained convolution module and a feature fusion layer. The fast spatial pyramid layer is connected to the first upsampling layer of the upward processing module. The last C2f module of the upward processing module is the first C2f module of the downward processing module. The downward processing module is finally connected to a C2f module with a channel number of 1024. The task head includes four convolutional layers with sizes of 160×160×45, 80×80×45, 40×40×45, and 20×20×45, respectively. The four C2f modules of the downward processing module are connected to the four convolutional layers respectively.

6. The remote sensing image ship detection method based on SPD-YOLOv8 according to claim 5, characterized in that: The second C2f module of the backbone network is connected to the second feature fusion layer of the upward processing module, the third C2f module of the backbone network is connected to the first feature fusion layer of the upward processing module, the first C2f module of the upward processing module is connected to the second feature fusion layer of the downward processing module, the second C2f module of the upward processing module is connected to the first feature fusion layer of the downward processing module, and the fast spatial pyramid layer is also connected to a C2f module with 1024 channels.

7. The remote sensing image ship detection method based on SPD-YOLOv8 according to claim 6, characterized in that: The fine-grained convolution module includes a 3×3 convolution kernel with a stride of 2 and an SPD module.

8. The remote sensing image ship detection method based on SPD-YOLOv8 according to claim 7, characterized in that: S5 includes inputting the aerial photography or satellite remote sensing image to be detected, and after preprocessing and image enhancement, using the SPD-YOLOv8 ship target detection model to detect the target, and outputting the ship's location information, category prediction and confidence score; S6 includes adjusting the model's hyperparameters and training strategies. Hyperparameters include learning rate, batch size, and training cycle. The generalization ability and stability of the model on different data sets are evaluated through validation sets or cross-validation. The performance of the model is quantified using evaluation metrics including precision, recall, F1 score, and mean average precision (mAP), and continuous iteration and improvement are performed to improve the robustness and accuracy of the model.

9. The remote sensing image ship detection method based on SPD-YOLOv8 according to claim 8, characterized in that: S7 includes converting the trained model into the model format for inference and setting up the server environment to support the efficient operation and calling of the model; establishing an API interface or integrating it into the data processing pipeline, obtaining input from the image data stream, and outputting the detection results to the application or database to automatically and in real time monitor and manage ships.

10. The remote sensing image ship detection method based on SPD-YOLOv8 according to claim 9, characterized in that: S8 includes using the SPD-YOLOv8 ship target detection model to detect the position and type of the ship from aerial or satellite images. The detected ship position is mapped to the geospatial coordinate system through the image pixel coordinates to obtain the coordinates of the ship in the geographic location. The ship time series information provided by the AIS system includes the position, speed, heading, and name of the ship. The unique identification or name of the ship is used to match and associate the ship target identified in each image with the AIS data. Analyze the movement trajectory of ships by analyzing and integrating image detection data and AIS time series data; Based on continuous detection results and AIS data, the actual position and movement trajectory of the ship are mapped out, the starting point, end point, speed change and channel selection information of the ship are confirmed, and the ship's trajectory is analyzed; by analyzing the time and frequency of the ship's stay at a specific location, the ship's common stop points or activity areas are identified, and the ship's operating mode and activity characteristics are analyzed; based on historical trajectory data, the routes and navigation paths selected by the ship are evaluated, and route planning is optimized to improve shipping efficiency and save fuel costs; detect and track ships that have not turned on AIS or have deliberately turned off AIS, and monitor activities in the sea area.

Images