Ship abnormal transfer monitoring method and system based on SAR (Synthetic Aperture Radar)

Abstract

The invention discloses an SAR-based ship abnormal transfer monitoring method and system, and relates to the technical field of smart maritime affairs, and the method comprises the steps: carrying out the preprocessing of an SAR image, obtaining an SAR image ship sample, and dividing the SAR image ship sample into a training set and a test set; constructing a dynamic directional convolutional block to replace a part of convolutional blocks in the Backbone network to obtain an improved YOLOv8 neural network; inputting the training set into a network for training, and using the test set to optimize the trained network to obtain an intelligent ship identification model; the image coordinates of the ship and the ship type are analyzed in combination with AIS data, the coordinates of the abnormal ship and the ship type are obtained, and monitoring of the abnormal ship is achieved; the system comprises a sample library construction module, an intelligent ship identification model training module, an intelligent ship identification model interpretation module, a clustering analysis module and an abnormal transfer early warning module. According to the invention, intelligent early warning can be carried out on abnormal ships, multi-scene monitoring is adapted, early warning can be triggered when suspected abnormal transfer behaviors are detected, and accurate decision support is provided for maritime law enforcement.

Description

Technical Field

[0001] The present invention relates to the field of smart maritime technology, and in particular to a SAR-based ship abnormal transshipment monitoring method and system. Background Art

[0002] Existing technologies primarily rely on AIS data analysis to monitor illegal transshipment, identifying features such as abnormal ship trajectories, interrupted or overlapping AIS signals, and inferring possible illegal activity. However, this approach has significant limitations: AIS signals can be easily shut down, forged, or tampered with, making it impossible to capture or accurately assess key ship behaviors in real time. Furthermore, relying solely on AIS data suffers from inherent limitations in spatial resolution and target detail extraction, failing to provide key visual information such as the relative position, type, and structural characteristics of ships. Therefore, in scenarios such as multiple ships docking side by side or cargo transshipment, AIS data often fails to adequately support the accurate identification and evidence extraction of abnormal activity. Furthermore, existing methods are largely ineffective in areas where AIS data is missing or unreliable, making it difficult to meet the demands of intelligent monitoring and precise crackdowns on illegal activity in high-risk waters. Summary of the Invention

[0003] The present invention provides a SAR-based abnormal transshipment monitoring method and system to overcome the technical problem that existing monitoring technologies for illegal transshipment mainly rely on AIS data analysis. When AIS signals appear or there are problems with AIS data, it is impossible to infer possible illegal activities based on AIS data, making it difficult to meet the needs of intelligent monitoring and precise crackdown on illegal activities in high-risk waters.

[0004] In order to achieve the above object, the technical solution of the present invention is:

[0005] A SAR-based ship abnormal transshipment monitoring method comprising:

[0006] S1: Acquire SAR images, preprocess the SAR images to obtain SAR image ship samples, and divide the SAR image ship samples into a training set and a test set;

[0007] S2: Introducing the YOLOv8 neural network, which includes a Backbone network, a Neck network, and a Head network, constructing a dynamic directional convolution block, and using the dynamic directional convolution block to replace some convolution blocks in the Backbone network to obtain an improved Backbone network, and then obtaining an improved YOLOv8 neural network;

[0008] S3: Inputting the training set into the improved YOLOv8 neural network for training, and using the test set to optimize the trained improved YOLOv8 neural network to obtain an intelligent ship recognition model, wherein the intelligent ship recognition model is used to recognize ship samples in SAR images and obtain image coordinates and ship types of the ships;

[0009] S4: Collect AIS data, analyze the image coordinates and ship type of the ship in combination with the AIS data, identify abnormal ships based on the analysis results, obtain the coordinates and ship type of the abnormal ships, and send the coordinates and ship type of the abnormal ships to the regulatory authorities to realize the monitoring of abnormal ships.

[0010] Furthermore, the improved Backbone network includes a first convolution block, a first dynamic directional convolution block, a first C2f module, a second dynamic directional convolution block, a second C2f module, a third dynamic directional convolution block, a third C2f module, a second convolution block and an SPPF module connected in sequence.

[0011] Furthermore, the first / second dynamic directional convolution block each includes a first branch and a second branch;

[0012] The first branch includes an improved directional attention mechanism and a multi-scale spatial attention mechanism connected in sequence, wherein the multi-scale spatial attention mechanism includes three layers of parallel first, second, and third RELU convolution modules and a softmax activation function;

[0013] The second branch includes a first convolutional layer;

[0014] The improved directional attention mechanism includes a two-dimensional adaptive average pooling layer branch and a double pooling convolution branch;

[0015] The two-dimensional adaptive average pooling layer branch includes a two-dimensional adaptive average pooling layer, a fourth RELU convolution module and a first Sigmod convolution module connected in sequence;

[0016] The dual-pooling convolution branch includes a parallel maximum pooling layer, an average pooling layer, and a second Sigmod convolution module connected to the maximum pooling layer and the average pooling layer at the same time.

[0017] Furthermore, the training set is input into the improved YOLOv8 neural network for training, including:

[0018] S31, inputting the training set into the first convolution block in the improved Backbone network for feature extraction to obtain feature input parameters;

[0019] S32: Inputting the feature input parameters into the first dynamic directional convolution block for feature segmentation, specifically comprising the following steps:

[0020] S321: Divide the feature input parameters equally along the channel dimension, as shown in formula (1),

[0021]

[0022] Among them, W fused represents the feature input parameter, and For E fused The feature input parameters of the first and second branches after equal division, It is a channel splicing operation;

[0023] S322: The data are respectively input into the three parallel 2D adaptive average pooling layer, the maximum pooling layer and the average pooling layer in the improved directional attention mechanism for statistical extraction, as shown in formulas (2)-(4).

[0024]

[0025] Among them, MaxPool() is the maximum pooling calculation, AvgPool() is the average pooling calculation, AdaptiveAvgPool2d() is the adaptive average pooling calculation, F max 、F avg and F Adaptavg They are the maximum pooling feature, average pooling feature and adaptive average pooling feature output after calculation respectively;

[0026] S323: F Adaptavg Input to the fourth RELU convolution module and the first Sigmod convolution module for feature extraction, as shown in formula (5),

[0027] A l =Sigmod(Conv 1×1 (ReLU(Conv 1×1 (F Adaptavg )))) (5)

[0028] A l Represents the weighted features of the output of the two-dimensional adaptive average pooling layer branch;

[0029] F max and F avg Input to the second Sigmod convolution module for convolution, as shown in formula (6),

[0030]

[0031] A r represents the weighted features output by the double-pooled convolution branch, Represents feature fusion operation;

[0032] Calculate A l and A r The tensor product of is shown in formula (7),

[0033]

[0034] Among them, A c To fusion features, stands for tensor product;

[0035] S324: A c Input to the multi-scale spatial attention mechanism for convolution extraction and weight fusion, as shown in formulas (8) and (9),

[0036] F i =ReLU(Conv n*n (A c )) (8)

[0037]

[0038] Among them, F i Represents the multi-scale spatial direction features extracted by the first, second, and third RELU convolution modules, i = 1, 2, 3, n represents the convolution kernel, ReLU() is the ReLU function calculation for the feature, Conv 1×1 In order to compress the multi-scale spatial directional features into a single channel operation, is the modulation coefficient to prevent gradient saturation, A s It is a multi-scale spatial attention feature that identifies the salient area of ​​the ship direction;

[0039] S325: Input to the first convolutional layer for feature extraction, as shown in formula (10),

[0040]

[0041] Among them, A is the direction sensitive weight value, BN() is the normalization layer calculation, X c is the normalized feature output after extraction by the second branch, ⊙ represents the Hadamard product operation;

[0042] S326: Fuse the multi-scale spatial attention features extracted by the first branch with the normalized features extracted by the second branch to obtain the spatial attention fusion feature, as shown in formula (11).

[0043] X s =A s +X c (11)

[0044] X s It is the spatial attention fusion feature;

[0045] S33, X s Input the image into the first C2f module for convolution, and output the multi-level features of the image through convolution within the first C2f module. The multi-level features are input into the second dynamic directional convolution block for feature extraction. The output result of the second dynamic directional convolution block is input into the second C2f module for convolution. The output result of the second C2f module is input into the third dynamic directional convolution block for convolution. The output result of the third dynamic directional convolution block is input into the third C2f module for convolution. The output result is then input into the second convolution module. The output result is input into the SPPF module to obtain the extracted features of the backbone network.

[0046] S34. Input the features output by the SPPF module into the Neck network to extract deep and shallow features, and then input the features extracted by the Neck network into the Head network to obtain the image coordinates and ship type of the ship.

[0047] Furthermore, the trained improved YOLOv8 neural network was optimized using the test set to obtain an intelligent ship recognition model, including:

[0048] Construct the loss function as shown in formula (12),

[0049]

[0050] in, is the total loss function, is the classification loss function, is the regression loss function, is the target loss function, λ1, λ2, and λ3 are the weight coefficients of each loss term respectively;

[0051] The test set is input into the trained improved YOLOv8 neural network, and the model parameters are updated using the stochastic gradient descent method and back propagation mechanism so that the loss function reaches the minimum value, and the intelligent ship recognition model is obtained.

[0052] Furthermore, AIS data is collected and analyzed based on the image coordinates and ship types of the ships. Based on the analysis results, abnormal ships are identified and their coordinates and types are obtained. These coordinates and types are sent to regulatory authorities to monitor abnormal ships, including:

[0053] S41. Calculating the spatial relationship between ships in each SAR image ship sample based on the image coordinates of the ships, specifically including:

[0054] Set a distance threshold d. Calculate the distance D between the bow and stern lines of any two ships in the ship identification results of the current SAR image ship sample based on the ship image coordinates. Determine whether D is less than the set distance threshold d. If so, the two ships with a distance D less than the threshold d are called a ship pair, denoted as the set {Si}, where i = 1, 2, ..., n. If the distance D is greater than the set distance threshold d, there is no association between the two ships, and no analysis is performed.

[0055] S42, matching the ship pair obtained in S41 with the AIS data, specifically including:

[0056] S421: Collect AIS data and save it to the AIS database;

[0057] S422: Convert the image coordinates of the vessel in {Si} to longitude and latitude coordinates, and retrieve AIS data for the corresponding area from the AIS database to determine whether there are sufficient AIS points. If there are sufficient points, proceed to S43 to determine whether there is AIS information fraud. If there are not sufficient AIS points, it is determined to be an abnormal transshipment, and the vessel in {Si} is marked as an abnormal vessel. Sufficient AIS points means that the number of different MMSI numbers is greater than or equal to r.

[0058] S43. Obtain the length, width, and ship type of the vessel in {Si}, and determine whether they are consistent with the length, width, and ship type of the vessel recorded in the AIS information. If they are consistent, there is no AIS information fraud, and the vessel in {Si} is a normal vessel. If they are inconsistent, there is AIS information fraud, and the vessel in {Si} is an abnormal vessel. Output the latitude and longitude coordinates and ship type of the vessels in the set {Si}, mark them as abnormal transshipments, and send the coordinates and ship type of the abnormal vessel to the regulatory authorities to monitor abnormal vessels.

[0059] Furthermore, the corresponding area is the union area in the set {Si} with the center points of the two ships as the center and the radius set.

[0060] Furthermore, the length, width and ship type of the ship in {Si} are obtained, and it is determined whether they are consistent with the length, width and ship type recorded in the AIS information. The judgment basis is shown in formula (13):

[0061]

[0062] in, is the indicator function, J is 1 if there is AIS information fraud, J is 0 if there is no AIS information fraud, L I 、W I 、T I are the length, width and type of the ship respectively, LA 、W A 、T A are the length, width and type of the ship in the AIS data, and r represents the number of different MMSI numbers.

[0063] Based on the same inventive concept, a SAR-based ship abnormal transshipment monitoring system is also proposed, which includes: a sample library construction module, an intelligent ship identification model training module, an intelligent ship identification model interpretation module, a clustering analysis module, and an abnormal transshipment warning module;

[0064] The sample library construction module is used to obtain a SAR image, divide it into several sub-images according to a certain pixel size and overlap, and mark the position and type of the ship in each sub-image using a rotating box method to form a SAR image ship sample;

[0065] The intelligent ship recognition model training module is used to divide the SAR image ship samples obtained by the sample library construction module into a training set and a test set, construct an improved YOLOv8 neural network, input the training set into the improved YOLOv8 neural network for training, and optimize the improved YOLOv8 neural network with the test set to obtain an intelligent ship recognition model;

[0066] The intelligent ship recognition model interpretation module is used to interpret the ships in the ship samples to be identified in the SAR image according to the intelligent ship recognition model, and obtain the image coordinates and ship types of the ships;

[0067] The cluster analysis module is used to analyze the ships in each SAR image ship sample and output the latitude and longitude coordinates and ship types of all abnormal transshipment ships;

[0068] The abnormal transshipment warning module is used to send the latitude and longitude coordinates and ship type information of abnormal transshipment ships within the monitoring range.

[0069] Beneficial effects: The present invention provides a method for monitoring abnormal ship transshipment based on SAR, which has the following advantages:

[0070] (1) The present invention designs a dynamic directional convolution block and an improved directional attention mechanism to form an improved YOLOv8 neural network, and trains it to obtain an intelligent ship recognition model. This model reduces background interference in the image, allowing the network to focus on detecting targets. When detecting SAR images, it can achieve accurate discrimination of side-by-side ships without manual intervention, thereby performing automated monitoring and significantly improving monitoring efficiency and recognition accuracy.

[0071] (2) The present invention integrates AIS data for multi-dimensional verification. If an AIS signal is found to be abnormal (such as closed, deceptive, etc.), the suspicious target can be automatically identified during the detection process, thereby improving the accuracy of anomaly detection and effectively reducing the false alarm rate;

[0072] (3) The present invention can provide intelligent early warning for abnormal ships and adapt to multi-scenario monitoring. Once suspected abnormal transshipment behavior is detected, an early warning can be triggered immediately, providing accurate decision-making support for maritime law enforcement. BRIEF DESCRIPTION OF THE DRAWINGS

[0073] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following is a brief introduction to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0074] Figure 1 This is a flow chart of a method for monitoring abnormal ship transshipment based on SAR according to the present invention;

[0075] Figure 2 This is a schematic diagram of the sample marking and ship length and width calculation of the present invention;

[0076] Figure 3 The structural diagram of the improved Backbone network designed for the present invention;

[0077] Figure 4 The structural diagram of the dynamic directional convolution block designed for the present invention;

[0078] Figure 5 The structural diagram of the directional attention mechanism designed for this invention;

[0079] Figure 6 This is a flow chart of the ship clustering analysis method of the present invention;

[0080] Figure 7 Schematic diagram of the distance between the bow and stern lines of two ships of the present invention;

[0081] Figure 8 a is the SAR image to be monitored in the present invention;

[0082] Figure 8 b is a schematic diagram of the abnormal ship transshipment results corresponding to the monitoring in the present invention;

[0083] Figure 9 The figure is a structural diagram of a SAR-based ship abnormal transshipment monitoring system of the present invention. DETAILED DESCRIPTION

[0084] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0085] SAR can provide real-time monitoring of ships in any weather and lighting conditions and detect abnormal behavior, while AIS provides information such as the ship's identity, speed, and location. The combination of the two provides strong technical support for timely detection and crackdown on illegal transshipment, smuggling, and illegal fishing, which not only improves maritime security but also helps protect the marine environment. Therefore, this embodiment provides a method for monitoring abnormal transshipment of ships based on SAR, such as Figure 1 Shown, including:

[0086] S1: Acquire SAR images, preprocess the SAR images to obtain SAR image ship samples, and divide the SAR image ship samples into a training set and a test set;

[0087] S2: Introducing the YOLOv8 neural network, which includes a Backbone network, a Neck network, and a Head network, constructing a dynamic directional convolution block, and using the dynamic directional convolution block to replace some convolution blocks in the Backbone network to obtain an improved Backbone network, and then obtaining an improved YOLOv8 neural network;

[0088] S3: Inputting the training set into the improved YOLOv8 neural network for training, and using the test set to optimize the trained improved YOLOv8 neural network to obtain an intelligent ship recognition model, wherein the intelligent ship recognition model is used to recognize ship samples in SAR images and obtain image coordinates and ship types of the ships;

[0089] S4: Collect AIS data, analyze the image coordinates and ship type of the ship in combination with the AIS data, identify abnormal ships based on the analysis results, obtain the coordinates and ship type of the abnormal ships, and send the coordinates and ship type of the abnormal ships to the regulatory authorities to realize the monitoring of abnormal ships.

[0090] Specifically, SAR images are first acquired and preprocessed to obtain SAR image ship samples, which are then divided into training and test sets. SAR can provide real-time ship monitoring in any weather and lighting conditions and detect abnormal behavior. Using the SAR training model can significantly improve the detection accuracy of abnormal ships and enhance the practicality of the model.

[0091] Secondly, the YOLOv8 neural network is introduced. The YOLOv8 neural network includes a Backbone network, a Neck network, and a Head network. A dynamic directional convolution block is constructed. The dynamic directional convolution block is used to replace some convolution blocks in the Backbone network to obtain an improved Backbone network, and then an improved YOLOv8 neural network is obtained. A dynamic directional convolution block is designed to replace the convolution block in the Backbone network in the Yolov8 neural network, so that network recognition is more focused on the target itself, reducing background interference in the image, and allowing the network to focus on detecting the target. Only some parameters are added during the design process, and no excessive parameters are added to the network itself, thereby improving the network's detection speed and frame rate.

[0092] Thirdly, the training set is input into the improved YOLOv8 neural network for training, and the trained improved YOLOv8 neural network is optimized using the test set to obtain an intelligent ship recognition model. The intelligent ship recognition model is used to identify ship samples in SAR images, obtain the image coordinates and ship type of the ship, and the obtained intelligent ship recognition model can perform automated monitoring, greatly improve monitoring efficiency and recognition accuracy, and adapt to multi-scenario monitoring. Once suspected abnormal transshipment behavior is detected, an early warning can be triggered immediately, providing accurate decision-making support for maritime law enforcement;

[0093] Finally, AIS data is collected and the image coordinates and ship types of the ships are analyzed in combination with the AIS data. Abnormal ships are judged based on the analysis results, and the coordinates and ship types of the abnormal ships are obtained. The coordinates and ship types of the abnormal ships are sent to the regulatory authorities to monitor abnormal ships. By constructing spatial topological relationships between ships and integrating AIS information, intelligent identification and precise positioning of suspected illegal transshipment behaviors are achieved. In complex sea conditions where AIS data is abnormal or missing, it still has the ability to distinguish anomalies based on remote sensing image features, which greatly enhances the intelligence, accuracy and system robustness of illegal behavior detection. At the same time, it provides strong technical support for maritime supervision and law enforcement and has broad practical application value.

[0094] In a specific embodiment, a scheme for acquiring SAR images, preprocessing the SAR images, obtaining SAR image ship samples, and dividing the samples into a training set and a test set is as follows:

[0095] Obtain a SAR image, segment it into several sub-images according to a certain pixel size and overlap, and use a rotating box method to mark the position and type of the ship in each sub-image to form a SAR image ship sample;

[0096] In this scheme, the SAR image is divided into several sub-images with 800×800 pixels and 50% overlap, and the minimum bounding rectangle of the ship is used as the location annotation information in each sub-image, such as Figure 2 As shown, the position information includes (x1, y1), (x2, y2), (x3, y3) and (x4, y4). The ship type information is annotated according to the ship texture features. The ship types include tankers, bulk carriers, passenger ships, LNG\LPG carriers, oil tankers, and other cargo ships.

[0097] The SAR image ship samples are divided into a training sample set and a validation sample set in a ratio of 7:3.

[0098] In this solution, SAR can provide real-time ship monitoring in any weather and lighting conditions and detect abnormal behavior. Using SAR training models can significantly improve the detection accuracy of abnormal ships and enhance the practicality of the model.

[0099] In a specific embodiment, a YOLOv8 neural network is introduced, and the YOLOv8 neural network includes a Backbone network, a Neck network, and a Head network. A dynamic directional convolution block is constructed, and the dynamic directional convolution block is used to replace some convolution blocks in the Backbone network to obtain an improved Backbone network. The solution for obtaining the improved YOLOv8 neural network is as follows:

[0100] like Figure 3 As shown, the improved Backbone network includes a first convolution block, a first dynamic directional convolution block, a first C2f module, a second dynamic directional convolution block, a second C2f module, a third dynamic directional convolution block, a third C2f module, a second convolution block and an SPPF module connected in sequence;

[0101] like Figure 4 As shown, the first / second dynamic directional convolution block includes a first branch and a second branch;

[0102] The first branch includes an improved directional attention mechanism and a multi-scale spatial attention mechanism connected in sequence, wherein the multi-scale spatial attention mechanism includes three layers of parallel first, second, and third RELU convolution modules and a softmax activation function;

[0103] The second branch includes a first convolutional layer;

[0104] like Figure 5 As shown, the improved directional attention mechanism includes a two-dimensional adaptive average pooling layer branch and a double pooling convolution branch;

[0105] The two-dimensional adaptive average pooling layer branch includes a two-dimensional adaptive average pooling layer, a fourth RELU convolution module and a first Sigmod convolution module connected in sequence;

[0106] The dual-pooling convolution branch includes a parallel maximum pooling layer, an average pooling layer, and a second Sigmod convolution module connected to the maximum pooling layer and the average pooling layer at the same time.

[0107] In this scheme, a dynamic directional convolution block is designed to replace the convolution block in the Backbone network of the Yolov8 neural network, so that the network recognition is more focused on the target itself, reducing background interference in the image, and allowing the network to focus on detecting the target. During the design process, only part of the parameters are added, and no excessive parameters are added to the network itself, thereby improving the network's detection speed and frame rate.

[0108] In a specific embodiment, the training set is input into the improved YOLOv8 neural network for training, and the test set is used for optimization to obtain an intelligent ship recognition model. The intelligent ship recognition model is used to identify SAR image ship samples. The solution for obtaining the image coordinates and ship type of the ship is:

[0109] S31, inputting the training set into the first convolution block in the improved Backbone network for feature extraction to obtain feature input parameters;

[0110] S32: Inputting the feature input parameters into the first dynamic directional convolution block for feature segmentation, specifically comprising the following steps:

[0111] S321: Divide the feature input parameters equally along the channel dimension, as shown in formula (14),

[0112]

[0113] Among them, W fused represents the feature input parameter, and All W fused The feature input parameters of the first and second branches after equal division, It is a channel splicing operation;

[0114] S322: The data are respectively input into the three parallel 2D adaptive average pooling layer, the maximum pooling layer and the average pooling layer in the improved directional attention mechanism for statistical extraction, as shown in formulas (15)-(17).

[0115]

[0116] Among them, MaxPool() is the maximum pooling calculation, AvgPool() is the average pooling calculation, AdaptiveAvgPool2d() is the adaptive average pooling calculation, F max 、F avg and F Adaptavg They are the maximum pooling feature, average pooling feature and adaptive average pooling feature output after calculation respectively;

[0117] S323: F Adaptavg Input to the fourth RELU convolution module and the first Sigmod convolution module for feature extraction, as shown in formula (18),

[0118] A l =Sigmod(Conv 1×1 (ReLU(Conv 1×1 (F Adaptavg )))) (18)

[0119] A l Represents the weighted features of the output of the two-dimensional adaptive average pooling layer branch;

[0120] F max and F avg Input to the second Sigmod convolution module for convolution, as shown in formula (19),

[0121]

[0122] A r represents the weighted features output by the double-pooled convolution branch, Represents feature fusion operation;

[0123] Calculate A l and A r The tensor product of is shown in formula (20),

[0124]

[0125] Among them, A c is the fusion feature after fusion, stands for tensor product;

[0126] S324: A c Input to the multi-scale spatial attention mechanism for convolution extraction and weight fusion, as shown in formulas (21) and (22),

[0127] F i =ReLU(Conv n*n (A c )) (twenty one)

[0128]

[0129] Among them, F i Represents the multi-scale spatial direction features extracted by the first, second, and third RELU convolution modules, i = 1, 2, 3, n represents the convolution kernel, and in this embodiment, the convolution kernels of the first, second, and third RELU convolution modules are 3*3, 5*5, and 7*7 respectively; ReLU() is the ReLU function calculation for the feature, Conv 1×1 To compress multi-scale spatial directional features into a single channel operation, is the modulation coefficient to prevent gradient saturation, A s It is a multi-scale spatial attention feature that identifies the salient area of ​​the ship direction;

[0130] S325: Input to the first convolutional layer for feature extraction, as shown in formula (23),

[0131]

[0132] Among them, A is the direction sensitive weight value, BN() is the normalization layer calculation, X c is the normalized feature output after extraction by the second branch, ⊙ represents the Hadamard product operation;

[0133] S326: Fuse the multi-scale spatial attention features extracted by the first branch with the normalized features extracted by the second branch to obtain the spatial attention fusion features, as shown in formula (24).

[0134] X s =A s +X c (twenty four)

[0135] X s It is the spatial attention fusion feature;

[0136] S33, X s Input the image into the first C2f module for convolution, and output the multi-level features of the image through convolution within the first C2f module. The multi-level features are input into the second dynamic directional convolution block for feature extraction. The output result of the second dynamic directional convolution block is input into the second C2f module for convolution. The output result of the second C2f module is input into the third dynamic directional convolution block for convolution. The output result of the third dynamic directional convolution block is input into the third C2f module for convolution. The output result is then input into the second convolution module. The output result is input into the SPPF module to obtain the extracted features of the backbone network.

[0137] S34. Input the features output by the SPPF module into the Neck network to extract deep and shallow features, and then input the features extracted by the Neck network into the Head network to obtain the image coordinates and ship type of the ship.

[0138] The intelligent ship identification model obtained in this solution can perform automated monitoring, greatly improving monitoring efficiency and identification accuracy, and adapting to multi-scenario monitoring. Once suspected illegal transshipment behavior is detected, the system can immediately trigger an early warning, providing accurate decision-making support for maritime law enforcement.

[0139] In a specific embodiment, AIS data is collected, and the image coordinates and ship types of the ships are analyzed in combination with the AIS data. An abnormal ship is determined based on the analysis results, and the coordinates and ship types of the abnormal ships are obtained. The coordinates and ship types of the abnormal ships are sent to the regulatory authorities. The solution for monitoring abnormal ships is as follows:

[0140] like Figure 6 As shown, the specific process of the ship clustering analysis method of the present invention is as follows:

[0141] S41. Calculating the spatial relationship between ships in each SAR image ship sample based on the image coordinates of the ships, specifically including:

[0142] Set a distance threshold d. Calculate the distance D between the bow and stern lines of any two ships in the ship identification results of the current SAR image ship sample based on the ship image coordinates. Determine whether D is less than the set distance threshold d. If so, the two ships with a distance D less than the threshold d are called a ship pair, denoted as the set {Si}, where i = 1, 2, ..., n. If the distance D is greater than the set distance threshold d, there is no association between the two ships, and no analysis is performed.

[0143] like Figure 7 As shown, if the distance D between the bow and stern lines (l1, l2) of the two ships is less than the threshold d, the two ships are called a ship pair. In this embodiment, the distance threshold d is set to 5;

[0144] S42, matching the ship pair obtained in S41 with the AIS data, specifically including:

[0145] S421: Collect AIS data and save it to the AIS database;

[0146] S422: Convert the image coordinates of the vessel in {Si} to longitude and latitude coordinates, and retrieve AIS data for the corresponding area from the AIS database to determine whether there are sufficient AIS points. If there are sufficient points, proceed to S43 to determine whether there is AIS information fraud. If there are not sufficient AIS points, it is determined to be an abnormal transshipment, and the vessel in {Si} is marked as an abnormal vessel. Sufficient AIS points means that the number of different MMSI numbers is greater than or equal to r, and in this embodiment, r=2.

[0147] Specifically, the AIS data of the corresponding area is retrieved from the AIS database. The corresponding area refers to the union area of ​​{Si} with the center points of the two ships as the center and a radius of 0.005 decimal degrees as the radius. It is determined whether there are sufficient AIS points. The criterion for sufficiency is that the number of different MMSI numbers is greater than or equal to 2.

[0148] S43. Obtain the length, width and ship type of the ship in {Si} and determine whether they are consistent with the length, width and ship type of the ship recorded in the AIS information. If they are consistent, there is no AIS information fraud and the ship in {Si} is a normal ship. If they are inconsistent, there is AIS information fraud and the ship in {Si} is an abnormal ship. Output the longitude and latitude coordinates and ship type of the ships in the set {Si} and mark them as abnormal transshipment. Send the coordinates and ship type of the abnormal ship to the regulatory department to realize the monitoring of abnormal ships. The judgment basis is shown in formula (25).

[0149]

[0150] in, is the indicator function, J is 1 if there is AIS information fraud, J is 0 if there is no AIS information fraud, L I 、W I 、T I are the length, width and type of the ship respectively, L A 、W A 、T A are the length, width and type of the ship in the AIS data, and r represents the number of different MMSI numbers.

[0151] Test results such as Figure 8 As shown, Figure 8 a is the SAR image to be monitored in the present invention, Figure 8 b is a schematic diagram of the abnormal ship transshipment results corresponding to the monitoring in the present invention;

[0152] S44. The obtained latitude and longitude coordinates and ship type of the abnormal transshipment ship within the monitoring range are sent to a designated terminal through the communication module. The terminal may include a monitoring center server, a maritime management platform or other relevant supervision systems, so that law enforcement personnel or management agencies can obtain abnormal ship information in real time.

[0153] In this solution, by constructing spatial topological relationships between ships and integrating AIS information, intelligent identification and precise positioning of suspected illegal transshipment activities are achieved. Even in complex sea conditions where AIS data is abnormal or missing, it still has the ability to distinguish anomalies based on remote sensing image features, greatly enhancing the intelligence, accuracy and system robustness of illegal behavior detection, providing strong technical support for maritime supervision and law enforcement, and has broad practical application value.

[0154] This embodiment also provides a SAR-based ship abnormal transport monitoring system, such as Figure 9 As shown, there are sample library construction module, intelligent ship identification model training module, intelligent ship identification model interpretation module, cluster analysis module, and abnormal transport warning module;

[0155] The sample library construction module is used to obtain a SAR image, divide it into several sub-images according to a certain pixel size and overlap, and mark the position and type of the ship in each sub-image using a rotating box method to form a SAR image ship sample;

[0156] The intelligent ship recognition model training module is used to divide the SAR image ship samples obtained by the sample library construction module into a training set and a test set, construct an improved YOLOv8 neural network, input the training set into the improved YOLOv8 neural network for training, and optimize the improved YOLOv8 neural network with the test set to obtain an intelligent ship recognition model;

[0157] The intelligent ship recognition model interpretation module is used to interpret the ships in the ship samples to be identified in the SAR image according to the intelligent ship recognition model, and obtain the image coordinates and ship types of the ships;

[0158] The cluster analysis module is used to analyze the ships in each SAR image ship sample and output the latitude and longitude coordinates and ship types of all abnormal transshipment ships;

[0159] The abnormal transshipment warning module is used to send the latitude and longitude coordinates and ship type information of abnormal transshipment ships within the monitoring range.

[0160] This system can automatically output visual warning information and has good scalability. It can be further applied to the monitoring of maritime violations such as illegal fishing and smuggling, helping to build a more complete intelligent ocean supervision system.

[0161] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for monitoring abnormal ship transshipment based on SAR, characterized in that: include: S1: Acquire SAR images, preprocess the SAR images to obtain SAR image ship samples, and divide the SAR image ship samples into a training set and a test set; S2: Introducing the YOLOv8 neural network, which includes a Backbone network, a Neck network, and a Head network, constructing a dynamic directional convolution block, and using the dynamic directional convolution block to replace some convolution blocks in the Backbone network to obtain an improved Backbone network, and then obtaining an improved YOLOv8 neural network; S3: Inputting the training set into the improved YOLOv8 neural network for training, and using the test set to optimize the trained improved YOLOv8 neural network to obtain an intelligent ship recognition model, wherein the intelligent ship recognition model is used to recognize ship samples in SAR images and obtain image coordinates and ship types of the ships; S4: Collect AIS data, analyze the image coordinates and ship type of the ship in combination with the AIS data, identify abnormal ships based on the analysis results, obtain the coordinates and ship type of the abnormal ships, and send the coordinates and ship type of the abnormal ships to the regulatory authorities to realize the monitoring of abnormal ships.

2. The method for monitoring abnormal ship transshipment based on SAR according to claim 1, characterized in that: The improved Backbone network includes a first convolution block, a first dynamic directional convolution block, a first C2f module, a second dynamic directional convolution block, a second C2f module, a third dynamic directional convolution block, a third C2f module, a second convolution block and an SPPF module connected in sequence.

3. The method for monitoring abnormal ship transshipment based on SAR according to claim 2, characterized in that: The first / second dynamic directional convolution block each includes a first branch and a second branch; The first branch includes an improved directional attention mechanism and a multi-scale spatial attention mechanism connected in sequence, wherein the multi-scale spatial attention mechanism includes three layers of parallel first, second, and third RELU convolution modules and a softmax activation function; The second branch includes a first convolutional layer; The improved directional attention mechanism includes a two-dimensional adaptive average pooling layer branch and a double pooling convolution branch; The two-dimensional adaptive average pooling layer branch includes a two-dimensional adaptive average pooling layer, a fourth RELU convolution module and a first Sigmod convolution module connected in sequence; The dual-pooling convolution branch includes a parallel maximum pooling layer, an average pooling layer, and a second Sigmod convolution module connected to the maximum pooling layer and the average pooling layer at the same time.

4. The method for monitoring abnormal ship transshipment based on SAR according to claim 3, characterized in that: Input the training set into the improved YOLO v 8 neural networks are trained, including: S31. Input the training set into the improved Backbon e The first convolution block in the network performs feature extraction to obtain feature input parameters; S32: Inputting the feature input parameters into the first dynamic directional convolution block for feature segmentation, specifically comprising the following steps: S321: Divide the feature input parameters equally along the channel dimension, as shown in formula (1), Among them, W fused represents the feature input parameter, and W fused The feature input parameters of the first and second branches after equal division, It is a channel splicing operation; S322: The data are input into the three parallel 2D adaptive average pooling layer, the maximum pooling layer and the average pooling layer in the improved directional attention mechanism for statistical extraction, as shown in formulas (2)-(4). Among them, MaxPool() is the maximum pooling calculation, AvgPool() is the average pooling calculation, AdaptiveAvgPool2d() is the adaptive average pooling calculation, F max 、F avg and F Adaptavg They are the maximum pooling feature, average pooling feature and adaptive average pooling feature output after calculation respectively; S323: F Adaptavg Input to the fourth RELU convolution module and the first Sigmod convolution module for feature extraction, as shown in formula (5), A l =Sigmod(Conv 1×1 (ReLU(Conv 1×1 (F Adaptavg )))) (5) A l Represents the weighted features of the output of the two-dimensional adaptive average pooling layer branch; F max and F avg Input to the second Sigmod convolution module for convolution, as shown in formula (6), A r represents the weighted features output by the double-pooled convolution branch, Represents feature fusion operation; Calculate A l and A r The tensor product of is shown in formula (7), Among them, A c To fusion features, stands for tensor product; S324: A c Input to the multi-scale spatial attention mechanism for convolution extraction and weight fusion, as shown in formulas (8) and (9), F i =ReLU(Conv n*n (A c )) (8) Among them, F i Represents the multi-scale spatial direction features extracted by the first, second, and third RELU convolution modules, i = 1, 2, 3, n represents the convolution kernel, ReLU() is the ReLU function calculation for the feature, Conv 1×1 To compress multi-scale spatial directional features into a single channel operation, is the modulation coefficient to prevent gradient saturation, A s It is a multi-scale spatial attention feature that identifies the salient area of ​​the ship direction; S325: Input to the first convolutional layer for feature extraction, as shown in formula (10), Among them, A is the direction sensitive weight value, BN() is the normalization layer calculation, X c is the normalized feature output after extraction by the second branch, ⊙ represents the Hadamard product operation; S326: Fuse the multi-scale spatial attention features extracted by the first branch with the normalized features extracted by the second branch to obtain the spatial attention fusion feature, as shown in formula (11). X s =A s +X c (11) X s It is the spatial attention fusion feature; S33, X s Input the image into the first C2f module for convolution, and output the multi-level features of the image through convolution within the first C2f module. The multi-level features are input into the second dynamic directional convolution block for feature extraction. The output result of the second dynamic directional convolution block is input into the second C2f module for convolution. The output result of the second C2f module is input into the third dynamic directional convolution block for convolution. The output result of the third dynamic directional convolution block is input into the third C2f module for convolution. The output result is then input into the second convolution module. The output result is input into the SPPF module to obtain the extracted features of the backbone network. S34. Input the features output by the SPPF module into the Neck network to extract deep and shallow features, and then input the features extracted by the Neck network into the Head network to obtain the image coordinates and ship type of the ship.

5. The method for monitoring abnormal ship transshipment based on SAR according to claim 1, characterized in that: The trained improved YOLOv8 neural network is optimized using the test set to obtain an intelligent ship recognition model, including: Construct the loss function as shown in formula (12), in, is the total loss function, is the classification loss function, is the regression loss function, is the target loss function, λ1, λ2, and λ3 are the weight coefficients of each loss term respectively; The test set is input into the trained improved YOLOv8 neural network, and the model parameters are updated using the stochastic gradient descent method and back propagation mechanism so that the loss function reaches the minimum value, and the intelligent ship recognition model is obtained.

6. The method for monitoring abnormal ship transshipment based on SAR according to claim 1, characterized in that: Collect AIS data, analyze the image coordinates and ship types of ships based on the AIS data, identify abnormal ships based on the analysis results, obtain the coordinates and ship types of abnormal ships, and send the coordinates and ship types of abnormal ships to the regulatory authorities to achieve monitoring of abnormal ships, including: S41. Calculating the spatial relationship between ships in each SAR image ship sample based on the image coordinates of the ships, specifically including: Set a distance threshold d. Calculate the distance D between the bow and stern lines of any two ships in the ship identification results of the current SAR image ship sample based on the ship image coordinates. Determine whether D is less than the set distance threshold d. If so, the two ships with a distance D less than the threshold d are called a ship pair, denoted as the set {Si}, where i = 1, 2, ..., n. If the distance D is greater than the set distance threshold d, there is no association between the two ships, and no analysis is performed. S42, matching the ship pair obtained in S41 with the AIS data, specifically including: S421: Collect AIS data and save it to the AIS database; S422: Convert the image coordinates of the vessel in {Si} to longitude and latitude coordinates, and retrieve AIS data for the corresponding area from the AIS database to determine whether there are sufficient AIS points. If there are sufficient points, proceed to S43 to determine whether there is AIS information fraud. If there are not sufficient AIS points, it is determined to be an abnormal transshipment, and the vessel in {Si} is marked as an abnormal vessel. Sufficient AIS points means that the number of different MMSI numbers is greater than or equal to r. S43. Obtain the length, width, and ship type of the vessel in {Si}, and determine whether they are consistent with the length, width, and ship type of the vessel recorded in the AIS information. If they are consistent, there is no AIS information fraud, and the vessel in {Si} is a normal vessel. If they are inconsistent, there is AIS information fraud, and the vessel in {Si} is an abnormal vessel. Output the latitude and longitude coordinates and ship type of the vessels in the set {Si}, mark them as abnormal transshipments, and send the coordinates and ship type of the abnormal vessel to the regulatory authorities to monitor abnormal vessels.

7. The method for monitoring abnormal ship transshipment based on SAR according to claim 6, characterized in that: The corresponding area is the union area in the set {Si} with the center points of the two ships as the center and the radius set.

8. The method for monitoring abnormal ship transshipment based on SAR according to claim 6, characterized in that: Obtain the length, width and ship type of the ship in {Si} and determine whether it is consistent with the length, width and ship type recorded in the AIS information. The judgment basis is shown in formula (13): in, is the indicator function, J is 1 if there is AIS information fraud, J is 0 if there is no AIS information fraud, L I 、W I 、T I are the length, width and type of the ship respectively, L A 、W A 、T A are the length, width and type of the ship in the AIS data, and r represents the number of different MMSI numbers.

9. A SAR-based ship abnormal transshipment monitoring system, applying the SAR-based ship abnormal transshipment monitoring method according to any one of claims 1 to 8, characterized in that: include: Sample library construction module, intelligent ship identification model training module, intelligent ship identification model interpretation module, cluster analysis module, abnormal transport warning module; The sample library construction module is used to obtain a SAR image, divide it into several sub-images according to a certain pixel size and overlap, and mark the position and type of the ship in each sub-image using a rotating box method to form a SAR image ship sample; The intelligent ship recognition model training module is used to divide the SAR image ship samples obtained by the sample library construction module into a training set and a test set, construct an improved YOLOv8 neural network, input the training set into the improved YOLOv8 neural network for training, and optimize the improved YOLOv8 neural network with the test set to obtain an intelligent ship recognition model; The intelligent ship recognition model interpretation module is used to interpret the ships in the ship samples to be identified in the SAR image according to the intelligent ship recognition model, and obtain the image coordinates and ship types of the ships; The cluster analysis module is used to analyze the ships in each SAR image ship sample and output the latitude and longitude coordinates and ship types of all abnormal transshipment ships; The abnormal transshipment warning module is used to send the latitude and longitude coordinates and ship type information of abnormal transshipment ships within the monitoring range.

Images