Dark ship identification method based on SAR image and AIS data

Abstract

The invention provides a hidden ship identification method based on an SAR image and AIS data. The method comprises the following steps: S1, obtaining the SAR image and the AIS data from an SAR image library and a ship navigation system; s2, performing target detection on the SAR image, and extracting the position and the size of a ship; cleaning the AIS data, and removing noise and invalid data in the AIS data; s3, establishing and training an LSTM model, and predicting the position of a ship at a specific moment based on historical AIS data by using the LSTM model; s4, constructing a GNN model, and performing ship and AIS data matching in the SAR image by using the GNN model; s5, outputting a matching result of S4, namely, a hidden ship without AIS data matching; and S6, predicting the future position of the hidden ship by using the LSTM model, and providing the next shooting position for the SAR satellite in combination with the operation cycle and trajectory of the SAR satellite constellation so as to maximize the coverage of the position area where the hidden ship appears. According to the method, a large amount of manual labeling and analysis workload is reduced, and the method is suitable for large-scale and long-term marine monitoring tasks.

Description

Technical Field

[0001] The present invention relates to the field of smart maritime technology, and in particular to a dark ship recognition method based on SAR images and AIS data. Background Art

[0002] A "dark ship" is a ship that does not have its Automatic Identification System (AIS) turned on or deliberately turns off its AIS. This behavior is usually intended to evade regulation and may be related to illegal activities such as illegal fishing, smuggling, and illegal immigration. Remote sensing technology to detect dark ships is essential for maintaining marine safety, environmental protection, and combating illegal activities. Through remote sensing technology, dark ships can be effectively monitored and tracked, criminal activities at sea can be prevented and responded to, the threat of illegal fishing and illegal dumping to the marine ecological environment can be reduced, maritime traffic safety can be ensured, the sustainable use of marine resources can be guaranteed, and strong technical support can be provided for law enforcement and management.

[0003] SAR is an active remote sensing technology that uses radar to transmit microwave signals and receive reflected signals to obtain images of the ground or sea surface. Compared with optical remote sensing, SAR is not restricted by weather and lighting conditions. It can work normally under conditions such as clouds, rain, snow, and at night, providing continuous monitoring capabilities. The combination of SAR and AIS data is an important means of ocean monitoring and dark ship detection.

[0004] At present, there are two problems when matching the positions of ships in SAR images with AIS data. First, since the SAR image is taken at a specific moment, and the AIS data is a discrete point within the time range, there may be no corresponding AIS point at the time the SAR image is taken. The existing technology estimates the position of the ship at a certain moment by simple interpolation of the ship's speed and route, but this method has limitations. Secondly, the existing SAR and AIS matching methods, such as the Hungarian algorithm, perform well in offshore scenarios, but in nearshore scenarios, due to complex factors such as densely arranged ships, the matching effect is not ideal. Therefore, in order to solve the above technical problems, it is urgent to construct a dark ship recognition method based on SAR images. Summary of the invention

[0005] In view of this, the purpose of the present invention is to propose a dark ship recognition method based on SAR images and AIS data to solve the technical problem that the existing SAR and AIS matching methods have unsatisfactory matching effects.

[0006] The technical means adopted by the present invention are as follows:

[0007] A dark ship recognition method based on SAR images and AIS data comprises the following steps:

[0008] S1, obtain SAR images and AIS data from SAR image library and ship navigation system;

[0009] S2, perform target detection on SAR images to extract the position and size of the ship; clean the AIS data to remove noise and invalid data in the AIS data;

[0010] S3, build and train the LSTM model, and use the LSTM model to predict the position of the ship at a specific time based on the AIS data processed by S2;

[0011] S4, constructing a GNN model based on the position of the ship at a specific time, and using the GNN model to match the ship in the SAR image with the AIS data;

[0012] S5, output the matching result of S4, that is, the dark ship with no AIS data matching;

[0013] S6. Use the LSTM model to predict the future position of the dark ship, and combine the operating cycle and trajectory of the SAR satellite constellation to provide the SAR satellite with the location of the next shot to maximize the coverage of the location area where the dark ship appears.

[0014] Furthermore, in S3, the steps to build and train the LSTM model are as follows:

[0015] S31, LSTM model input features include the longitude and latitude of the ship at time t (x t ,y t ), speed v t , heading θ t and ship type, combining the input features into a vector X t =[x t ,y t ,v t ,θ t ,e t ], where e t is the embedding representation of the ship type;

[0016] S32, LSTM model receives time series input, and the data of T time steps is represented as X t-T ,X t-T+1 ,...,X t}, and generate the hidden state h for each time step t ;

[0017] S33. For each time step, LSTM updates the hidden state h t , expressed as:

[0018] h t ,c t =LSTM(X t ,ht-1 ,c t-1 )

[0019] Among them, c t is the memory unit state, h t is the hidden state of the output;

[0020] S34, the hidden state h at the last time step t Then connect a fully connected layer to map the output to the predicted ship position (x t+1 ,y t+1 ):

[0021]

[0022] Where W and b are the learned weights and biases;

[0023] S35. Use mean square error as the loss function of the LSTM model:

[0024]

[0025] Where N is the number of samples; is the true position of the i-th sample; Predict locations for the model;

[0026] S36. Use the Adam optimizer to train the model to obtain the position of the ship at a specific moment and update the model through the back propagation algorithm.

[0027] Furthermore, the specific steps of S4 are as follows:

[0028] S41. Each ship and AIS point detected by SAR is regarded as a node, and the node is represented as Among them, x i represents the SAR feature, y i Represents AIS features; establishes edges between nodes based on location and time as preliminary matching conditions, and the weight of the edge represents the similarity between nodes;

[0029] S42. Aggregate node features using graph convolutional networks, expressed as:

[0030]

[0031] in, represents the set of neighbor nodes of node i, W (l) is the weight matrix of the lth layer, σ is the activation function;

[0032] S43, matching is performed based on the similarity of node features, and for each SAR node, the AIS node with the highest similarity is found for matching;

[0033] S44. Screening matching pairs through gating mechanism:

[0034] Among them, W g is the weight matrix, σ is the activation function, and ⊙ is the element multiplication.

[0035] Furthermore, in S4, a data augmentation method is used when constructing a GNN model based on the position of the ship at a specific moment. The steps of data augmentation are as follows:

[0036] For each ship position (x, y), generate a random movement distance d and θ, and calculate the new position (x′, y′):

[0037] x′=x+d·cos(θ), y′=y+d·sin(θ)

[0038] Randomly delete 1 to n points from the AIS data to simulate the presence of a hidden ship.

[0039] Furthermore, the specific steps of S6 are as follows:

[0040] S61. According to the LSTM model, the dark ship will be calculated at the next time t n The ship's position is:

[0041]

[0042] in, is the current dark ship position, is the current speed, is the current heading, type is the ship type;

[0043] Dark Ship Future Location Area A ship For all t n The set of is as follows:

[0044] A ship ={(x n ,y n )]t n ∈[t0,t0+T predict ]}

[0045] Among them, T predict is the future duration predicted by the LSTM model;

[0046] S62, the position of the SAR satellite at time t (x sat (t),y sat (t)) is expressed as:

[0047] (x sat (t),y sat(t)) = f sat (t mod T sat )

[0048] Among them, f sat (t) is a periodic function describing the satellite trajectory, T sat is the satellite's operating period, t mod T sat represents the modulo operation of time t on the satellite orbit period, so that every time T sat At the same time, the satellite's position will repeat;

[0049] S63, find the optimal shooting time t opt , so that:

[0050] max(Overlap(A ship ,A sat (t opt )))

[0051] Among them, A sat (t opt ) is the satellite at t opt Overlap (A ship ,A sat (t opt )) represents the overlapping area between the area where the dark ship will appear in the future and the area photographed by the satellite.

[0052] The present invention also provides a storage medium, characterized in that the storage medium includes a stored program, wherein when the program is run, any of the above-mentioned dark ship identification methods based on SAR images and AIS data is executed.

[0053] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes any one of the above-mentioned dark ship identification methods based on SAR images and AIS data through the computer program.

[0054] Compared with the prior art, the present invention has the following advantages:

[0055] The present invention achieves accurate identification of "hidden ships" at sea by combining SAR images with AIS data. The LSTM model predicts the future position of the ship and combines the orbital period of the SAR satellite to effectively guide the satellite's subsequent shooting of the hidden ship and maximize the coverage of potential ship activity areas. This significantly improves the efficiency and accuracy of hidden ship detection, especially in complex sea areas.

[0056] The present invention utilizes the complementarity of SAR images and AIS data, combines the missing or incomplete ship information in AIS data with the ship detection results in SAR images, and enhances the ability to detect hidden ships. Through methods such as data augmentation, the robustness of ship identification is further improved, and it can cope with diverse maritime targets in different environments.

[0057] The present invention reduces a large amount of manual labeling and analysis workload and is suitable for large-scale, long-term marine monitoring tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0059] Figure 1 It is a schematic diagram of the process of the present invention.

[0060] Figure 2 This is the result diagram of the hidden ship identification of the present invention. DETAILED DESCRIPTION

[0061] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0062] like Figure 1 As shown, the present invention provides a dark ship recognition method based on SAR images and AIS data, comprising the following steps:

[0063] S1, obtain SAR images and AIS data from SAR image library and ship navigation system;

[0064] S2, perform target detection on SAR images to extract the position and size of the ship; clean the AIS data to remove noise and invalid data in the AIS data;

[0065] S3, build and train the LSTM model, and use the LSTM model to predict the position of the ship at a specific time based on the AIS data processed by S2;

[0066] The steps to build and train the LSTM model are as follows:

[0067] S31, LSTM model input features include the longitude and latitude of the ship at time t (x t ,y t ), speed v t , heading θ t and ship type, combining the input features into a vector X t =[x t ,y t ,v t ,θ t ,e t ], where e t is the embedding representation of the ship type;

[0068] S32, LSTM model receives time series input, and the data of T time steps is represented as X t-T ,X t-T+1 ,...,X t}, and generate the hidden state h for each time step t ;

[0069] S33. For each time step, LSTM updates the hidden state h t , expressed as:

[0070] h t ,c t =LSTM(X t ,h t-1 ,c t-1 )

[0071] Among them, c t is the memory unit state, h t is the hidden state of the output;

[0072] S34, the hidden state h at the last time step t Then connect a fully connected layer to map the output to the predicted ship position (x t+1 ,y t+1 ):

[0073]

[0074] Where W and b are the learned weights and biases;

[0075] S35. Use mean square error as the loss function of the LSTM model:

[0076]

[0077] Where N is the number of samples; is the true position of the i-th sample; Predict locations for the model;

[0078] S36. Use the Adam optimizer to train the model to obtain the position of the ship at a specific moment and update the model through the back propagation algorithm.

[0079] S4, constructing a GNN model based on the position of the ship at a specific time, and using the GNN model to match the ship in the SAR image with the AIS data;

[0080] S41. Each ship and AIS point detected by SAR is regarded as a node, and the node is represented as Among them, x i represents the SAR feature, y i Represents AIS features; establishes edges between nodes based on location and time as preliminary matching conditions, and the weight of the edge represents the similarity between nodes;

[0081] S42. Aggregate node features using graph convolutional networks, expressed as:

[0082]

[0083] in, represents the set of neighbor nodes of node i, W (l) is the weight matrix of the lth layer, σ is the activation function;

[0084] S43, matching is performed based on the similarity of node features, and for each SAR node, the AIS node with the highest similarity is found for matching;

[0085] S44. Screening matching pairs through gating mechanism:

[0086] Among them, W g is the weight matrix, σ is the activation function, and ⊙ is the element multiplication.

[0087] In S4, the data augmentation method is used to build the GNN model based on the position of the ship at a specific time. The data augmentation steps are as follows:

[0088] For each ship position (x, y), generate a random movement distance d and θ, and calculate the new position (x′, y′):

[0089] x′=x+d·cos(θ), y′=y+d·sin(θ)

[0090] Randomly delete 1 to n points from the AIS data to simulate the presence of a hidden ship.

[0091] S5, output the matching result of S4, that is, the dark ship with no AIS data matching;

[0092] S6. Use the LSTM model to predict the future position of the dark ship, and combine the operating cycle and trajectory of the SAR satellite constellation to provide the SAR satellite with the location of the next shot to maximize the coverage of the location area where the dark ship appears.

[0093] S61. According to the LSTM model, the dark ship will be calculated at the next time t n The ship's position is:

[0094]

[0095] in, is the current dark ship position, is the current speed, is the current heading, type is the ship type;

[0096] Dark Ship Future Location Area A ship For all t n The set of is as follows:

[0097] A ship ={(x n ,y n )|t n ∈[t0,t0+T predict ]}

[0098] Among them, T predict is the future duration predicted by the LSTM model;

[0099] S62, the position of the SAR satellite at time t (x sat (t),y sat (t)) is expressed as:

[0100] (x sat (t),y sat (t)) = f sat (t mod T sat )

[0101] Among them, f sat (t) is a periodic function describing the satellite trajectory, T sat is the satellite's operating period, t mod T sat represents the modulo operation of time t on the satellite orbit period, so that every time T sat At the same time, the satellite's position will repeat;

[0102] S63, find the optimal shooting time t opt , so that:

[0103] max(Overlap(A ship ,A sat (t opt)))

[0104] Among them, A sat (t opt ) is the satellite at t opt Overlap (A ship ,A sat (t opt )) represents the overlapping area between the area where the dark ship will appear in the future and the area photographed by the satellite.

[0105] Example

[0106] A dark ship recognition method based on SAR images and AIS data comprises the following steps:

[0107] S1, obtain SAR images and AIS data from SAR image library and ship navigation system;

[0108] Specifically, 200 SAR images of the waters around Brazil and Japan were selected from the SAR image database, as well as the AIS data within the corresponding range at the time when each image was taken.

[0109] S2, perform target detection on SAR images, extract the location and size of the ship; clean the AIS data to remove noise and invalid data;

[0110] Among them, S2 performs target detection on SAR images, extracts the position and size of ships, and uses the SDFSD-v1.0 dataset to train the Oriented RCNN model to extract ship targets. The dataset includes 15 types of ships. The Oriented RCNN model trained with this dataset can identify ship types such as tankers, bulk carriers, container ships, warships, tugboats, fishing boats, cargo ships, and passenger ships.

[0111] S3, using the LSTM model to predict the location of the ship at a specific moment based on historical AIS data;

[0112] The model training process includes:

[0113] S31, input features include the latitude and longitude of the ship at time t (x t ,y t ), speed v t , heading θ t and ship type, combining these features into a vector X t = [ x t ,y t ,v t ,θ t ,e t] , where e t is the embedding representation of the ship type.

[0114] S32, LSTM model receives time series input, and the data of T time steps is represented as X t-T ,X t-T+1 ,...,X t}, and generate the hidden state h for each time step t ;

[0115] S33. For each time step, LSTM updates the hidden state h t , this process can be expressed as:

[0116] h t ,c t =LSTM(X t ,h t-1 ,c t-1 )

[0117] Among them, c t is the memory unit state, h t is the hidden state of the output;

[0118] S34, the hidden state h at the last time step t Then connect a fully connected layer to map the output to the predicted ship position (x t+1 ,y t+1 ):

[0119]

[0120] Where W and b are the learned weights and biases;

[0121] S35. Use mean square error as loss function:

[0122]

[0123] Where N is the number of samples, is the true position of the i-th sample, Predict locations for the model;

[0124] S36. Use the Adam optimizer to train the model to obtain the position of the ship at a specific moment and update the model through the back propagation algorithm;

[0125] S4, using the GNN model to match ships and AIS data in SAR images;

[0126] Specifically, the GNN model construction process is as follows:

[0127] S41. Each ship and AIS point detected by SAR is regarded as a node, and the node is represented as Among them, x irepresents the SAR feature, y i Represents AIS features; establishes edges between nodes based on location and time as preliminary matching conditions, and the weight of the edge represents the similarity between nodes;

[0128] S42. Aggregate node features using graph convolutional networks, expressed as:

[0129]

[0130] in, represents the set of neighbor nodes of node i, W (l) is the weight matrix of the lth layer, σ is the activation function;

[0131] S43, matching is performed based on the similarity of node features, and for each SAR node, the AIS node with the highest similarity is found for matching;

[0132] S44. Screening matching pairs through gating mechanism:

[0133] Among them, W g is the weight matrix, σ is the activation function, and ⊙ is the element multiplication;

[0134] In order to better train the GNN model and improve the matching effect between SAR images and AIS data, the data augmentation method is used to generate more samples. Data augmentation mainly includes moving the SAR ship position and simulating dark ships. The data augmentation method is as follows:

[0135] For each ship position (x, y), generate a random movement distance d and θ, and calculate the new position (x′, y′):

[0136] x′=x+d·cos(θ), y′=y+d·sin(θ)

[0137] Randomly delete 1 to n points from the AIS data to simulate the presence of a hidden ship;

[0138] S5, output the matching result, i.e., the dark ship with no AIS data matching;

[0139] See also Figure 2 , Figure 2 This is a diagram of the dark ship recognition result of the present invention. It can be seen that the present invention provides a dark ship recognition method based on SAR images and AIS data, and dark ship recognition on SAR images can achieve higher accuracy.

[0140] S6. Use the LSTM model to predict the future position of the dark ship, and combine the operation cycle and trajectory of the SAR satellite constellation to provide the SAR satellite with the position of the next shot to maximize the coverage of the location area where the dark ship appears. The specific process is as follows:

[0141] S61. According to the LSTM model, the dark ship will be calculated at the next time t n The ship's position is:

[0142]

[0143] in, is the current dark ship position, is the current speed, is the current heading, type is the ship type;

[0144] Dark Ship Future Location Area A ship For all t n A collection of:

[0145] A ship ={(x n ,yn)|t n ∈[t0,t0+T predict ]}

[0146] Among them, T predict is the future duration predicted by the LSTM model;

[0147] S62, the position of the SAR satellite at time t (x sat (t),y sat (t)) can be expressed as:

[0148] (x sat (t),y sat (t)) = f sat (t mod T sat )

[0149] Among them, f sat (t) is a periodic function describing the satellite trajectory, T sat is the satellite's operating period, t mod T sat represents the modulo operation of time t on the satellite orbit period, so that every time T sat At the same time, the satellite's position will repeat;

[0150] S63, find the optimal shooting time t opt , so that:

[0151] max(Overlap(A ship ,A sat (t opt )))

[0152] Among them, A sat (t opt ) is the satellite at t opt Overlap (Aship ,A sat (t opt )) represents the overlapping area between the area where the dark ship will appear in the future and the area photographed by the satellite.

[0153] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A dark ship recognition method based on SAR images and AIS data, characterized in that: The steps include: S1, obtain SAR images and AIS data from SAR image library and ship navigation system; S2, perform target detection on SAR images to extract the position and size of the ship; clean the AIS data to remove noise and invalid data in the AIS data; S3, build and train the LSTM model, and use the LSTM model to predict the position of the ship at a specific time based on the AIS data processed by S2; S4, constructing a GNN model based on the position of the ship at a specific time, and using the GNN model to match the ship in the SAR image with the AIS data; S5, output the matching result of S4, that is, the dark ship with no AIS data matching; S6. Use the LSTM model to predict the future position of the dark ship, and combine the operating cycle and trajectory of the SAR satellite constellation to provide the SAR satellite with the location of the next shot to maximize the coverage of the location area where the dark ship appears.

2. The dark ship identification method based on SAR images and AIS data according to claim 1 is characterized in that: In S3, the steps to build and train the LSTM model are as follows: S31, LSTM model input features include the longitude and latitude of the ship at time t (x t ,y t ), speed v t , heading θ t and ship type, combining the input features into a vector X t =[x t ,y t ,v t ,θ t ,e t ], where e t is the embedding representation of the ship type; S32, LSTM model receives time series input, and the data of T time steps is represented as {X t-T ,X t-T+1 ,...,X t }, and generate the hidden state h for each time step t ; S33. For each time step, LSTM updates the hidden state h t , expressed as: h t ,c t =LSTM(X t ,h t-1 ,c t-1 ) Among them, c t is the memory unit state, h t is the hidden state of the output; S34, the hidden state h at the last time step t Then connect a fully connected layer to map the output to the predicted ship position (x t+1 ,y t+1 ): Where W and b are the learned weights and biases; S35. Use mean square error as the loss function of the LSTM model: Where N is the number of samples; is the true position of the i-th sample; Predict locations for the model; S36. Use the Adam optimizer to train the model to obtain the position of the ship at a specific moment and update the model through the back propagation algorithm.

3. The dark ship identification method based on SAR images and AIS data according to claim 1 is characterized in that: The specific steps of S4 are as follows: S41. Each ship and AIS point detected by SAR is regarded as a node, and the node is represented as Among them, x i represents the SAR feature, y i Represents AIS features; establishes edges between nodes based on location and time as preliminary matching conditions, and the weight of the edge represents the similarity between nodes; S42. Aggregate node features using graph convolutional networks, expressed as: in, represents the set of neighbor nodes of node i, W (l) is the weight matrix of the lth layer, σ is the activation function; S43, matching is performed based on the similarity of node features, and for each SAR node, the AIS node with the highest similarity is found for matching; S44. Screening matching pairs through gating mechanism: Among them, W g is the weight matrix, σ is the activation function, and ⊙ is the element multiplication.

4. The dark ship identification method based on SAR images and AIS data according to claim 1 is characterized in that: In S4, the data augmentation method is used to build the GNN model based on the position of the ship at a specific time. The data augmentation steps are as follows: For each ship position (x, y), generate a random movement distance d and θ, and calculate the new position (x′, y′): x′=x+d·cos(θ), y′=y+d·sin(θ) Randomly delete 1 to n points from the AIS data to simulate the presence of a hidden ship.

5. The dark ship identification method based on SAR images and AIS data according to claim 1 is characterized in that: S6 specific steps are as follows: S61. According to the LSTM model, the dark ship will be calculated at the next time t n The ship's position is: in, is the current dark ship position, is the current speed, is the current heading, type is the ship type; Dark Ship Future Location Area A ship For all t n The set of is as follows: A ship ={(x n ,y n )|t n ∈[t0,t0+T predict ]} Among them, T predict is the future duration predicted by the LSTM model; S62, the position of the SAR satellite at time t (x sat (t),y sat (t)) is expressed as: (x sat (t),y sat (t))=f sat (t mod T sat ) Among them, f sat 9t) is a periodic function describing the satellite trajectory, T sat is the satellite's operating period, t mod T sat represents the modulo operation of time t on the satellite orbit period, so that every time T sat At the same time, the satellite's position will repeat; S63. Find the optimal shooting time t opt , so that: Among them, A sat (t opt ) is the satellite at t opt Overlap (A ship ,A sat (t opt )) represents the overlapping area between the area where the dark ship will appear in the future and the area photographed by the satellite.

6. A storage medium, characterized in that: The storage medium includes a stored program, wherein when the program is run, the dark ship recognition method based on SAR images and AIS data described in any one of claims 1 to 5 is executed.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: The processor executes the dark ship identification method based on SAR images and AIS data as described in any one of claims 1 to 5 through the operation of the computer program.

Images