Ship trajectory prediction and anomaly detection method, device and equipment based on deep cross LSTM network, and storage medium

Abstract

The invention provides a ship trajectory prediction and anomaly detection method, device and equipment based on a deep cross LSTM network, and a storage medium, and the method comprises the steps: S1, preprocessing AIS data, and extracting ship navigation features; step S2, constructing a deep cross LSTM network model based on the attention mechanism and a coder-decoder, which comprises the following steps: in a coder part, converting input multivariate time series data into high-dimensional vector representation, and in combination with the multi-head attention mechanism and an LSTM network, capturing a dependency relationship and periodic change among the time series data, and constructing a deep cross LSTM network model based on the multi-head attention mechanism and the LSTM network; the learning of the network model on the data relationship is enhanced and the calculation efficiency is improved; and S3, training the deep cross LSTM network model, and predicting ship navigation features by using the trained deep cross LSTM network model. The network model is applied to two different prediction tasks, namely a joint prediction task and a historical data prediction task.

Description

Technical Field

[0001] The present invention relates to the field of marine ship monitoring, and in particular to a ship trajectory prediction and anomaly detection method, device, equipment and storage medium based on a deep cross LSTM network. Background Art

[0002] Maritime traffic monitoring and maritime management are of great significance to ship safety. By using the Automatic Identification System (AIS) to accurately grasp the dynamic information of ships in real time and take reasonable measures to effectively guide the navigation behavior of ships in a timely manner, the automatic tracking and collision warning of ships can be realized, which is of great significance to improving the efficiency of water traffic supervision and reducing traffic risks. However, ships are affected by many factors during navigation, and there are difficulties in information transmission at sea, which leads to noise in AIS data. Therefore, ship trajectory prediction is more difficult than trajectory prediction in other fields. There are roughly three types of ship trajectory prediction methods: classical statistical theory prediction method, traditional neural network method and deep learning method.

[0003] (I) The classical statistical theory prediction method is a method based on mathematical statistics. This method assumes that there is a certain correlation between future data analysis and historical data analysis, and uses this correlation to build a model. In the dynamic situation of ship navigation, each feature will reflect a specific change pattern.

[0004] (II) Traditional neural networks and deep learning can be used to learn complex spatiotemporal relationships between variables, and can also be used to conduct ship trajectory prediction research. However, in practical applications, the ship's AIS is a time series data, and the trajectory data is a variable with a connection between the past and the present. Traditional mathematical modeling and machine learning methods are not suitable for situations with complex trajectory changes. Therefore, by introducing a network with long short-term memory (LSTM), further improvements have been made to the ship trajectory prediction problem to improve the accuracy of the prediction.

[0005] The main problems with the existing technologies include: First, the existing ship trajectory prediction methods are mainly based on machine learning or dynamic models. However, traditional neural networks often have problems such as difficulty in fully mining time features, difficulty in adjusting parameters, and easy to fall into local optimality when processing time series data. The dynamic model is subject to the dynamics and diversity of sea conditions and the complexity of ship movement, so it is difficult to model, the prediction accuracy is low, and the stability is poor. Second, although the LSTM network has made certain progress in ship status or trajectory prediction, there are still some limitations. For example, the LSTM network has the problem of gradient explosion, and all features of the input sequence are given the same weight, and the previous and next time series cannot be distinguished, resulting in the inability to pay special attention to the key information in the sequence, so the calculation amount is large and the prediction accuracy is not high. Third, the ship's navigation status or prediction only relies on AIS data and prediction models, lacks the fusion and analysis of multi-source data, such as combining satellite remote sensing data, meteorological data, etc., which affects the prediction accuracy. Summary of the invention

[0006] The present invention proposes a ship trajectory prediction and anomaly detection method, device, equipment and storage medium based on a deep cross LSTM network, and further proposes an LSTM deep learning prediction network framework - a deep cross LSTM network (LSTM-DCross), a long short-term memory network ( , Long Short-Term Memory) is a time-recurrent neural network that is designed to solve general A neural network specially designed to solve the long-term dependency problem of (recurrent neural network). This network adds encoding and (multi-head) attention mechanisms on the basis of LSTM network, effectively mining the correlation between the sequence to be tested and the input sequence; for two scenarios, the joint prediction method and historical data prediction method are used for ship navigation behavior to predict ship trajectory and detect anomalies based on LSTM-DCross. Among them, the joint prediction method is suitable for the scenario of GPS failure, while the historical data prediction method is suitable for the situation of complete AIS failure. Even if the AIS data is missing or invalid from a certain moment, this method can still rely entirely on the accurate data of historical AIS to complete effective predictions for future moments.

[0007] The present invention uses the "LSTM+attention mechanism" network under the "encoder+decoder" framework, first encodes the input data, then uses the attention mechanism to calculate the encoded data relevance, and then performs LSTM network calculations, and uses the "encoder+decoder" structure to characterize the historical data dependency. This processing method is more scientific and reasonable than directly inputting data or performing LSTM network calculations first. Encoding the data first can effectively compress the amount of data, extract key features, and provide more valuable information input for subsequent calculations. Using the attention mechanism to calculate the encoded data relevance first can highlight important data features and allow the model to focus more on factors that have an important impact on trajectory prediction. Compared with directly inputting data or performing LSTM network calculations first, the present invention is more scientific and reasonable, and can better capture the dynamic changes and internal laws of ship trajectories. And the historical data prediction method of the present invention is still applicable when AIS fails completely, and has a broader application scenario.

[0008] The present invention proposes a ship trajectory prediction and anomaly detection method based on a deep cross LSTM network, the method comprising the following steps:

[0009] Step S1, preprocessing AIS data to extract ship navigation characteristics;

[0010] Step S2, constructing a deep cross LSTM network model based on the attention mechanism and the encoder-decoder; including: in the encoder part, converting the input multivariate time series data into a high-dimensional vector representation, and combining the multi-head attention mechanism and the LSTM network to capture the dependencies and periodic changes between the time series data, so as to enhance the network model's learning of data relationships and improve computational efficiency;

[0011] Step S3: train the deep cross LSTM network model and use the trained deep cross LSTM network model to predict the ship navigation characteristics.

[0012] The present invention also proposes a ship trajectory prediction and anomaly detection device based on a deep cross LSTM network, the device comprising:

[0013] AIS data preprocessing module, which preprocesses AIS data and extracts ship navigation characteristics;

[0014] The network model building module builds a deep cross LSTM network model based on the attention mechanism and encoder-decoder. In the encoder part, the input multivariate time series data is converted into a high-dimensional vector representation. At the same time, the multi-head attention mechanism and LSTM network are combined to capture the dependencies and periodic changes between time series data, so as to enhance the network model's learning of data relationships and improve computational efficiency.

[0015] The ship navigation characteristics prediction module trains the deep cross LSTM network model and uses the trained deep cross LSTM network model to predict the ship navigation characteristics.

[0016] The present invention also proposes an electronic device, comprising: one or more processors; and a memory for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the above-described method.

[0017] The present invention also provides a computer-readable storage medium on which executable instructions are stored. When the instructions are executed by a processor, the processor implements the above-mentioned method.

[0018] The deep cross LSTM network proposed in this invention has three technical advantages:

[0019] 1. Combining the attention mechanism with the LSTM network to extract features from the encoded ship time series data can effectively mine the change cycle and long-term dependency in the time series data, and improve the model's ability to model complex time series data;

[0020] 2. Adopting encoder-decoder structure and multi-head attention cross mechanism, effectively mining the relationship between input data and output data to be predicted;

[0021] 3. The model is universal and can be applied to joint prediction tasks and historical data prediction tasks respectively. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 This is a schematic diagram of the deep cross LSTM network structure;

[0023] Figure 2 It is a flow chart of the ship behavior prediction and anomaly detection method;

[0024] Figure 3 It is a schematic diagram of normal and abnormal ship position characteristics;

[0025] Figure 4 The AIS track diagram of three ships in the data set ①, Figure 4 (a) is the track chart of the ship with MMSI No. 412331140. Figure 4 (b) is the track chart of the ship with MMSI No. 414343000. Figure 4 (c) is the track chart of the ship with MMSI No. 538004382;

[0026] Figure 5 For the data set ②③, the AIS track diagram of the ship, Figure 5 (a) is the track chart of the ship with MMSI No. 413631000. Figure 5(b) is the point track diagram of the ship with MMSI number 413631000;

[0027] Figure 6 The prediction results of BP, LSTM-ATT and LSTM-DCross networks for the tracks of three ships under the joint prediction method;

[0028] Figure 7 The prediction results of BP, LSTM-ATT and LSTM-DCross for ship trajectories using historical data prediction methods;

[0029] Figure 8 The prediction results of BP, LSTM-ATT and LSTM-DCross for ship heading using historical data prediction method;

[0030] Fig. 9 The prediction results of BP, LSTM-ATT and LSTM-DCross for ship speed using historical data prediction method;

[0031] Fig.10 This is a radar chart comparing the errors of the three prediction networks;

[0032] Fig.11 The prediction effect of deep cross LSTM;

[0033] Fig.12 AIS ship track data and thermal infrared images at a certain time of night;

[0034] Fig.13 Verify the ship point trajectory by combining multi-source data. DETAILED DESCRIPTION

[0035] In order to make the purpose, technical scheme and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention. In addition, the technical features involved in each embodiment of the present invention described below can be combined with each other as long as they do not conflict with each other. To achieve the above-mentioned purpose, the present invention adopts the following technical scheme.

[0036] The present invention proposes a ship trajectory prediction and anomaly detection method based on a deep cross LSTM network, the method comprising the following steps:

[0037] Step S1, AIS data preprocessing, extracting ship navigation characteristics; In step S1, cubic spline interpolation method is used to fill the data, and mean variance normalization method is used to normalize the AIS data.

[0038] Step S2: Construct a deep cross LSTM (LSTM-DCross) based on attention and encoding-decoding mechanism.

[0039] The structure of the deep cross LSTM (LSTM-DCross) network proposed in this invention is as follows: Figure 1 As shown in the figure, in the encoder part, the input multivariate time series data is converted into a high-dimensional vector representation, which involves the encoding conversion of the data. Then, the multi-head attention mechanism and LSTM network are combined to capture the main dependencies and periodic changes between the time series data to enhance the model's learning of data relationships and improve computational efficiency. The decoder part uses the last few vectors of the input sequence as the initial input vector to start generating the target sequence. Through the multi-head cross-attention mechanism, the model can calculate the correlation between the sequence to be tested and the input sequence to more accurately predict the target sequence.

[0040] Encoder:

[0041] like Figure 1 As shown on the left, the main task of the encoder is to uniquely encode the input multivariate time series data, convert it into a high-dimensional vector representation, and combine the attention mechanism to capture the historical and periodic change relationship between the input sequences, thereby enhancing the model's learning ability. The following are the detailed steps of the encoder:

[0042] Step S2.1 Multivariate time series data encoding conversion: First, the input multivariate time series data Perform encoding conversion, where Represents the length of the input time series, each is a four-dimensional vector, including longitude, latitude, speed and heading. In the present invention, the input multivariate time series data For the continuous time series of AIS data of ships, a high-dimensional vector with time and location characteristics is generated through position coding and time coding. This step is expressed as:

[0043] (1)

[0044] in, represents the position and time encoding function, which is used to encode the input sequence Perform position coding and time coding. Represents the input sequence Middle The position and time encoding vector of each time point.

[0045] When generating high-dimensional vectors, the specific formulas for time and position encoding are as follows:

[0046] (2)

[0047] (3)

[0048] in, represents the time step, Represents the time step location, Represents the dimension of the model. The encoding function performs sine and cosine transforms of different frequencies on the input sequence, so that the model can capture the timing information at different time steps of the sequence.

[0049] Step 2.2 Attention mechanism to learn sequence data relationships and LayerNorm layer normalization: Use the attention mechanism to learn the interdependence of information keys at each moment in the sequence data, and enhance the model's understanding and representation of the input sequence. When processing input sequence data, the multi-head attention mechanism can focus on different parts of information at the same time, so that it can calculate multiple attention distributions in parallel and learn the correlation between different parts of the sequence data. This part of the calculation process can be expressed as:

[0050] (4)

[0051] in, , , Respectively represent the query vector, key vector and value vector obtained by linear transformation, , and is the weight matrix of the linear transformation, is the hidden state output by the encoder, is the dimension of the key vector.

[0052] The calculation formula of the multi-head attention mechanism is as follows:

[0053] (5)

[0054] in, is the output weight matrix of the multi-head attention mechanism, Represents the attention distribution of each head. The multi-head attention mechanism learns the correlation of different parts of the input sequence by parallelizing multiple attention heads.

[0055] Afterwards, the LayerNorm layer is used to normalize the output of the encoder to maintain the stability of model training and help accelerate the convergence of the model.

[0056] Step 2.3 LSTM network processing and linear transformation: Use LSTM network to process the output of the encoder to capture the long-term dependencies between the input sequence data. LSTM network can remember a certain amount of information at each time step and forget unnecessary information, so as to better capture the relationship between the input sequence data. Specifically, the calculation process of LSTM network is as follows:

[0057] LSTM memory cell update:

[0058] (6)

[0059] in, It is the coefficient to adjust the memory state value of the last calculation. is the memory state value calculated at the previous time step, is the intermediate state value or the internal memory unit output value, is the input gate calculated value, is the output status value.

[0060] Input gate input:

[0061] (7)

[0062] in, is the weight matrix of the input gate, is the output value of the previous time step, is the current input vector, that is, the output value of the LayerNorm layer, is the bias value of the input gate.

[0063] Output of forget gate:

[0064] (8)

[0065] in, is the weight matrix of the forget gate, is the bias value of the forget gate.

[0066] Output of candidate cells:

[0067] (9)

[0068] in, is the weight matrix of the internal memory unit, is the bias value of the internal memory unit, It is the intermediate state value, or the output value of the internal memory unit.

[0069] Calculation of output gate:

[0070] (10)

[0071] in, is the weight matrix of the output gate, is the output gate bias value.

[0072] Hidden state calculation:

[0073] (11)

[0074] in, is the output value of the LSTM layer, It is the state of memory cells.

[0075] Through linear transformation, the output of the encoder will be mapped to a new feature space to prepare for subsequent decoder processing.

[0076] Decoder:

[0077] like Figure 1 As shown on the right, the task of the decoder is to gradually generate the target sequence based on the representation vector generated by the encoder. The calculation process of the decoder is as follows:

[0078] Step 2.4 Input vector initialization: Use the end part of the input sequence as the initial input sequence of the decoder. The input sequence is expressed as:

[0079] (12)

[0080] in, is the starting vector of the input vector, is the starting index of the input sequence, is the ending index of the input sequence.

[0081] Step 2.5 Cross-Attention Layer: Temporal and positional encoding of the input vector:

[0082] (13)

[0083] in, Represents the input sequence Perform position coding and time coding. is the starting number of the input sequence, is the end number of the input sequence. is the input sequence The state vector after position and time encoding.

[0084] Afterwards, under the action of the multi-head cross attention mechanism, the correlation between the sequence to be predicted and the input sequence is calculated:

[0085] (14)

[0086] in, , , , respectively represent the query vector, key vector and value vector obtained by linear transformation, , and is the weight matrix of the linear transformation. Formula (14) expresses the association between the query vector of the decoder and the key vector of the encoder, and then the obtained weight is applied to the value vector of the encoder to generate the output of the decoder.

[0087] Step 2.6 LSTM network layer: At each time step, receives the vector from the encoder and combines it with the weighted input of the attention layer to generate the output sequence for the next time step:

[0088] (15)

[0089] in, is the decoder at time step The output vector of is the hidden state of the decoder at time step t+1. is the decoder at time step The output vector of It represents the output of the attention mechanism, which is used as the external input of LSTM after appropriate transformation.

[0090] Step S2.7 Linear transformation layer, LayerNorm layer and loss function: Linear transformation is performed on the output of the decoder to convert the feature space to the target space dimension. After that, the LayerNorm layer is applied for layer normalization to maintain the stability of model training. Finally, the mean absolute error (MAE) is used as the loss function, and the model weights are updated through the Adam optimizer to improve the accuracy of the prediction.

[0091] Step S3, training the deep cross LSTM network model and using the trained deep cross LSTM network model to predict the ship's navigation characteristics;

[0092] In actual maritime situations, most abnormal behaviors of ships can be reflected by the characteristics of ship trajectories, and the status of ship trajectories is mainly reflected in the position, speed and heading of the ship. The present invention comprehensively evaluates the position, heading and speed of the ship trajectory, converts the standard of abnormal detection into the deviation value of the three aspects of trajectory characteristics, and performs abnormal detection on non-compliant ship behaviors.

[0093] The ship behavior detection process includes the following: ① AIS data preprocessing. AIS historical data collection and cleaning to improve AIS data quality. ② Training the prediction model of deep cross LSTM to predict the four characteristics of the ship. ③ Real-time ship status anomaly detection. Compare the predicted status information of the ship (position, heading, speed) with the received AIS information at the same time. The detection and verification of the ship's position trajectory can be verified by remote sensing thermal infrared data. The specific process of marine ship behavior prediction and anomaly detection based on deep cross LSTM network is as follows: Figure 2 shown.

[0094] According to the AIS data, the current position (latitude and longitude), speed and heading of the ship are determined. By comparing the deviation between them and the predicted characteristics, it is possible to determine whether the ship's behavior is abnormal. For the ship's position characteristics, thermal infrared data can also be introduced to verify the original AIS position value, the predicted value and the thermal infrared data. Figure 3 Figures (a) and (b) show the normal and abnormal characteristics of the ship's position. The threshold can be adjusted according to the sea conditions and the actual performance of the ship.

[0095] Implementation steps:

[0096] In order to enable people in the technical field to better understand the solution of the present invention, the technical solution of the present invention will be clearly and completely described in combination with specific examples and the drawings in the examples of the present invention. The study area of ​​the example is a certain district, and the specific implementation steps are as follows:

[0097] Step 1. Obtain the ship’s AIS data and perform data preprocessing

[0098] Dataset ① comes from the AIS data of three ships, namely, an oil tanker (MMSI=412331140) from 20:44 to 22:44 on July 12, 2022, and two ships from 21:04 to 22:21 on March 7, 2023 (MMSI=414343000) and 20:21 to 22:21 on March 7, 2023 (MMSI=538004382). The track table contains MMSI (primary key), collection time, location (longitude, latitude), speed, and heading. Some static information of the three ships is shown in Table 1. After data preprocessing, the three routes of dataset ① are as follows: Figure 4 As shown in (a), (b), and (c).

[0099] Table 1 Dataset ① Static information of ships

[0100]

[0101] Datasets ② and ③ were collected at a port between 20:14 and 22:06 on February 25, 2023. The ship name is PING XIANG16, and the MMSI is 413631000. The route diagrams of datasets ② and ③ are shown in Figure 5 (a). In addition to AIS data, dataset ③ also includes thermal infrared data at 21:14:04 on February 25, 2023 (Beijing time), as shown in Figure 5 (a). Figure 5 as shown in (b).

[0102] The time interval for AIS equipment to update track data is from a few seconds to a few minutes. Generally, the track points are denser at faster speeds, which will generate more data and increase the calculation amount of track prediction. However, the track points are sparser at slower speeds, which is not conducive to track prediction. The AIS data is homogenized by interpolation (interpolation interval is 10s), which is conducive to the accuracy of ship track prediction and facilitates the estimation of track prediction error at each step.

[0103] For the error evaluation index between the predicted trajectory points and the actual trajectory points, the mean absolute error (MAE) and the mean square error (RMSE) are used for evaluation and calculation, which are expressed as:

[0104] (16)

[0105] (17)

[0106] in, and Represents the predicted trajectory point and coordinate, and The actual trajectory point and coordinate, Represents the total number of trajectory points.

[0107] Step 2. Predict ship navigation characteristics

[0108] Predict the navigation characteristics of ships, including location (latitude and longitude), speed and heading. The present invention divides the navigation conditions of ships at sea into two scenarios, and proposes two prediction methods based on LSTM-DCross network for the two scenarios.

[0109] 1. Joint prediction method for GPS signal failure scenarios

[0110] In the case of GPS failure and signal loss, but the ship's speed and heading sensors are still working properly, the ship's GPS positioning information (i.e. longitude and latitude) in the future is predicted. For this scenario, the joint prediction method assumes that the ship's GPS information fails after time t. This method analyzes the AIS data before time t, obtains network parameters, and combines accurate AIS ship position information to predict the ship's speed and heading in a period of time after time t.

[0111] The prediction process based on the joint prediction method is as follows: Assume that the training sample is , Indicates the time period for selecting the training set. is the input value of the model. The specific experimental steps are as follows:

[0112] a) The data of three ships are selected to train the LSTM-DCross neural network model. The training set, validation set, and test set are divided into 7:2:1. Data set ① is used as the data source.

[0113] b) Use the first 70% of the data in the training set to train the model.

[0114] c) Based on the speed and heading of the prediction set, the longitude and latitude are predicted. The BP neural network, LSTM-ATT network and the LSTM-DCross network proposed in this paper are used for prediction. The MAE and RMSE indicators are used to evaluate and calculate the prediction effects of the three networks.

[0115] The data set ① uses 70% of the ship behavior feature sequence data at 317, 498, and 530 moments as the initial input of the network, and uses the joint prediction method to predict the position information (latitude and longitude) of the three ships. The same training and test data are used to train and test the BP neural network and LSTM-ATT network. The prediction results of the three models for the longitude and latitude of the ship are compared. Figure 6 shown. Figure 6 (a) Figure 6 (b) Figure 6 (c) is a comparison of the track prediction of the ship (MMSI=412331146) by three networks. Figure 6 (d) Figure 6 (e), Figure 6 (f) is the comparison of the track prediction of the ship (MMSI=414343000) by the three networks. Figure 6 (g), Figure 6 of (h), Figure 6(i) is a comparison of the three networks for ship (MMSI=538004382) track prediction. It can be seen intuitively that at most times, the LSTM-ATT network and the deep cross LSTM network (LSTM-DCross) have better ship track prediction effects, which are significantly better than the BP neural network. The prediction effect of the LSTM-DCross network is the most accurate.

[0116] The predicted values ​​of longitude and latitude by the joint prediction method depend on the speed and heading data at the prediction time. Therefore, it is suitable for application scenarios where the GPS is damaged but the speed and heading sensors are working normally.

[0117] 2. Historical data prediction method for the scenario of complete AIS information failure

[0118] In the case that the ship's GPS, speed and heading sensors all fail or lose signals, the ship's GPS positioning information (i.e. longitude and latitude), speed and heading in the future are predicted. For this scenario, the historical data prediction method assumes that the ship's AIS information is completely ineffective after time t. This method relies entirely on data information at past times and can predict the ship's navigation status for a period of time after time t, including longitude and latitude, speed and heading, based on the ship's position (longitude and latitude), speed and heading information before time t.

[0119] The network prediction process based on historical data prediction method is as follows:

[0120] a) Select the initial time of the data set. With a time granularity of 10s, select the data of one ship as the input of the LSTM-DCross neural network model. ,and A set of input values ​​representing latitude, longitude, speed and heading.

[0121] b) Predicting the future The latitude and longitude as well as the speed and heading values ​​within the time period. The input and output data volumes are determined based on the adaptive dynamic window selection, that is, according to the speed, the greater the speed, the greater the value of the input data volume, and vice versa. After testing and analysis, according to the speed of the experimental sample (the specific value range is between 10 / kn and 10.5 / kn), the input and output data volumes are determined to be 130 and 60, that is, 130 moments (about 21 minutes), The predicted time is 60 time periods (about 10 minutes). The Lianyungang ships (MMSI=413631000) in the experimental time range of data set ② of the present invention within the time period of 20:31:37 - 21:02:06 on February 25, 2023 are predicted to be: the time period of 21:02:36 - 21:16:48 on February 25, 2023.

[0122] The BP neural network, LSTM-ATT and the LSTM-DCross network proposed in this invention are used to predict the input sequence, and the prediction results are the actual ship position (latitude and longitude), speed and heading. In order to compare the effectiveness of the deep cross LSTM network, this experiment simultaneously trained the BP and LSTM-ATT comparison models. The settings of each network parameter are shown in Table 2. The experimental results are compared. Figure 7 (a), (b), (c), Figure 8 (a), (b), (c) and Fig. 9 As shown in (a), (b), and (c).

[0123] Table 2 Network parameter settings for historical data prediction method

[0124]

[0125] The error comparison and analysis of each network is shown in Table 3. Based on the error index, the performance comparison radar chart of each network is shown in Table 3. Fig.10 It can be clearly seen that the LSTM-DCross network has the best prediction performance for ship navigation characteristics (latitude and longitude, speed, and heading), and the LSTM-ATT network also has good prediction accuracy, which is significantly higher than the prediction effect of the BP network.

[0126]

[0127] Table 3 Network error comparison

[0128] Step 3: Ship track verification based on multi-source data

[0129] Taking the Lianyungang ship (MMSI=413631000) in the time period of 20:32:25-21:03:36 on February 25, 2023 in data set ② as an example, the ship trajectory points are verified by combining multi-source data - the original AIS value, the corresponding predicted value based on historical data LSTM-DCross, and thermal infrared data. The experimental results of the historical prediction method based on LSTM-DCross for this data set in the time period of 21:04:06-21:17:26 on February 25, 2023 are as follows: Fig.11 As shown in (a), (b), and (c).

[0130] Fig.12The following is the thermal infrared data of a port on February 25, 2023, where the gray track represents the track of the target ship with MMSI = 423631000. Combining the AIS original data and predicted data of the ship at 13:14:4.370339 (UTC), it can be seen intuitively from ArcMap that the positions of the AIS initial value, the corresponding predicted value, and the thermal infrared value are similar, as shown in Figure 1. Fig.13 As shown, it can be concluded that the prediction effect is accurate and the ship is sailing normally. After measurement by ArcMap tool, the distance range between AIS track point and thermal infrared point (target ship) is within 200 meters, and the distance range between LSTM-DCross network prediction point and thermal infrared point (target ship) is within 400 meters.

[0131] After the above steps, the ship behavior feature prediction model based on the LSTM-DCross network proposed in the present invention shows high prediction accuracy in terms of the mean absolute error (MAE) and mean square error (MSE), two key indicators for measuring prediction accuracy. The prediction accuracy of different models is ranked from high to low as follows: LSTM-DCross network, LSTM-ATT network, and BP neural network. At the same time, the thermal infrared data, AIS data and their prediction values ​​based on the deep cross algorithm can be mutually verified, proving the effectiveness of the present invention.

Claims

1. A ship trajectory prediction and anomaly detection method based on deep cross LSTM network, characterized in that: The method comprises the following steps: Step S1, preprocessing AIS data to extract ship navigation characteristics; Step S2, constructing a deep cross LSTM network model based on the attention mechanism and the encoder-decoder, including: in the encoder part, converting the input multivariate time series data into a high-dimensional vector representation, and combining the multi-head attention mechanism and the LSTM network to capture the dependencies and periodic changes between the time series data, so as to enhance the network model's learning of data relationships and improve computational efficiency; Step S3: train the deep cross LSTM network model and use the trained deep cross LSTM network model to predict the ship's navigation characteristics.

2. The method according to claim 1, characterized in that In step S1, the cubic spline interpolation method is used to fill the data, and the mean variance normalization method is used to normalize the AIS data.

3. The method according to claim 1, characterized in that In step S2, in the encoder part, the input multivariate time series data is converted into a high-dimensional vector representation, and the multi-head attention mechanism and LSTM network are combined to capture the dependencies and periodic changes between the time series data. The specific steps are as follows: Step S2.1, multivariate time series data encoding conversion; in the decoder part, the end part of the input sequence is used as the initial input vector to start generating the sequence to be predicted; Step S2.2, learning the sequence data relationship based on the attention mechanism and using the LayerNorm layer for normalization; Step S2.3, LSTM network processing and linear transformation; Step S2.4, input vector and use the end part of the input sequence as the input vector of the decoder; Step S2.5: In the cross-attention layer, the input vector is temporally and positionally encoded; Step S2.6, in the LSTM network layer, at each time step, the vector from the encoder is received and combined with the weighted input of the attention layer to generate the output sequence for the next time step; Step S2.7, linear transformation layer, LayerNorm layer and loss function, including: performing a linear transformation on the output of the decoder to convert the feature space into the target space dimension; Afterwards, the LayerNorm layer is applied for layer normalization to maintain the stability of model training.

4. The method according to claim 3, characterized in that Step S2.1, multivariate time series data encoding conversion, including: first, the input multivariate time series data Perform encoding conversion, where Represents the length of the time series, each It is a four-dimensional vector, including longitude, latitude, speed and heading. The input multivariate time series data is represented by the continuous AIS data of the ship. The position and time coding vector is generated by coding the position and time. The formula is expressed as follows: （1） in, It means generating the corresponding position and time encoding vector for each time point in the input sequence. The specific formula is as follows: （2） （3） in, represents the time step, Represents the time step location, Represents the model dimension. By performing sine and cosine transforms of different frequencies on the input sequence, the model can capture the timing information at different time steps of the sequence.

5. The method according to claim 4, characterized in that Step S2.2, learning sequence data relationships and LayerNorm layer normalization based on the attention mechanism; including using the attention mechanism to learn the interdependence of information keys at each moment in the sequence data, and enhancing the model's understanding and representation capabilities of the input sequence; when processing the input sequence data, the multi-head attention mechanism simultaneously pays attention to information from different parts, so that it can calculate multiple attention distributions in parallel and learn the correlation between different parts of the sequence data. This part of the calculation process is expressed by the formula: （4） in, , , Respectively represent the query vector, key vector and value vector obtained by linear transformation, , and is the weight matrix of the linear transformation, is the hidden state output by the encoder, is the dimension of the key vector; The multi-head attention mechanism is calculated as follows: （5） in, is the output weight matrix of the multi-head attention mechanism, Represents the attention distribution of each head; the multi-head attention mechanism learns the correlation between different parts of the input sequence by parallelizing multiple attention heads; Afterwards, the LayerNorm layer is applied to normalize the output of the encoder to maintain the stability of model training, avoid gradient vanishing or exploding, and accelerate the convergence of the model.

6. The method according to claim 5, characterized in that Step S2.3, LSTM network processing and linear transformation, including: using the LSTM network to process the output of the encoder to capture the long-term dependency between the input sequence data; the LSTM network remembers a certain amount of information at each time step and forgets unnecessary information, so as to better capture the relationship between the input sequence data and remember historical information. The calculation process of the LSTM network is as follows: LSTM memory cell update: （6） in, It is the coefficient to adjust the memory state value of the last calculation. is the memory state value of the last calculation, is the intermediate state value or the internal memory unit output value, is the input gate calculated value, is the output state value; Output of the input gate : （7） in, is the weight matrix of the input gate, is the output value of the last calculation, is the bias value of the input gate; Output of forget gate : （8） in, is the weight matrix of the forget gate, is the bias value of the forget gate; Output of candidate cells : （9） in, is the weight matrix of the internal memory unit, is the bias value of the internal memory unit; Calculation of output gate: （10） in, is the weight matrix of the output gate, is the output gate bias value; Hidden state calculation: （11） in, is the hidden state, It is the memory cell state. After that, a linear transformation is performed to map the output of the encoder to a new feature space in preparation for subsequent decoder processing.

7. The method according to claim 6, characterized in that Step S2.4, input vector, use the end part of the input sequence as the input vector of the decoder: （12） in, is the starting vector of the input vector, is the starting number of the input sequence, The end number of the input sequence.

8. The method according to claim 7, characterized in that Step S2.5, cross attention layer; temporal and positional encoding of the input vector: （13） in, Represents the input sequence Perform position coding and time coding. is the starting number of the input sequence, is the end number of the input sequence. is the input sequence, The state vector after the position and time encoding stage; Afterwards, under the action of the multi-head cross attention mechanism, the correlation between the sequence to be predicted and the input sequence is calculated: （14） in, , , , respectively represent the query vector, key vector and value vector obtained by linear transformation, , and is the weight matrix of the linear transformation; Formula (14) expresses the association between the decoder's query vector and the encoder's key vector, and then the obtained weight is applied to the encoder's value vector to generate the decoder's output.

9. The method according to claim 8, characterized in that Step S2.6, LSTM network layer; including: at each time step, receiving the vector from the encoder and combining it with the weighted input of the attention layer to generate the output sequence for the next time step: （15） in, is the decoder at time step The output vector of is the decoder at time step The hidden state of is the decoder at time step The output vector of It represents the output of the attention mechanism, which is used as the external input of LSTM after appropriate transformation.

10. The method according to claim 9, characterized in that In step S3, the ship's navigation characteristics are predicted, including the position, i.e., longitude and latitude, speed, and heading; the ship's navigation conditions are divided into two scenarios, and two prediction methods are proposed for the two scenarios; specifically: Scenario 1: Joint prediction method for GPS signal failure scenario; In the event of GPS failure and signal loss, but the ship's speed and heading sensors are still working properly, the ship's GPS positioning information, i.e. longitude and latitude, for a period of time in the future can be predicted; For this scenario, the joint prediction method assumes After the time, the ship's GPS information becomes invalid. This method analyzes the time The previous AIS data, network parameters, and accurate AIS ship position information are combined to predict the ship's position at the time. Predict the speed and course for a period of time thereafter; Scenario 2: Historical data prediction method for the scenario of complete failure of AIS information; In the case that the ship's GPS, speed and heading sensors all fail or lose signals, the ship's GPS positioning information, i.e., longitude and latitude, speed and heading in the future, is predicted; for this scenario, the historical data prediction method assumes that After the time, the ship AIS information is completely invalid. This method relies on the data information of the past time. The previous position, longitude, speed and heading information of the ship are used to predict the ship’s The navigation status within a period of time after the time, including longitude and latitude, speed and heading.

11. A ship trajectory prediction and anomaly detection device based on a deep cross LSTM network, characterized in that: The device comprises: AIS data preprocessing module, which preprocesses AIS data and extracts ship navigation characteristics; The network model building module builds a deep cross LSTM network model based on the attention mechanism and encoder-decoder. In the encoder part, the input multivariate time series data is converted into a high-dimensional vector representation. At the same time, the multi-head attention mechanism and LSTM network are combined to capture the dependencies and periodic changes between time series data, so as to enhance the network model's learning of data relationships and improve computational efficiency. The ship navigation characteristics prediction module trains the deep cross LSTM network model and uses the trained deep cross LSTM network model to predict the ship navigation characteristics.

12. An electronic device, characterized in that: include: one or more processors; A memory for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors are enabled to implement the method according to any one of claims 1 to 10.

13. A computer-readable storage medium, characterized in that: Executable instructions are stored thereon, and when the instructions are executed by a processor, the processor implements the method according to any one of claims 1 to 10.

Images