Ship automatic identification system navigation track prediction method

Abstract

The application relates to a ship automatic identification system navigation track prediction method, which first adopts a SimCTSE time sequence training method to train a historical AIS data set, enhances the ability of an encoder in a CLC-Transformer model to extract time sequence features, thereby enhances a prediction effect, and improves ship navigation track prediction accuracy. The CLC-Transformer model proposed in the application has a plurality of one-dimensional convolution layers, a self-attention mechanism layer, and introduces a residual connection and a batch normalization structure, so that the model has a high nonlinear processing capability. Through a training strategy based on region division, the application binds the output feature vector group of the model and the target potential motion direction together, divides the possible motion region of the target in the future, so that the space searched by the model after division is reduced, and it is easier to optimize through a gradient descent algorithm. The method of the application simplifies the model training process, enhances the prediction effect of the model, and thereby improves the prediction accuracy of the ship navigation track.

Description

Technical Field

[0001] This invention relates to a ship trajectory prediction technology, and more particularly to a method for predicting the trajectory of an Automatic Identification System (AIS) based on the CLC-Transformer model. Background Technology

[0002] With the rapid development of the world economy, maritime transportation activities, as an important part of modern transportation, are becoming increasingly busy and complex. As a result, the Automatic Identification System (AIS) has accumulated a large amount of shipping data. Analyzing this shipping data helps decision-makers obtain navigation information of target vessels, promptly identify potential safety hazards, and formulate corresponding measures, thereby ensuring the safe navigation of ships and the normal production and operation of ports.

[0003] Existing ship trajectory prediction methods fall into two categories: those based on traditional machine learning and those based on deep learning. Traditional machine learning-based methods tend to manually analyze target motion patterns to determine model parameters and thresholds. However, this approach ignores the hidden relationships between input and output in the target ship trajectory prediction task, resulting in limited accuracy. Deep learning-based methods rely on the model's nonlinear processing capabilities for accuracy. However, past deep learning-based methods typically used relatively simple models with limited capabilities. Therefore, these models often produce similar or identical outputs for different inputs, exhibiting an output pattern averaging problem. This makes them unable to fit the target ship's trajectory, resulting in a large discrepancy between the predicted trajectory and the actual value, leading to low accuracy in ship trajectory prediction. Summary of the Invention

[0004] To address the issue of low prediction accuracy in existing ship trajectory prediction methods, a new method for predicting the navigation trajectory of an Automatic Identification System (AIS) is proposed.

[0005] The technical solution of this invention is: a method for predicting the navigation trajectory of a ship's automatic identification system, specifically including the following steps:

[0006] S1. Obtain the relative current selected historical ship AIS operation status data of the target ship to be predicted, input the historical AIS operation status data of the target ship to be predicted into the trained CLC-Transformer model, and obtain the directional feature vector group containing time information.

[0007] The CLC-Transformer model includes: an encoder and a decoder.

[0008] S101. Obtain the historical AIS operating status data of the selected target ship to be predicted, input it into the encoder, and obtain the motion time sequence characteristics of the target.

[0009] S102. Input the preset ship direction feature vector set and the target motion timing features output by the encoder end into the decoder end to obtain the direction feature vector set containing timing information;

[0010] S2. Based on the directional feature vector group containing temporal information, associate a directional feature vector containing temporal information with the potential motion region of the predicted target.

[0011] S3. Input the directional feature vector containing time-series information obtained in S2 and the final navigation point in the preset target ship AIS navigation data into the trajectory decoder to obtain the predicted ship AIS navigation trajectory.

[0012] Furthermore, the encoder-side model structure in the CLC-Transformer model includes: a block-based long short-term memory network (C-LSTM) and a MixResNet;

[0013] The C-LSTM structure, with a parameter-shared Long Short-Term Memory (LSTM) network at its core, first pads the data with zeros according to the data length and the preset number of segments, ensuring that the length of the padded temporal data is divisible by the number of segments. Then, the input temporal data is converted into multiple temporal data blocks according to the number of segments, and these blocks are fed into the LSTM network. C-LSTM is responsible for pre-extracting temporal data features and converting variable-length temporal data into temporal embedding data with a fixed number of blocks and a fixed embedding dimension, enabling the subsequent MixResNet to process the variable-length temporal data. MixResNet consists of three one-dimensional residual convolutional branches with different scales of 3, 5, and 7. This structure re-extracts features from the data at different convolutional scales, and then performs channel fusion of the features extracted from the three branches through a subsequent 1x1 convolutional Mix layer. The output is then obtained through a fully connected layer. MixResNet is responsible for multi-scale feature extraction and fusion of temporal data to obtain better temporal feature embeddings.

[0014] Furthermore, the encoder-side model training method in the CLC-Transformer model is as follows:

[0015] 1) Data Processing: Historical AIS operational status time-series data of the target vessel to be predicted is acquired for model training. The time-series data is divided to obtain anchor point data and neighborhood data, specifically:

[0016] 1.1) Acquire time series data, according to a preset fixed window W n The time series data is segmented by size to obtain the segmented time series data;

[0017] 1.2) In each fixed window W nThe center of the data generates anchor data W according to the preset anchor data length. anc Then, within a selectable range of values ​​from the anchor data center to its left or right, the center of the neighborhood data is determined, and neighborhood data W is generated according to the preset neighborhood data length. pos And ensure that the neighborhood W pos The fixed window data W will not be retrieved. n In addition, the length of the neighborhood data is the same as the length of the anchor data, thus obtaining a set of anchor data and a set of neighborhood data;

[0018] Among them, anchor data and neighborhood data within the same window are a pair of positive samples, while neighborhood data and anchor data in different windows are a pair of negative samples;

[0019] 2) Encoder training: The SimCTSE method is used to pre-train the encoder model in the CLC-Transformer model using anchor data and neighborhood data. The pre-trained temporal model is the encoder in the CLC-Transformer model, specifically:

[0020] 2.1) Input the generated anchor data and neighborhood data into the CLCE that needs to be pre-trained, and let the encoder model embed the anchor data and neighborhood data to obtain anchor embedding and neighborhood embedding data, where the data at position i' of the anchor embedding and the data at position i' of the neighborhood embedding are a pair of positive embedding samples.

[0021] 2.2) Perform join and copy operations on the anchor embedded data and the neighborhood embedded data to obtain the Query matrix and Key matrix:

[0022] First, the tail of an anchor embedding in each pair of embedded positive samples and the head of a neighbor embedding are connected to obtain the Query matrix;

[0023] Then, copy the Query matrix to obtain the Key matrix;

[0024] 2.3) Calculate the cosine similarity between each vector in the Query matrix and each vector in the Key matrix, and remove cosine values ​​of identical embedded data to obtain the cosine similarity matrix Logits between the anchor embedded data and the neighbor embedded data; the formula for calculating cosine similarity is as follows:

[0025]

[0026] Where x1 is any value in the Query matrix and x2 is any value in the Key matrix;

[0027] Each value in the i-th row of the Logits matrix is ​​the cosine similarity between the i-th embedded data in the Query matrix and each embedded data in the Key matrix;

[0028] 2.4) Generate labels for each row vector in the Logits matrix based on the positive sample pair relationship between the embedded anchor data and the embedded neighborhood data:

[0029] First, for the vector group in the i-th row of the Logits matrix, each column vector in this vector group is the cosine similarity between the embedding vector in the i-th row of the Query matrix and each embedding vector in the Key matrix;

[0030] Then, for the i-th row vector group in the Logits matrix, according to the positive sample pair relationship, a column vector is selected from the row vector group. This column vector represents the cosine similarity between the i-th embedding vector in the Query and the vector in the Key matrix that can form a positive sample pair relationship with the i-th embedding vector in the Query. The number of this column vector is used as the label of the i-th row vector group.

[0031] In this context, the value at position (i, j) in the Logits matrix represents the cosine similarity between the i-th embedding vector in the Query matrix and the j-th embedding vector in the Key matrix. The generated labels are used for parameter optimization during model training. During the optimization process, the labels tell the model which position in each row of the Logits matrix should have the highest cosine similarity. The model is learning how to maximize the cosine similarity of the label positions.

[0032] The cross-entropy function is used as the loss function for pre-training the editor model, and the pre-trained time series model is finally obtained.

[0033] 2.5) Using the Logits matrix obtained in 2.3 and the labels obtained in 2.4, a learning task is performed to maximize the cosine similarity of the label positions, thereby obtaining the trained encoder.

[0034] Furthermore, the specific implementation method of step S102 is as follows: the encoder-side model trained in the CLC-Transformer model extracts features from the target's past AIS running data, obtains the target's motion temporal features, and passes these features as the Key and Value of the attention mechanism to the decoder-side of the CLC-Transformer model; the decoder-side converges the target motion temporal features from the encoder-side through self-attention and attention mechanisms, and converges these features into the target's potential motion direction vector group, which serves as the final output of the CLC-Transformer model.

[0035] Furthermore, the decoder-side structure of the CLC-Transformer model includes, in sequence: a word embedding layer, a first multi-head self-attention layer, a first addition and normalization layer, a second multi-head attention layer, a second addition and normalization layer, a fully connected layer, and a third addition and normalization layer.

[0036] First, the preset ship direction feature vector set is initialized using a word embedding layer to obtain the initialized direction feature vector set; the number of vectors in the initialized direction feature vector set is the same as the number of potential directions of target motion.

[0037] Then, the initialized directional feature vector group is input into the first multi-head self-attention layer. Each attention head undergoes three different linear transformations to obtain the Key, Query, and Value vectors in the attention mechanism. The Query and Key vectors are first scaled and dot-producted to obtain the attention score. The attention score is then multiplied by the Value vector to obtain the output of the self-attention layer.

[0038] Then, the output of the first multi-head attention layer passes through the first addition and normalization layer, and the output of the self-attention layer is directly added to the input of the previous self-attention layer and then undergoes layer normalization processing.

[0039] Then, the above operation is repeated, but at this time the input of Key and Value in the second multi-head attention layer is the output of the encoder, and the input of Query is the output of the first addition and normalization layer. After calculation, the output is obtained and passed through the second addition and normalization layer, but at this time the added value is the output of the first addition and normalization layer.

[0040] Finally, the data undergoes a nonlinear transformation through three fully connected layers without changing the dimension of the input data. The output of the fully connected layers is then fed into the third addition and normalization layer to obtain the final output.

[0041] Furthermore, in step S2, based on the directional feature vector group containing temporal information, a directional feature vector containing temporal information is associated with the potential motion region of the predicted target. The specific association method is as follows: S201, Based on the positional relationship between the first point in the input temporal data of the CLC-Transformer model and the endpoint in the preset AIS data, a region feature selection matrix is ​​generated. The matrix has 1 row and the number of columns is the same as the number of output vectors of the CLC-Transformer model. The matrix consists of multiple 0s and one 1.

[0042] S202. Perform matrix multiplication on the region feature selection matrix obtained in S201 and the feature vector group output by the CLC-Transformer model. The resulting matrix is ​​a directional feature vector containing temporal information.

[0043] Furthermore, the trajectory decoder includes: LSTM neurons and fully connected layers;

[0044] The directional feature vector containing temporal information obtained by S2 and the endpoint trajectory in the preset AIS data are input into the LSTM neuron. The data output by the LSTM neuron is input into the fully connected layer. The fully connected layer outputs the target's future trajectory at time n. The target's future trajectory at time n is input back into the LSTM neuron. The data output by the LSTM neuron is input into the fully connected layer. The fully connected layer outputs the target's future trajectory at time n+1. This process is repeated to obtain the target's trajectory over a future period.

[0045] A CLC-Transformer model for predicting navigation trajectories in an Automatic Identification System (AIS), comprising an encoder and a decoder;

[0046] The encoder structure includes: a block-based long short-term memory network C-LSTM and a MixResNet; the block-based long short-term memory network C-LSTM pre-extracts the features of the time series data and converts the variable-length time series data into time series embedded data with a fixed number of blocks and a fixed embedding dimension, which is then sent to the MixResNet; the MixResNet processes the variable-length time series data and performs multi-scale feature extraction on the time series data.

[0047] The encoder inputs historical AIS operational status data of the target vessel to obtain the target's motion temporal features. The decoder structure includes, in sequence: a word embedding layer, a first multi-head self-attention layer, a first addition and normalization layer, a second multi-head attention layer, a second addition and normalization layer, a fully connected layer, and a third addition and normalization layer. It receives the target's motion temporal features output by the encoder and aggregates the target's motion temporal features into the target's potential motion direction vector group to obtain a direction feature vector group containing temporal information.

[0048] Preferably, the block-based long short-term memory network C-LSTM is based on a parameter-shared long short-term memory network, which is responsible for the pre-extraction of temporal data features and converts variable-length temporal data into temporal embedded data with a fixed number of blocks and a fixed embedding dimension, so that the subsequent MixResNet can process the variable-length temporal data.

[0049] The MixResNet consists of three one-dimensional residual convolutional branches at different scales, which extract features from the data at different convolutional scales. The features extracted by the three branches are then fused through a subsequent 1x1 convolutional Mix layer, and the output is obtained through a fully connected layer. MixResNet is responsible for multi-scale feature extraction and fusion of temporal data to obtain better temporal feature embedding.

[0050] Preferably, the word embedding layer at the decoder initializes a preset set of required ship direction feature vectors to obtain an initialized set of direction feature vectors; the number of vectors in the initialized set of direction feature vectors is the same as the number of potential directions of target motion.

[0051] The beneficial effects of this invention are as follows: The ship automatic identification system (AIS) trajectory prediction method first employs the SimCTSE (Simple Contrastive Learning of Time Series Embedding) temporal model pre-training method. It fully utilizes data from the AIS dataset to pre-train the CLCE (Chunked LSTM Convolution Encoder) in the CLC-Transformer model, enhancing the CLCE's ability to extract temporal features. This strengthens the prediction performance of the CLC-Transformer model, making the predicted ship trajectories more accurate and improving prediction accuracy. The proposed CLC-Transformer model features multiple one-dimensional convolutional layers, a self-attention mechanism layer, and incorporates residual connections and batch normalization, giving the model high nonlinear processing capabilities. Furthermore, this invention uses a region-partitioning-based training strategy to bind the output feature vectors of the CLC-Transformer model to the target's potential motion direction, dividing the target's future possible motion regions. This reduces the search space of the model after partitioning, making it easier to optimize using gradient descent algorithms. This invention simplifies the model training process while enhancing the model's predictive performance, thereby improving the accuracy of ship trajectory prediction. Attached Figure Description

[0052] Figure 1 This is a schematic diagram of the backbone structure of the CLC-Transformer model of the present invention;

[0053] Figure 2 This is a schematic diagram of positive and negative samples of anchor point data and neighborhood data in this invention;

[0054] Figure 3 This is a schematic diagram of the time series model pre-training process based on SimCTSE according to the present invention;

[0055] Figure 4 This is a schematic diagram of the region-based training strategy of the present invention;

[0056] Figure 5 This is a schematic diagram of the trajectory decoder of the present invention;

[0057] Figure 6This is a diagram showing the trajectory prediction effect of the in20pred10 invention.

[0058] Figure 7 This is a diagram showing the trajectory prediction effect of the in30pred15 model of the present invention.

[0059] Figure 8 This is a diagram showing the trajectory prediction effect of the in40pred20 model of the present invention.

[0060] Figure 9 This is a diagram showing the trajectory prediction effect of the in50pred25 model of the present invention. Detailed Implementation

[0061] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments. These embodiments are based on the technical solution of the present invention and provide detailed implementation methods and specific operating procedures. However, the scope of protection of the present invention is not limited to the following embodiments.

[0062] The specific process of the ship automatic identification system (AIS) navigation trajectory prediction method based on the CLC-Transformer model in this embodiment is as follows:

[0063] S1. Obtain the selected historical AIS operating status data of the target ship to be predicted relative to the current situation. Input the historical AIS operating status data of the target ship to be predicted into the trained CLC-Transformer model to obtain the directional feature vector group containing time series information.

[0064] The historical AIS operating status data of the target vessel can be manually defined and is related to the time interval of the current AIS equipment data collection and the amount of AIS data input, including: longitude coordinates, latitude coordinates, speed, and heading;

[0065] like Figure 1 The schematic diagram of the CLC-Transformer model backbone structure shown includes: encoder end and decoder end;

[0066] S101. Obtain the historical AIS operating status data of the selected target vessel to be predicted, input it into the encoder, and obtain the motion time sequence characteristics of the target.

[0067] S102. Input the preset ship direction feature vector set and the target motion timing features output by the encoder end into the decoder end to obtain the direction feature vector set containing timing information;

[0068] The decoder includes, in sequence: a word embedding layer, a first multi-head self-attention layer, a first addition and normalization layer, a second multi-head attention layer, a second addition and normalization layer, a fully connected layer, and a third addition and normalization layer.

[0069] S2. Based on the directional feature vector group containing temporal information, associate a directional feature vector containing temporal information with the potential motion region of the predicted target.

[0070] S3. Input the directional feature vector containing time-series information obtained in S2 and the final navigation point in the preset target ship AIS navigation data into the trajectory decoder to obtain the predicted ship AIS navigation trajectory.

[0071] AIS track data contains a large amount of data lacking labels, so past implementation methods could not make full use of AIS track data. This implementation method can make full use of this unlabeled track data to train the model through the SimCTSE method, thus solving the pain point of previous methods.

[0072] The encoder in the CLC-Transformer model is trained in the following way:

[0073] 1. Data processing: such as Figure 2 As shown, historical AIS operational status time-series data of the target vessel to be predicted is obtained for model training. The time-series data is divided to obtain anchor point data and neighborhood data, specifically:

[0074] 1.1 Acquire time-series data, according to a preset fixed window W n The time series data is segmented by size to obtain the segmented time series data;

[0075] 1.2, in each fixed window W n The center of the data generates anchor data W according to the preset anchor data length. anc Then, within a selectable range of values ​​from the anchor data center to its left or right, the center of the neighborhood data is determined, and neighborhood data W is generated according to the preset neighborhood data length. pos And ensure that the neighborhood W pos The fixed window data W will not be retrieved. n In addition, the length of the neighborhood data is the same as the length of the anchor data, thus obtaining a set of anchor data and a set of neighborhood data;

[0076] Among them, anchor data and neighborhood data within the same window are a pair of positive samples, while neighborhood data and anchor data in different windows are a pair of negative samples;

[0077] like Figure 2 As shown, the large frame represents the segmentation of time series data according to a fixed window size, such as a fixed segmentation window size of 50, and the segmented data is denoted as W. n The lightest gray box indicates that anchor data W is generated at the center of each fixed window of data according to the preset anchor data length. ancFor example, if the preset data length is 20, the darker gray box indicates that within the selectable value range from the anchor data center to its left and right sides, neighborhood data W is generated according to the preset neighborhood data length. pos And ensure neighborhood data W pos The fixed window data W will not be retrieved. n Furthermore, the length of the neighborhood data is the same as the length of the anchor data. Since time-series data within the same window are similar, anchor data and neighborhood data from the same window are positive samples to each other, while data from different windows are negative samples to each other. By processing time-series data using this method, a set of anchor data and a set of neighborhood data can be obtained.

[0078] 2. Encoder-side training: such as Figure 3 As shown, the SimCTSE method is used to pre-train the block-based convolutional encoder CLCE in the CLC-Transformer model using anchor data and neighborhood data. The pre-trained temporal model is the encoder end of the CLC-Transformer model, specifically:

[0079] 2.1 Input the generated anchor data and neighborhood data into the CLCE that needs to be pre-trained, and let the encoder model embed the anchor data and neighborhood data to obtain anchor embedding and neighborhood embedding data, where the data at position i' of the anchor embedding and the data at position i' of the neighborhood embedding are a pair of positive embedding samples.

[0080] like Figure 1 The CLCE model structure shown includes: Block Long Short-Term Memory Network (C-LSTM) and MixResNet;

[0081] The C-LSTM architecture, centered on a parameter-shared Long Short-Term Memory (LSTM) network, first pads the data with zeros according to the data length and the preset number of segments, ensuring that the length of the padded temporal data is divisible by the number of segments. Then, the input temporal data is converted into multiple temporal data blocks according to the number of segments, and each block is fed into the LSM network (since each temporal data block uses the same LSM network, it becomes a parameter-shared LSM network). C-LSTM is responsible for pre-extracting temporal data features and converting variable-length temporal data into temporal embedding data with a fixed number of blocks and a fixed embedding dimension, enabling the subsequent MixResNet to process variable-length temporal data.

[0082] MixResNet mainly consists of three one-dimensional residual convolutional branches at different scales (3, 5, and 7). This structure extracts features from the data at different convolutional scales, and then performs channel fusion of the features extracted from the three branches through a subsequent Mix layer (1x1 convolution), before passing through a fully connected layer to obtain the output. MixResNet is mainly responsible for multi-scale feature extraction and fusion of temporal data to obtain better temporal feature embeddings.

[0083] 2.2 Perform join and copy operations on the anchor embedded data and the neighborhood embedded data to obtain the Query matrix and Key matrix:

[0084] First, the tail of an anchor embedding in each pair of embedded positive samples and the head of a neighbor embedding are connected to obtain the Query matrix;

[0085] Then, copy the Query matrix to obtain the Key matrix;

[0086] 2.3. Calculate the cosine similarity between each vector in the Query matrix and each vector in the Key matrix, and remove cosine values ​​of identical embedded data to obtain the cosine similarity matrix Logits between the anchor embedded data and the neighbor embedded data. The formula for calculating cosine similarity is as follows:

[0087]

[0088] Where x1 is any value in the Query matrix and x2 is any value in the Key matrix;

[0089] Each value in the i-th row of the Logits matrix is ​​the cosine similarity between the i-th embedded data in the Query matrix and each embedded data in the Key matrix;

[0090] 2.4 Generate labels for each row vector in the Logits matrix based on the positive sample pair relationship between the embedded anchor data and the embedded neighborhood data:

[0091] First, for the vector group in the i-th row of the Logits matrix, each column vector in this vector group is the cosine similarity between the embedding vector in the i-th row of the Query matrix and each embedding vector in the Key matrix.

[0092] Then, for the i-th row vector group in the Logits matrix, based on the positive sample pair relationship, a column vector can be selected from this row vector group (this column vector represents the cosine similarity between the i-th embedding vector in the Query and the vector in the Key matrix that can form a positive sample pair relationship with the i-th embedding vector in the Query), and the number of this column vector is used as the label of the i-th row vector group.

[0093] In this model, the value at position (i, j) in the Logits matrix represents the cosine similarity between the i-th embedding vector in the Query matrix and the j-th embedding vector in the Key matrix. The generated labels are used for parameter optimization during model training. During optimization, the labels tell the model which position in each row of the Logits matrix should have the highest cosine similarity. The model learns how to maximize the cosine similarity at the label positions.

[0094] Therefore, for the first batch_size rows of the Logits matrix, considering that the similarity scores at the diagonal positions have been removed, the label numbers are (batch_size-1) to (2*batch_size-2), while for the last batch_size rows of the Logits matrix, the label numbers are 0 to (batch_size-1). At this point, training labels are generated based on the positive and negative sample pairs, and the labels are the classification labels for each row of the similarity matrix. In other words, the model pre-training problem is transformed into a (batch_size*2-1) classification problem for each row of the Logits similarity matrix.

[0095] This invention uses the cross-entropy function as the loss function for pre-training the CLCE model, ultimately obtaining the pre-trained time series model. The pre-training process is as follows: Figure 3 As shown;

[0096] Where 2*batch_size is the total number of rows in the Logits matrix.

[0097] 2.5. Using the Logits matrix obtained in 2.3 and the labels obtained in 2.4, a learning task is performed to maximize the cosine similarity of the label positions, thereby obtaining the trained encoder.

[0098] In this embodiment, the SimCTSE method is used to pre-train the CLCE model. SimCTSE is a self-supervised time series model training method based on the idea of ​​contrastive learning.

[0099] In step S102, the preset required direction feature vector set for the ship and the target motion timing features output from the encoder are input to the decoder to obtain a direction feature vector set containing timing information. The specific implementation method is as follows:

[0100] like Figure 1 The decoder shown uses a decoder in a Transformer network, which includes, in sequence: a word embedding layer, a first multi-head self-attention layer, a first addition and normalization layer, a second multi-head attention layer, a second addition and normalization layer, a fully connected layer, and a third addition and normalization layer.

[0101] First, the word embedding layer is used to initialize the preset ship direction feature vector set to obtain the initialized direction feature vector set.

[0102] The number of vectors in the initialized direction feature vector group is the same as the number of potential directions of target motion. In this invention, it is set to 8. If the target has eight directions of motion, the preset direction feature vector is (0,1,2,3,4,5,6,7).

[0103] The direction vector group is implemented by the embedding layer. The input of the embedding layer is a set of numbers starting from 0. Assuming that the number of regions is 8, the input of the embedding layer is (0,1,2,3,4,5,6,7). The embedding layer can set the embedding dimension of each number (let's assume it is 128-dimensional). Then the output of the embedding layer is an 8x128 matrix.

[0104] In the decoder of this invention, the embedding layer is used to convert a set of numbers representing region numbers into an embedding vector, which is the initialization of the directional feature vector set.

[0105] Then, the initialized directional feature vector group is input into the first multi-head self-attention layer. Each attention head undergoes three different linear transformations to obtain the Key, Query, and Value vectors in the attention mechanism. The Query and Key vectors are first subjected to scaling and dot product operations to obtain the attention score. The attention score is then multiplied by the Value vector to obtain the output of the self-attention layer.

[0106] Then, the output of the first multi-head attention layer is passed through the first addition and normalization layer, and the output of the self-attention layer is directly added to the input of the previous self-attention layer and then processed by layer normalization.

[0107] Then, the above operation is repeated, but this time the input of Key and Value in the second multi-head attention layer is the output of the encoder, and the input of Query is the output of the first addition and normalization layer. After calculation, the output is obtained again and passed through the second addition and normalization layer, but this time the added value is the output of the first addition and normalization layer.

[0108] Finally, the data undergoes a nonlinear transformation through three fully connected layers without changing the dimension of the input data. The output of the fully connected layers is then fed into the third addition and normalization layer to obtain the final output.

[0109] The target motion temporal features are the Key and Value of the second multi-head attention layer.

[0110] In this implementation, the encoder of the CLC-Transformer model extracts features from the target's past AIS running data using a pre-trained CLCE model, obtaining the target's temporal motion features. These features are then passed as the Key and Value of the attention mechanism to the decoder of the CLC-Transformer model. The decoder then aggregates the target's temporal motion features from the encoder using a self-attention mechanism and an attention mechanism, converging these features into a set of latent motion direction vectors of the target, which serves as the final output of the CLC-Transformer model.

[0111] like Figure 4 As shown, in step S2, based on the directional feature vector group containing temporal information, a directional feature vector containing temporal information is associated with the potential motion region of the predicted target. The specific association method is as follows:

[0112] S201. Based on the positional relationship (longitude and latitude coordinates) between the first point in the input time series data of the CLC-Transformer model and the final point in the AIS data, generate a region feature selection matrix. This matrix has 1 row and the same number of columns as the output vector of the CLC-Transformer model. The matrix consists of multiple 0s and one 1.

[0113] If the target movement has eight directions, a coordinate system is established with the first point as the center, and the coordinate system is divided into eight regions. The endpoint is located in the first region of the first point, then the region feature selection matrix is ​​[1 0 0 0 0 0 0 0];

[0114] S202. Perform matrix multiplication on the region feature selection matrix obtained in S201 and the feature vector group output by the CLC-Transformer model. The resulting matrix is ​​a directional feature vector containing temporal information that is associated with the potential motion region of the predicted target based on the directional feature vector group containing temporal information.

[0115] In this embodiment, since the feature selection matrix is ​​generated from the positional relationship between the endpoint and the first point in the input data, this method can bind the vector set output by the model to the potential motion region of the predicted target. This is because ships, due to their large mass, are unlikely to perform highly maneuverable turns such as U-turns in a short period of time during navigation. Therefore, the magnitude of the change in direction is usually not large. Thus, it can be assumed that the ship will maintain its original direction of motion or the magnitude of the change in direction will not be large in the future. That is, the endpoint in the input data can largely determine the future navigation direction of the target ship. This selection process is as follows: Figure 4 As shown.

[0116] like Figure 5As shown, the trajectory decoder includes: LSTM neurons and fully connected layers;

[0117] The directional feature vector containing temporal information obtained by S2 and the endpoint trajectory in the preset AIS data are input into the LSTM neuron. The data output by the LSTM neuron is input into the fully connected layer. The fully connected layer outputs the target's future trajectory at time n. The target's future trajectory at time n is input back into the LSTM neuron. The data output by the LSTM neuron is input into the fully connected layer. The fully connected layer outputs the target's future trajectory at time n+1. This process is repeated to obtain the target's trajectory over a future period.

[0118] To verify the beneficial effects of the present invention, the following experiments were conducted:

[0119] This embodiment uses an AIS dataset of fishing vessels from the East China Sea, compiled from historical tracking data of real fishing vessels at sea. The dataset stores vessel data in CSV format, with each CSV file containing data ranging from hundreds to tens of thousands of records. The data is collected by the Automatic Identification System (AIS) on the vessels and includes information such as the target vessel's ID, latitude and longitude, speed, direction of travel, and time. Because the length of each record in the vessel dataset varies significantly, making batch training of deep learning models inconvenient, fixed-length data segmentation is necessary. Four datasets were generated from the training set of the original data based on different segmentation lengths: in20-pred10, in30-pred15, in40-pred20, and in50-pred25. The segmentation window sizes for these four datasets are 30, 45, 60, and 75, respectively. Each segment contains the input data (20, 30, 40, 50) and the trajectory labels output by the CLC-Transformer model (10, 15, 20, 25). These four datasets will be used for model performance validation experiments. During the trajectory segmentation process, this invention offset each trajectory data and its label. Specifically, the initial point of each segment is defined as the origin, and subsequent trajectory data are modified with the offset relative to the origin. This method references the processing methods used in pedestrian and vehicle trajectory prediction, enabling the CLC-Transformer model to focus more on the target's motion pattern. The experimental data statistics are shown in Table 1.

[0120] Table 1

[0121]

[0122] To verify the predictive ability of the CLC-Transformer model for ship trajectories, experiments were conducted on four datasets: in20-pred10, in30-pred15, in40-pred20, and in50-pred25. These datasets represent input target sequence lengths of 20 and output sequence lengths of 10, 30 and 15, 40 and 20 respectively, and 50 and 25 respectively. The model uses the proposed CLC-Transformer model. The encoder of the CLC-Transformer model is a CLCE model pre-trained using the SimCTSE method, and the decoder is a composite structure of self-attention mechanism, attention mechanism, and fully connected layers. The input data consists of the target's past operational status information (longitude, latitude, speed, and heading) and a group of directional feature vectors initialized by the embedding layer. After a region segmentation selection strategy, the model finally outputs the directional feature vectors and passes them to the subsequent trajectory decoder to predict the target's trajectory.

[0123] The experiment uses two evaluation indicators, Average Displacement Error (ADE) and Final Displacement Error (FDE), to measure the model's ability to predict the target trajectory. Average Displacement Error is the average Euclidean distance difference between each predicted position and each corresponding true position, and Final Displacement Error is the average Euclidean distance difference between the predicted final position and the true final position. The calculation formulas for both are shown in formula (2) and formula (3).

[0124]

[0125]

[0126] Among them, t i Represents the actual trajectory at each time step. t represents the trajectory predicted by the model at each time step; final The true trajectory representing the final point. The value represents the final trajectory predicted by the model, and ||2 represents the L2 norm of the solution matrix, which is the Euclidean distance between the predicted trajectory and the actual trajectory value.

[0127] The experiment compared the LSTM encoder-decoder and the CNN+LSTM combined network. The LSTM encoder-decoder served as the baseline model, consisting of a single-layer Long Short-Term Memory (LSTM) encoder and a LSTM decoder. The encoder extracts temporal features and embeds them into the temporal data; this embedding is the hidden output of the LSTM network. The subsequent decoder is also an LSTM network. The initial input to the decoder is the coordinates of the final point in the input trajectory. Simultaneously, the hidden output of the encoder is used as the initial hidden state of the decoder's LSTM network. By setting a model prediction length, the decoder's output is iteratively transformed through a fully connected layer and used as the new decoder input, repeating this process until the output reaches the preset model prediction length. The CNN+LSTM combined network is currently considered a high-performing model in ship target trajectory prediction. This model first processes the input ship information data using convolutions with a kernel size of 3 and 512 output channels, employing the ReLU activation function. Then, the LSTM network processes the convolution output, and the hidden layer of the LSTM network is used as the output for the subsequent LSTM decoder to generate the target's future trajectory. The final experimental results are shown in Tables 2 and 3, which are comparison tables of the average displacement loss of the model-predicted trajectory and the final displacement loss of the model-predicted trajectory.

[0128] Table 2

[0129]

[0130]

[0131] Table 3

[0132]

[0133] To assess the performance of the CLC-Transformer model in predicting the trajectory of target intent, this invention randomly sampled data from four datasets and visualized the output of the CLC-Transformer model and the actual trajectories of the data. The results are as follows: Figure 6 , Figure 7 , Figure 8 , Figure 9 As shown.

[0134] The CLC-Transformer model outperformed existing models on four datasets: in20-pred10, in30-pred15, in40-pred20, and in50-pred25. However, the model's trajectory prediction performance declined with increasing prediction time series length, and the decrease in the average displacement loss index was relatively small, although the final decrease in the displacement loss index was significant. While increasing the prediction time series length increases the model's input information, theoretically more beneficial for predicting target ship trajectories, it also introduces more randomness into the prediction process. Relying solely on past operational information of the target ship makes it difficult to predict trajectories over longer time series. To better predict target ship trajectories, the model needs more information about the target ship, such as the sea area where the target ship is located and the navigation information of other nearby targets.

[0135] To address the region-partitioning-based training strategy used by the CLC-Transformer model, this invention conducted ablation experiments. Two CLC-Transformer models were trained on four datasets: in20-pred10, in30-pred15, in40-pred20, and in50-pred25. The former did not use the region-partitioning-based training strategy, while the latter did. The effectiveness of the region-partitioning-based training strategy was verified by comparing the prediction performance of the two models on the datasets. The ablation experiment results are shown in Tables 4 and 5, which compare the average displacement loss of the predicted trajectory and the final displacement loss of the predicted trajectory. During training, the CLC-Transformer (without region-partitioning-based training strategy) did not use the region-partitioning-based strategy; the feature vector group processing method was similar to that in the BERT model, taking the first vector of the feature vector group as the model output. In contrast, the CLC-Transformer used the region-partitioning-based training strategy, selecting the directional feature vector from the model output as the model output.

[0136] Table 4

[0137]

[0138] Table 5

[0139]

[0140] Experimental results show that the region-partitioning-based training strategy effectively improves the trajectory prediction accuracy of the proposed CLC-Transformer, while maintaining high prediction accuracy even in long-term prediction scenarios. This experiment demonstrates that the region-partitioning-based training strategy can encode the physical motion constraints of the target ship into the CLC-Transformer model, thus enhancing the model's accuracy in predicting the target ship's trajectory. The region-partitioning-based training strategy proposed in this invention is shown in Table 6:

[0141] Table 6

[0142]

[0143]

[0144] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these all fall within the protection scope of the present invention. Therefore, the protection scope of this invention patent should be determined by the appended claims.

Claims

1. A method for predicting the navigation trajectory of a ship's automatic identification system, characterized in that, Specifically, the steps include the following: S1. Obtain the relative current selected historical ship AIS operation status data of the target ship to be predicted, input the historical AIS operation status data of the target ship to be predicted into the trained CLC-Transformer model, and obtain the directional feature vector group containing time information. The CLC-Transformer model includes: an encoder and a decoder. S101. Obtain the historical AIS operating status data of the selected target ship to be predicted, input it into the encoder, and obtain the motion time sequence characteristics of the target. S102. Input the preset ship direction feature vector set and the target motion timing features output by the encoder end into the decoder end to obtain the direction feature vector set containing timing information; S2. Based on the directional feature vector group containing temporal information, associate a directional feature vector containing temporal information with the potential motion region of the predicted target. S3. Input the directional feature vector containing time-series information obtained in S2 and the final navigation point in the preset target ship AIS navigation data into the trajectory decoder to obtain the predicted ship AIS navigation trajectory. The encoder-side model structure of the CLC-Transformer model includes: a block-based long short-term memory network C-LSTM and a MixResNet; The C-LSTM structure is based on a parameter-shared long short-term memory network. First, the data is padded with zeros according to the data length and the preset number of segments to ensure that the length of the padded temporal data is divisible by the number of segments. Then, the input temporal data is converted into multiple temporal data blocks according to the number of segments, and fed into the long short-term memory network respectively. C-LSTM is responsible for the pre-extraction of temporal data features and converts the variable-length temporal data into temporal embedding data with a fixed number of blocks and a fixed embedding dimension, so that the subsequent MixResNet can process the variable-length temporal data. MixResNet consists of three one-dimensional residual convolutional branches with different scales of 3, 5, and 7. This structure extracts features from the data at different convolutional scales and then performs channel fusion of the features extracted by the three branches through a subsequent 1x1 convolutional Mix layer. Finally, the output is obtained through a fully connected layer. MixResNet is responsible for multi-scale feature extraction and fusion of temporal data to obtain better temporal feature embedding.

2. The method for predicting the navigation trajectory of an automatic identification system for ships according to claim 1, characterized in that, The encoder-side model training method in the CLC-Transformer model is as follows: 1) Data Processing: Obtain historical AIS operational status time-series data of the target vessel to be predicted for model training. Divide the time-series data to obtain anchor point data and neighborhood data, specifically: 1.1) Acquire time-series data according to a preset fixed window W n The time series data is segmented by size to obtain the segmented time series data; 1.2) In each fixed window W n The center of the data generates anchor data W according to the preset anchor data length. anc Then, within a selectable range of values ​​from the anchor data center to its left or right, the center of the neighborhood data is determined, and neighborhood data W is generated according to the preset neighborhood data length. pos， And ensure the neighborhood W pos The fixed window data W will not be retrieved. n In addition, the length of the neighborhood data is the same as the length of the anchor data, thus obtaining a set of anchor data and a set of neighborhood data; Among them, anchor data and neighborhood data within the same window are a pair of positive samples, while neighborhood data and anchor data in different windows are a pair of negative samples; 2) Encoder Training: The SimCTSE method is used to pre-train the encoder model in the CLC-Transformer model using anchor data and neighborhood data. The pre-trained temporal model is the encoder in the CLC-Transformer model, specifically: 2.1) Input the generated anchor data and neighborhood data into the CLCE that needs to be pre-trained, and let the encoder model embed the anchor data and neighborhood data to obtain anchor embedding and neighborhood embedding data, where the anchor embedding is the first... i’ Data at location and neighborhood embedding i’ The data at each location is a pair of embedded positive samples; 2.2) Perform join and copy operations on the anchor embedding data and the neighborhood embedding data to obtain... Query Matrix and Key matrix: First, concatenate the tail of the anchor embedding data and the head of the neighbor embedding data in each pair of positive embedding samples to obtain... Query matrix; Then, Query Matrix copying, obtaining Key matrix; 2.3) For Query Each vector in the matrix and Key The cosine similarity is calculated for each vector in the matrix, and cosine values ​​of identical embeddings are removed to obtain the cosine similarity matrix between the anchor embeddings and the neighbor embeddings. Logits The formula for calculating cosine similarity is as follows: （1） in, yes Query Any value in the matrix, yes Key Any value in the matrix; Logits The first in the matrix i Each value in the row is Query The first in the matrix i Embedded data and Key Cosine similarity of each embedded data in the matrix; 2.4) Based on the relationship between the embedded anchor data and the embedded neighborhood data, the embedded positive sample pairs are... Logits Each row vector in the matrix generates a label: First of all, for Logits The first in the matrix i A row vector group, in which each column vector is a row vector group. Query The first in the matrix i Row embedding vectors and Key Cosine similarity of each embedding vector in the matrix; Then, for Logits The first in the matrix i For a row vector group, based on the positive sample pair relationship, select a column vector from the row vector group. This column vector represents... Query The first in i each embedding vector and Key In the matrix that can be Query The Middle i The cosine similarity of the vectors forming the positive sample pair relationship is calculated using the embedded vectors, and the index of this column vector is used as the first... i Labels of row vector groups; in, Logits In the matrix ( i , j The value of the position represents Query The first in the matrix i each embedding vector and Key The first in the matrix j The cosine similarity of the embedded vectors is used to generate labels, which are then used for parameter optimization during model training. In the optimization process, the labels tell the model what parameters to optimize. Logits The model learns how to maximize the cosine similarity of label positions in each row of the matrix. The cross-entropy function is used as the loss function for pre-training the editor model, and the pre-trained time series model is finally obtained. 2.5) Using the information obtained in 2.3 Logits The matrix and the labels obtained in section 2.4 are used for a learning task to maximize the cosine similarity of the label positions, thereby obtaining the trained encoder.

3. The method for predicting the navigation trajectory of a ship's automatic identification system according to claim 2, characterized in that, The specific implementation method of step S102 is as follows: The encoder-side model trained in the CLC-Transformer model extracts features from the target's past AIS running data to obtain the target's motion temporal features, and uses these features as the attention mechanism. Key and Value Passed to the decoder of the CLC-Transformer model; The decoder uses self-attention and attention mechanisms to aggregate the temporal features of the target motion from the encoder, and aggregates these features into a set of potential motion direction vectors of the target, which serves as the final output of the CLC-Transformer model.

4. The method for predicting the navigation trajectory of an automatic identification system for ships according to claim 3, characterized in that, The decoder-side structure of the CLC-Transformer model includes, in sequence: word embedding layer, first multi-head self-attention layer, first addition and normalization layer, second multi-head attention layer, second addition and normalization layer, fully connected layer, and third addition and normalization layer. First, the preset ship direction feature vector set is initialized using a word embedding layer to obtain the initialized direction feature vector set; the number of vectors in the initialized direction feature vector set is the same as the number of potential directions of target motion. Then, the initialized directional feature vector set is input into the first multi-head self-attention layer, and each attention head undergoes three different linear transformations to obtain the attention mechanism. Key, Query, Value vector, Query and Key The vector is first scaled and dot-producted to obtain the attention score. The attention score is then compared with... Value Vector multiplication yields the output from the attention layer; Then, the output of the first multi-head attention layer passes through the first addition and normalization layer, and the output of the self-attention layer is directly added to the input of the previous self-attention layer and then undergoes layer normalization processing. Then, repeat the above operation, but at this time in the second multi-head attention layer Key and Value The input is the output of the encoder. Query The input is the output of the first summation and normalization layer. After recalculation, the output is obtained and then passed through the second summation and normalization layer. However, the summation value is the output of the first summation and normalization layer. Finally, the data undergoes a nonlinear transformation through three fully connected layers without changing the dimension of the input data. The output of the fully connected layers is then fed into the third addition and normalization layer to obtain the final output.

5. The method for predicting the navigation trajectory of an automatic identification system for ships according to claim 4, characterized in that, In step S2, based on the directional feature vector group containing temporal information, a directional feature vector containing temporal information is associated with the potential motion region of the predicted target. The specific association method is as follows: S201. Based on the positional relationship between the first point in the input time series data of the CLC-Transformer model and the final point in the preset AIS data, generate a region feature selection matrix. The matrix has 1 row and the same number of columns as the output vector of the CLC-Transformer model. The matrix consists of multiple 0s and one 1. S202. Perform matrix multiplication on the region feature selection matrix obtained in S201 and the feature vector group output by the CLC-Transformer model. The resulting matrix is ​​a directional feature vector containing temporal information.

6. The method for predicting the navigation trajectory of a ship's automatic identification system according to any one of claims 1 to 5, characterized in that, The trajectory decoder includes: LSTM neurons and fully connected layers; The directional feature vector containing temporal information obtained by S2 and the endpoint trajectory in the preset AIS data are input into the LSTM neuron. The data output by the LSTM neuron is input into the fully connected layer. The fully connected layer outputs the target's future trajectory at time n. The target's future trajectory at time n is input back into the LSTM neuron. The data output by the LSTM neuron is input into the fully connected layer. The fully connected layer outputs the target's future trajectory at time n+1. This process is repeated to obtain the target's trajectory over a future period.

7. A prediction module for the navigation trajectory of a ship's automatic identification system, characterized in that, The prediction module uses a CLC-Transformer model, which includes an encoder and a decoder. The encoder structure includes: a block-based long short-term memory network C-LSTM and a MixResNet; the block-based long short-term memory network C-LSTM pre-extracts the features of the time series data and converts the variable-length time series data into time series embedded data with a fixed number of blocks and a fixed embedding dimension, which is then sent to the MixResNet; the MixResNet processes the variable-length time series data and performs multi-scale feature extraction on the time series data. The encoder inputs the historical AIS operational status data of the target vessel to obtain the target's motion temporal characteristics; The decoder structure comprises, in sequence: a word embedding layer, a first multi-head self-attention layer, a first addition and normalization layer, a second multi-head attention layer, a second addition and normalization layer, a fully connected layer, and a third addition and normalization layer; it receives the target's motion temporal features output by the encoder, and aggregates these features into a set of potential motion direction vectors to obtain a set of direction feature vectors containing temporal information; the block-based long short-term memory network C-LSTM, with a parameter-shared long short-term memory network at its core, is responsible for pre-extracting temporal data features and converting variable-length temporal data into temporal embedding data with a fixed number of blocks and a fixed embedding dimension, enabling subsequent MixResNet processing of variable-length temporal data; The MixResNet consists of three one-dimensional residual convolutional branches at different scales, which extract features from the data at different convolutional scales. The features extracted by the three branches are then fused through a subsequent 1x1 convolutional Mix layer, and the output is obtained through a fully connected layer. MixResNet is responsible for multi-scale feature extraction and fusion of temporal data to obtain better temporal feature embedding.

8. The prediction module for the navigation trajectory of an automatic identification system for ships according to claim 7, characterized in that, The word embedding layer at the decoder initializes the preset set of directional feature vectors required by the ship to obtain the initialized set of directional feature vectors; the number of vectors in the initialized set of directional feature vectors is the same as the number of potential directions of the target motion.

Images