Ship motion track prediction method based on ship track big data

Abstract

The invention discloses a ship motion track prediction method based on ship track big data. The method comprises the following steps: collecting ship AIS data and preprocessing the ship AIS data to obtain ship track data; a sliding window is adopted to sample ship trajectory data, ship historical trajectory point state features are extracted, and a time interval relation matrix is constructed; the method comprises the following steps: constructing a Transform architecture model which only retains a decoder, discretizing an extracted trajectory feature and a to-be-predicted time interval into corresponding one-hot coding vectors so as to obtain embedded vectors, merging the embedded vectors of the two into a joint feature embedded vector, and stacking ship trajectory point sequences to construct an obtained joint feature embedded matrix, linearly adding the time interval relation embedded matrix and the position feature embedded matrix to obtain an embedded matrix, and inputting the embedded matrix into a Transform architecture model for training to obtain a prediction model; and outputting a ship motion trail prediction result through a decoder. According to the method, the accurate prediction of the non-uniform trajectory sequence is realized, and the problems of non-uniform time interval and low prediction accuracy of the ship trajectory data are solved.

Description

Technical Field

[0001] The present invention relates to the technical field of ship motion trajectory prediction, and in particular to a ship motion trajectory prediction method based on ship track big data. Background Art

[0002] AIS (Automatic Identification System) is an aid to navigation system used for maritime safety and communication between ships and shores, and between ships. The transmitting equipment of the AIS system can automatically and periodically send the ship's identity information, navigation information and other data at a specified frequency, and the receiving equipment can receive AIS information transmitted by other ships, shore-based facilities and satellite systems. In the actual scenario of ship navigation, ship motion prediction has high requirements for accuracy and real-time performance. Traditional ship trajectory prediction methods discard timestamps and only retain the sequence of ship motion trajectories. These methods all assume that all adjacent trajectory sequences in the sequence have the same time interval.

[0003] At present, the ship trajectory prediction algorithm based on the kinematic equation is a method to predict the future trajectory of a ship by analyzing and simulating its motion state (such as position, speed, heading, etc.). This method mainly relies on the dynamic characteristics and kinematic model of the ship, and uses mathematical equations to describe the movement law of the ship under different conditions. However, the algorithm also has the disadvantages of high model complexity, large amount of calculation, and difficulty in fully considering environmental factors (such as wind, waves, etc.), which to a certain extent limits its prediction accuracy and scope of application.

[0004] At present, there are also algorithms for predicting ship trajectories through RNN (recurrent neural network) variant models such as LSTM (long short-term memory network). LSTM and other RNN variant models strengthen the dependence on long-distance features through the "gate" structure based on the recurrent neural network. In ship trajectory prediction, LSTM and other RNN variant models usually take the ship's historical position (such as longitude and latitude), speed, heading, etc. as input sequences, learn the nonlinear characteristics of the trajectory through the motion state of the ship's trajectory, and predict the ship's position at a certain point in the future. LSTM and other RNN variant models usually assume that the input data is at equal time intervals during training. However, in practical applications, due to various factors, there are often unequal time intervals between adjacent track points in the ship trajectory data. In order to solve this problem, it is usually necessary to interpolate the data so that the input data meets the requirements of equal time intervals. However, interpolation processing often brings a series of side effects, such as information distortion, increased computational complexity, or the generated trajectory may not conform to the actual navigation law. These problems will more or less have an adverse effect on the accuracy of ship trajectory prediction; in addition, although models such as LSTM have alleviated the gradient vanishing problem to a certain extent, when processing very long sequences, they may still face the problem of insufficient capture of long-term dependencies, and LSTM captures the time dependency in sequence data through its internal hidden state. In each step of calculation, the current output depends not only on the current input, but also on the hidden state of the previous step. This mechanism enables it to process sequences of arbitrary length, but also limits its ability to parallelize calculations.

[0005] Therefore, how to solve the problem of uneven time intervals of ship trajectory data and improve prediction accuracy is an urgent problem that technicians in this field need to solve. Summary of the invention

[0006] In order to overcome the defects and shortcomings of the prior art, the present invention provides a ship motion trajectory prediction method based on ship track big data. The present invention adopts a Transformer architecture that only retains the decoder to model the trajectory sequence, and introduces a time interval embedding module to address the difficulty of uneven time intervals of AIS data. Furthermore, a self-attention network that integrates time interval relationship information is designed to achieve accurate prediction of non-uniform trajectory sequences, thus solving the problems of uneven time intervals of ship trajectory data and low prediction accuracy.

[0007] In order to achieve the above object, the present invention adopts the following technical solutions:

[0008] The present invention provides a method for predicting ship motion trajectory based on ship track big data, comprising the following steps:

[0009] Collect ship AIS data and perform data preprocessing to obtain ship trajectory data;

[0010] The ship trajectory data is sampled using a sliding window to obtain sampled data, trajectory features are extracted from the sampled data, state features of the ship's historical trajectory points are obtained, and a time interval relationship matrix is ​​constructed;

[0011] Construct a Transformer architecture model, discretize the extracted trajectory features and the time interval to be predicted into corresponding one-hot encoding vectors and obtain embedding vectors, then merge the embedding vectors of the two into a joint feature embedding vector, and then stack the fixed-length ship trajectory point sequence to obtain the joint feature embedding matrix A, and pass the time interval relationship matrix through the embedding layer to obtain the time interval relationship embedding matrix B, and embed the position information sequence to obtain the position feature embedding matrix P;

[0012] After transforming the dimension of the time interval relationship embedding matrix B, it is linearly added to the joint feature embedding matrix A and the position feature embedding matrix P to obtain the embedding matrix E. The embedding matrix E is input into the Transformer architecture model for training to obtain the trained prediction model.

[0013] The trajectory data to be predicted is input into the trained prediction model, and the ship motion trajectory prediction result is output through the decoder.

[0014] As a preferred technical solution, data preprocessing specifically includes data cleaning and data set segmentation;

[0015] The data cleaning specifically includes:

[0016] Remove irrelevant feature attributes in the trajectory data set, and retain MMSI code, longitude lon, latitude lat, speed over ground SOG, course over ground COG, and sampling time time as the feature information of the ship trajectory point;

[0017] The trajectory data set is grouped according to MMSI, and the data sequence corresponding to each MMSI is sorted by time. The sorted data is deduplicated according to the sampling time, and the redundant data repeatedly received by the sensor at the same time is deleted;

[0018] Eliminate abnormal dynamic data;

[0019] Eliminate some parking trajectory data;

[0020] By identifying the parking trajectory during driving, the trajectory is segmented, the entire trajectory is segmented into the trajectory before and after the mid-way stop, and the trajectory whose number of trajectory points after segmentation is greater than a certain set value is selected;

[0021] The data set segmentation specifically includes:

[0022] According to the MMSI grouping, a set proportion of the time series data of the groups are randomly selected from each group as the training set, and the remaining groups are used as the test set. The time series data of each MMSI group is sorted by time.

[0023] As a preferred technical solution, a sliding window is used to sample the ship trajectory data, specifically including:

[0024] The size of the sampling sliding window is set to n+1, the step size is set to 1, and the trajectory features are extracted from the sampled data, wherein the trajectory features include latitude, longitude, speed over the ground, heading over the ground, and sampling time;

[0025] For the sequence data in the sliding window, the first n data are used as the input of the model, and the last n data are used as the label output of the model. When the length m of the ship trajectory data is less than n+1, the first m-1 data are used as the input of the model, and the last m-1 data are used as the label output of the model. For the input and label output of the model, padding items are added on the right until the length is n.

[0026] As a preferred technical solution, the construction of the time interval relationship matrix specifically includes:

[0027] Get the time interval between adjacent track points in all track points, divide it by the shortest time interval in the ship track sequence, and get the personalized interval, which is specifically expressed as:

[0028]

[0029] in, represents the shortest time interval, t i -t j represents the time interval between two time points i and j, i = 1, 2, 3, ... n, j = 1, 2, 3, ... n;

[0030] The time interval relationship matrix is ​​constructed as:

[0031]

[0032] Among them, M u Represents the time interval relationship matrix, which is a symmetric matrix with main diagonal elements equal to 0.

[0033] As a preferred technical solution, the extracted trajectory features and the time interval to be predicted are discretized into corresponding one-hot encoding vectors and an embedding vector is obtained, and then the embedding vectors of the two are merged into a joint feature embedding vector, which specifically includes:

[0034] Get trajectory features It is scattered into a one-hot encoding vector:

[0035]

[0036] Then the state feature embedding vector is obtained through the embedding search operation:

[0037]

[0038] Get the one-hot encoded vector of the time interval to be predicted And the embedding vector of the time interval to be predicted

[0039] The joint feature embedding vector after merging the state feature embedding vector and the time interval embedding vector to be predicted is expressed as:

[0040]

[0041] Among them, lat represents latitude, lon represents longitude, SOG represents speed over the ground, and COG represents course over the ground.

[0042] As a preferred technical solution, the fixed-length ship trajectory point sequence is stacked to obtain a joint feature embedding matrix A, which is specifically expressed as:

[0043]

[0044] in, For an n-row d model A real matrix with columns, n represents the trajectory length, d model Represents the model's latent dimension.

[0045] As a preferred technical solution, the time interval relationship matrix is ​​embedded in the embedding layer to obtain the time interval relationship embedding matrix B, which is specifically expressed as:

[0046]

[0047] Among them, each item in the time interval relationship matrix The embedding vector is obtained by embedding search The dimension of the time interval relationship embedding matrix B is transformed, and the transformed matrix is ​​C, where The dimension is consistent with the joint feature embedding matrix A, which is an n-row d model A real matrix with columns, n represents the trajectory length, d model Represents the model’s latent dimensions;

[0048] Embed the position information sequence to obtain the position feature embedding matrix Then the obtained learnable position feature embedding matrix P is linearly added with the joint feature embedding matrix A and the time interval relationship embedding matrix B as the input of the model, and finally the embedding matrix containing the position information and the time interval relationship is obtained. Specifically expressed as:

[0049] E=[A1+C1+P1,A2+C2+P2,…,A n +C n +P n ].

[0050] As a preferred technical solution, the decoder is provided with N layers of decoding layers with completely identical structures and different parameters, and the decoding layers include 1 masked multi-head self-attention module, 2 residual connections and layer normalization, and 1 feedforward network;

[0051] The masked multi-head self-attention module includes an input linear layer, a scaled dot product attention layer, a merging layer, and an output linear layer;

[0052] The input linear layer inputs the query vector Q, the key vector K and the value vector V respectively, wherein the query vector Q, the key vector K and the value vector V are the embedding vectors E of the trajectory point at time t respectively. t Multiply by the weight matrix W Q , weight matrix W K , weight matrix W V get;

[0053] The scaled dot product attention layer performs a self-attention operation;

[0054] The merging layer merges the results of multiple attention heads;

[0055] The output linear layer performs another spatial mapping on the merged result and outputs the attention information, and its output is the output result of the masked multi-head self-attention module.

[0056] As a preferred technical solution, the scaled dot product attention layer performs a self-attention operation, specifically including:

[0057] Calculate the similarity between the query vector Q and each key vector K, and use the dot product method to obtain the weight;

[0058] Use the Softmax function to normalize the weights;

[0059] The weight and the corresponding value vector V are weighted and summed to obtain the final self-attention vector. The formula for the self-attention operation is as follows:

[0060]

[0061] Among them, d K is the dimension of vector Q;

[0062] The merging layer merges the results of multiple attention heads, expressed as:

[0063] MultiHead(Q,K,V)=Concat(head1,head2,…,head h )W O

[0064]

[0065] Where h is the number of attention heads, is the transformation parameter of the data; W i Q ∈ are the transformation parameters of the query vector Q, key vector K and value vector V respectively. Concat represents the concat function, which merges the generated results of each attention head.

[0066] As a preferred technical solution, the residual and layer normalization are specifically expressed as:

[0067] sub_layer_output=x+SubLayer(LayerNorm(x))

[0068] Among them, x represents the input, SubLayer is the residual operation, and LayerNorm is the layer normalization operation.

[0069] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0070] (1) Traditional time series analysis methods, such as recurrent neural networks (RNNs) and their variants, usually require that the input data have equal time intervals. However, in ship motion trajectory data, due to various factors (such as signal loss, changes in data collection frequency, etc.), the time intervals are often unequal. The present invention can directly process data with unequal time intervals through the self-attention mechanism of Transformer without data interpolation or resampling, thereby retaining the original characteristics and integrity of the data and avoiding information distortion.

[0071] (2) In order to more accurately capture the temporal features in the ship trajectory, the present invention introduces a time perception mechanism into the Transformer model, which can explicitly embed the temporal information into the input of the model and introduces a time interval relationship matrix. Such time encoding enables the model to understand not only the spatial position relationship in the data, but also the temporal order and time difference relationship in the data, thereby more accurately predicting the future trajectory of the ship, because the motion trajectory of the ship is highly dependent on time.

[0072] (3) Since the Transformer model has powerful feature extraction and learning capabilities, it can capture the complex patterns and nonlinear features in the ship trajectory. At the same time, the introduction of the time perception mechanism further enhances the model's ability to understand time information. The combination of the two enables the prediction method based on Transformer and time perception to achieve higher accuracy in ship trajectory prediction, especially when dealing with long-term predictions and prediction tasks under complex environmental conditions. The Transformer model can capture long-distance dependencies. In the prediction of ship motion trajectories, this means that the model can predict farther future positions rather than just short-term motion trends, which is of great significance for formulating long-term navigation plans and responding to potential risks.

[0073] (4) Compared with traditional RNN and its variant models, the Transformer model can process elements in a sequence in parallel during training, thereby greatly improving computational efficiency. This enables the Transformer-based prediction method to complete model training and prediction tasks in a shorter time and better meet real-time requirements. BRIEF DESCRIPTION OF THE DRAWINGS

[0074] Figure 1 It is a schematic diagram of trajectory segmentation based on parking trajectory points of the present invention;

[0075] Figure 2 A schematic diagram of the network structure of the Transformer architecture model including time interval embedding of the present invention;

[0076] Figure 3 It is a schematic diagram of embedding the state features of the trajectory points of the present invention;

[0077] Figure 4 It is a schematic diagram of embedding representation of time interval relationship matrix of the present invention;

[0078] Figure 5 Schematic diagram of the process of the masked multi-head self-attention mechanism of the present invention. DETAILED DESCRIPTION

[0079] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0080] like Figure 1As shown, this embodiment provides a ship motion trajectory prediction method based on ship track big data. The method adopts the Transformer architecture that only retains the decoder to model the trajectory sequence, and introduces a time interval embedding module to address the difficulty of uneven time intervals of AIS data. It embeds a time interval relationship matrix instead of a general time interval or timestamp, which can better understand the time sequence and time difference relationship. Then, a self-attention network that integrates time interval information is designed, and time features are introduced into the prediction of ship motion trajectories to achieve accurate prediction of uneven trajectory sequences.

[0081] The method specifically comprises the following steps:

[0082] Step 1: Collect ship AIS data and pre-process it to obtain ship track data, including:

[0083] Step 1-1: Data cleaning, including the following parts:

[0084] (1) Irrelevant feature attributes in the trajectory dataset are eliminated, and only MMSI code, longitude (lon), latitude (lat), speed over ground (SOG), course over ground (COG), and sampling time (time) are retained as the feature information of the ship trajectory points.

[0085] (2) Eliminate redundant data: The trajectory data set is grouped according to MMSI, and the data sequence corresponding to each MMSI is sorted by time. The sorted data is deduplicated according to the sampling time, and the redundant data received repeatedly by the sensor at the same time is deleted.

[0086] (3) Eliminate abnormal dynamic data: Ship trajectory data is collected by a variety of sensor devices. Data distortion may occur during the collection process. In the data cleaning stage, outliers need to be processed first to improve the prediction accuracy of the model. Abnormal dynamic data can be eliminated using the following two methods:

[0087] a. Statistical method: Elimination is based on the principle of 3 times the standard deviation;

[0088] b. Judge and eliminate according to the experience value, as shown in the following formula:

[0089] speed>=0&speed<30 (1)

[0090] cog>=0&cog<360 (2)

[0091] Wherein, lon and lat represent the longitude and latitude of the ship's WGS84 geographic coordinate system respectively; speed represents the speed over the ground; and cog represents the course over the ground.

[0092] (4) Eliminate some parking trajectory data.

[0093] The ship trajectories in the study waters can be roughly divided into three categories, namely, mooring trajectories, straight-line trajectories, and curved-line trajectories. The characteristics of mooring trajectories are mainly manifested in two aspects. First, the speed is low for a long period of time and moves irregularly near the same position; second, the trajectory shape is a cluster in a small range or an irregular cluster structure at the same position. The main purpose of ship trajectory prediction is to predict the trajectory of ships in a sailing state in the future. The study area belongs to the port waters, and there are a large number of moored ships. The mooring trajectory sequence is low-value data in the prediction task of this embodiment, and may even interfere with the final prediction effect. Therefore, for the ship trajectory task, it is very important to eliminate the mooring trajectory sequence. However, in the data cleaning part of this embodiment, only the ship trajectories in the anchored state (before sailing and after mooring) are eliminated, and the mooring trajectories generated by the shorter stay during the ship's travel are not processed temporarily.

[0094] Theoretically, the speed of a ship at anchor is 0, but due to the effect of wind and waves, the speed of a ship at anchor may not be 0. Based on this, the speed threshold is set to 0.05 kn / s. If the trajectory data with a speed continuously less than 0.05 kn / s at the beginning of the ship trajectory sequence data, it can be considered as the anchored state before departure; correspondingly, at the end of the sequence data, the trajectory data with a speed continuously less than 0.05 kn / s can be considered as the anchored state after anchoring. Trajectory data identified as these two states can be eliminated. Of course, it is still necessary to group and sort according to the MMSI code before elimination.

[0095] (5) Trajectory segmentation based on parking trajectory points

[0096] Ship trajectory segmentation plays an important role in the preprocessing stage of ship trajectory prediction, and is of great significance for subsequent feature extraction and feature mining. In addition, dividing a long trajectory sequence into multiple short trajectory sequences helps to reduce the computational complexity of the trajectory prediction task, and the feature similarity between each sub-trajectory can be guaranteed, providing rich semantic information and giving full play to the available value of the trajectory. In the previous step, only the ship trajectories in the anchored state (before departure and after mooring) were eliminated, while the mooring trajectories generated by the shorter stops during the ship's travel were not processed. Therefore, the mooring trajectories during the travel process can be identified and eliminated, such as Figure 1As shown, the entire trajectory can be divided into the trajectory before and after the stopover. The identification criteria can refer to the content of Table 1 below, and the data with a time interval of more than 12s are marked as stopover trajectory points. However, since some ships may stop more than once, some trajectories after segmentation are too short. Due to the need for model training, the trajectory sequence length cannot be too short. At this time, one more screening condition can be added: the number of trajectory points of the segmented trajectory must be greater than or equal to 20. In this way, the original trajectory is finally segmented into a sequence set of normal navigation trajectories.

[0097] Table 1 Ship status and corresponding reporting frequency

[0098] Ship status Reporting frequency anchored boat 3 minutes / time Vessels with a speed of 0-14 knots 12 seconds / time Vessels with a speed of 0-14 knots and changing course 4 seconds / time Vessels with a speed of 14-23 knots 6 seconds / time Vessels sailing at 14-23 knots and changing course 2 seconds / time Vessels with a speed exceeding 23 knots 3 seconds / time Vessels travelling at speeds exceeding 23 knots and changing course 2 seconds / time

[0099] Step 1-2: Dataset segmentation;

[0100] In general, the training and test data of machine learning are divided by randomly selecting a certain proportion (e.g. 80%) of all data as training data, and the rest as test data. However, the data division here is different. It is necessary to first group according to MMSI, and then randomly select a certain proportion of the time series data of the group as the training set from the group, and the data of the remaining group as the test set. The time series data of each MMSI group should be sorted by time for the next step of processing.

[0101] Step 2: Use a sliding window to sample the ship trajectory data, obtain sampled data, extract features from the sampled data, and construct a time interval relationship matrix;

[0102] In this embodiment, step 2 uses a sliding window to sample each preprocessed trajectory data, and the sampling steps are as follows: the size of the sampling sliding window is n+1, the step size is 1, and the trajectory features are extracted from the sampled data. The trajectory features required for modeling include latitude, longitude, ground speed, ground heading, and sampling time; among them, the sequence data in the window, the first n (n represents the maximum length of input and output) data are used as the input of the model, and the last n are used as the output (label) of the model. In the case where the trajectory length m is less than n+1, the first m-1 data are used as the input of the model, and the last m-1 data are used as the output of the model. At this time, for both the input and the output, padding items are added on the right until the length is n.

[0103] In this embodiment, the steps of step 2 to construct the time interval relationship matrix are as follows:

[0104] Consider a fixed-length time series, t=(t1, t2, …t n), it is obvious that for the prediction of ship trajectory, we only need to care about the relative length of the time interval in the sequence, and it has nothing to do with the absolute timestamp. Therefore, for all time intervals, divide them by the shortest time interval in the ship trajectory sequence to obtain the personalized interval. Among them, the shortest time interval is obtained by: obtaining the time interval between two adjacent trajectory points, and then finding the smallest one from these time intervals;

[0105] Specifically, given a fixed length time series t = (t1, t2, ... t n ), the time interval between two time points i and j is t i -t j , then the time interval set is R u In this embodiment, the minimum time interval of a user is expressed as The scaled time interval is:

[0106]

[0107] in, N * represents the set of positive integers;

[0108] The length of the ship trajectory sequence is n, so there will be a time interval between the n trajectory points, and these time intervals constitute the time interval relationship matrix. Therefore, the time interval relationship matrix M of the trajectory sequence is u ∈N n×n for:

[0109]

[0110] Among them, m u is a symmetric matrix with all diagonal elements being 0;

[0111] Step 3: The extracted state features (including latitude, longitude, ground speed and ground heading) and the time interval to be predicted are discretized into one-hot encoding vectors, and then input into the Transformer architecture model together with the time interval relationship matrix for modeling. The data passes through the spatiotemporal embedding module and decoder in turn, and finally outputs the trajectory prediction result. The spatiotemporal embedding module includes a joint feature embedding unit, a time interval relationship matrix embedding unit and a position feature embedding unit.

[0112] like Figure 2 As shown in Figure 1, the network structure of the Transformer architecture model containing time interval embedding is shown in Figure 2, where the state features of the ship's historical trajectory points are obtained. like Figure 3 As shown, it is discretized into a one-hot encoding vector Then the embedding vector is obtained through the embedding lookup operation Similarly, the unique hot encoding vector of the time interval to be predicted (i.e. the time interval between the next time point to be predicted and the current time point) can be obtained: and the embedding vector Then the state feature embedding and the time interval embedding to be predicted are combined to obtain where d model is the model potential dimension, R represents a real number set and d model =d lat +d lon +d SOG +d COG +d next-interval , where d lat represents the embedding dimension of lat, d lon represents the embedding dimension of lon, d SOG represents the embedding dimension of SOG, d COG represents the embedding dimension of COG, d next-interval Represents the embedding dimension of the time interval to be predicted.

[0113] Then, the ship trajectory point sequences with a fixed length of n are stacked to obtain the joint feature embedding matrix Represents a real matrix with n rows and d model Column, n represents the trajectory length, d model Indicates the potential dimension of the model. In this embodiment, n is set to 42, d lat =512,d lon =512,d SOG =128,d COG =128,d next-interval =64, then d model =1344;

[0114] like Figure 4 As shown, the obtained time interval relationship matrix M u Use the corresponding embedding matrix B∈R n×n×d′ (where d′×n=d model ), in this embodiment, d′=32, the time interval relationship matrix is ​​expressed as a time interval relationship embedding matrix after passing through the embedding layer, which is expressed as:

[0115]

[0116] Each of these The embedding vector is obtained by embedding search in Transform the dimensions of matrix B to make it consistent with the dimensions of matrix A, that is, C = [C1, C2, ..., C n ],in,

[0117] In step 3, since the weight calculation of the self-attention mechanism depends on Q and K to calculate the correlation, the input position information is not considered. Even if the positions of the elements in the sequence are exchanged, the final result is similar, which is unreasonable in actual ship trajectory prediction. Therefore, it is necessary to introduce position information and embed the position information sequence (using trainable position feature embedding) to obtain the position feature embedding matrix Among them, the position information sequence is 0, 1, 2, 3...; after the position information sequence passes through the embedding layer, the position feature embedding matrix can be obtained;

[0118] Then the obtained learnable position feature embedding P is linearly added to the previous joint feature embedding matrix A and the time interval relationship embedding matrix B as the input of the model, and finally the embedding matrix containing the position information and the time interval relationship is obtained. It is expressed as:

[0119] E=[A1+C1+P1,A2+C2+P2,…,A n +C n +P n ] (6)

[0120] in,

[0121] In step 3, the embedding matrix E enters the decoder, and then, by stacking multiple decoding layers and increasing the number of layers, complex nonlinear relationships are introduced, such as Figure 2 As shown in the model structure diagram, N is the number of decoding layers. In this embodiment, N is 8. The decoder has N layers of decoding layers with exactly the same structure and different parameters. The decoding layer consists of 1 masked multi-head self-attention module, 2 residual connections and layer normalization, and 1 feedforward network. The input feature dimension and output feature dimension of each decoding layer are exactly the same; the most important part of the decoder is the masked multi-head self-attention module. The working principle of this module is as follows Figure 5 The input of the module is three linear layers, which pass through the scaled dot product attention layer and the merging layer in turn, and are output at the linear layer at the output. The three linear layers at the input input the query vector Q, the key vector K, and the value vector V, which are the embedding vectors E of the trajectory point at time t respectively. t Multiply by matrix W Q , W K , W V Get, where W Q , W K , W V are different weight matrices.

[0122] The scaled dot product attention layer is mainly a self-attention operation, which includes the following steps:

[0123] (1) Calculate the similarity between the query vector Q and each key vector K, and use the dot product method to obtain the weight;

[0124] (2) Use the Softmax function to normalize the weights;

[0125] (3) The weight and the corresponding value vector V are weighted and summed to obtain the final self-attention vector. The formula for self-attention operation is as follows:

[0126]

[0127] Among them, d K is the dimension of vector Q, where d K =d model / h, h is the number of attention heads, the value of h is 2, The purpose of the scaling operation is to provide a more stable gradient for subsequent training. K is the key vector.

[0128] It is necessary to explain that in order to ensure that it is based only on the information before the current position (the decoder decodes in sequence), a mask operation is added after the scaled dot product operation, setting all values ​​after the current position to -∞.

[0129] The merging layer merges the results of multiple attention heads. The formula is as follows:

[0130] MultiHead(Q,K,V)=Concat(head1,head2,…,head h )W O (8)

[0131] where head i =Attention(QW i Q ,KW i K ,VW i V ) (9)

[0132] Where h is the number of attention heads, is the transformation parameter of the data; are the transformation parameters of the query vector Q, key vector K and value vector V respectively (note that QW i Q It is not a matrix multiplication, and d Q =d K =d v =d model / h); the concat function is a merging layer that merges the generated results of each attention head; the output linear layer performs another spatial mapping on the merged result and outputs the attention information, and its output is the output result of the masked multi-head self-attention module.

[0133] In addition to the masked multi-head self-attention mechanism, the decoder also contains a fully connected feedforward network (FFN). FFN can introduce nonlinear properties. The significance of giving the self-attention mechanism nonlinear properties is that it can improve the expressiveness and generalization ability of the model. The nonlinear properties allow the model to learn more complex dependencies, thereby more accurately predicting the ship's trajectory. The network consists of two layers of linear connections, with a GELU (Gaussian Error Linear Unit) activation function in the middle. GELU is a nonlinear activation function designed to better handle nonlinear relationships in neural networks. It is defined as:

[0134] GELU(x)=xP(X≤x) (10)

[0135] Among them, x is the input, which is the data flowing in the model, X~N(μ,σ 2 ), v and σ are parameters that may need to be further optimized. Generally, μ = 0 and σ = 1 are selected, which is the standard normal distribution.

[0136] GELU can be approximated as:

[0137]

[0138] Compared with the standard ReLU activation function, the GELU function is smoother. GELU is continuously differentiable near the function value of 0. Unlike ReLU, it is discontinuous at x=0, so GELU is easier to optimize in backpropagation.

[0139] FFN(x)=GELU(xW1+b1)W2+b2 (12)

[0140] Where x is the input of FFN, W1,W2∈R d×d is a learnable parameter matrix, b1,b2∈R d is the bias term.

[0141] In addition, the decoder also uses layer normalization and residual connection technology to enhance the representation ability and stability of the model and help the model better capture the complex relationships in the sequence. The layer normalization and residual formulas are as follows:

[0142] sub_layer_output=x+SubLayer(LayerNorm(x)) (13)

[0143] Among them, x is the input, SubLayer is the residual operation, specifically MaskedMultiHeadAttention or Feedforward, and LayerNorm is the layer normalization operation.

[0144] Then, the training data obtained in step 2 is input into the model to train the prediction model, and then the test data is input into the model to obtain the model prediction results and output them, and the accuracy of the model is evaluated.

[0145] The above embodiments are preferred implementation modes of the present invention, but the implementation modes of the present invention are not limited to the above embodiments. Any other changes, modifications, substitutions, combinations, and simplifications that do not deviate from the spirit and principles of the present invention should be equivalent replacement methods and are included in the protection scope of the present invention.

Claims

1. A ship motion trajectory prediction method based on ship track big data, characterized in that: The steps include: Collect ship AIS data and perform data preprocessing to obtain ship trajectory data; The ship trajectory data is sampled using a sliding window to obtain sampled data, trajectory features are extracted from the sampled data, state features of the ship's historical trajectory points are obtained, and a time interval relationship matrix is ​​constructed; Construct a Transformer architecture model, discretize the extracted trajectory features and the time interval to be predicted into corresponding one-hot encoding vectors and obtain embedding vectors, then merge the embedding vectors of the two into a joint feature embedding vector, and then stack the fixed-length ship trajectory point sequence to obtain the joint feature embedding matrix A, and pass the time interval relationship matrix through the embedding layer to obtain the time interval relationship embedding matrix B, and embed the position information sequence to obtain the position feature embedding matrix P; After transforming the dimension of the time interval relationship embedding matrix B, it is linearly added to the joint feature embedding matrix A and the position feature embedding matrix P to obtain the embedding matrix E. The embedding matrix E is input into the Transformer architecture model for training to obtain the trained prediction model. The trajectory data to be predicted is input into the trained prediction model, and the ship motion trajectory prediction result is output through the decoder.

2. The ship motion trajectory prediction method based on ship track big data according to claim 1 is characterized in that: Data preprocessing specifically includes data cleaning and data set segmentation; The data cleaning specifically includes: Remove irrelevant feature attributes in the trajectory data set, and retain MMSI code, longitude lon, latitude lat, speed over ground SOG, course over ground COG, and sampling time time as the feature information of the ship trajectory point; The trajectory data set is grouped according to MMSI, and the data sequence corresponding to each MMSI is sorted by time. The sorted data is deduplicated according to the sampling time, and the redundant data repeatedly received by the sensor at the same time is deleted; Eliminate abnormal dynamic data; Eliminate some parking trajectory data; By identifying the parking trajectory during driving, the trajectory is segmented, the entire trajectory is segmented into the trajectory before and after the mid-way stop, and the trajectory whose number of trajectory points after segmentation is greater than a certain set value is selected; The data set segmentation specifically includes: According to the MMSI grouping, a set proportion of the time series data of the groups are randomly selected from each group as the training set, and the remaining groups are used as the test set. The time series data of each MMSI group is sorted by time.

3. The ship motion trajectory prediction method based on ship track big data according to claim 1 is characterized in that: The sliding window is used to sample the ship trajectory data, including: The size of the sampling sliding window is set to n+1, the step size is set to 1, and the trajectory features are extracted from the sampled data, wherein the trajectory features include latitude, longitude, speed over the ground, heading over the ground, and sampling time; For the sequence data in the sliding window, the first n data are used as the input of the model, and the last n data are used as the label output of the model. When the length m of the ship trajectory data is less than n+1, the first m-1 data are used as the input of the model, and the last m-1 data are used as the label output of the model. For the input and label output of the model, padding items are added on the right until the length is n.

4. The ship motion trajectory prediction method based on ship track big data according to claim 1 is characterized in that: The constructing of the time interval relationship matrix specifically includes: Get the time interval between adjacent track points in all track points, divide it by the shortest time interval in the ship track sequence, and get the personalized interval, which is specifically expressed as: in, represents the shortest time interval, t i -t j represents the time interval between two time points i and j, i = 1, 2, 3, ... n, j = 1, 2, 3, ... n; The time interval relationship matrix is ​​constructed as: Among them, M u Represents the time interval relationship matrix, which is a symmetric matrix with main diagonal elements equal to 0.

5. The ship motion trajectory prediction method based on ship track big data according to claim 1 is characterized in that: The extracted trajectory features and the time interval to be predicted are discretized into corresponding one-hot encoding vectors and an embedding vector is obtained, and then the embedding vectors of the two are merged into a joint feature embedding vector, specifically including: Get trajectory features It is scattered into a one-hot encoding vector: Then the state feature embedding vector is obtained through the embedding search operation: Get the one-hot encoded vector of the time interval to be predicted And the embedding vector of the time interval to be predicted The joint feature embedding vector after merging the state feature embedding vector and the time interval embedding vector to be predicted is expressed as: Among them, lat represents latitude, lon represents longitude, SOG represents speed over the ground, and COG represents course over the ground.

6. The ship motion trajectory prediction method based on ship track big data according to claim 5 is characterized in that: The fixed-length ship trajectory point sequence is stacked to obtain the joint feature embedding matrix A, which is specifically expressed as: in, For an n-row d model A real matrix with columns, n represents the trajectory length, d model Represents the model's latent dimension.

7. The ship motion trajectory prediction method based on ship track big data according to claim 4 is characterized in that: The time interval relationship matrix is ​​obtained through the embedding layer to obtain the time interval relationship embedding matrix B, which is specifically expressed as: Among them, each item in the time interval relationship matrix The embedding vector is obtained by embedding search The dimension of the time interval relationship embedding matrix B is transformed, and the transformed matrix is ​​C, where The dimension is consistent with the joint feature embedding matrix A, which is an n-row d model A real matrix with columns, n represents the trajectory length, d model Represents the model’s latent dimensions; Embed the position information sequence to obtain the position feature embedding matrix Then the obtained learnable position feature embedding matrix P is linearly added with the joint feature embedding matrix A and the time interval relationship matrix embedding matrix B as the input of the model, and finally the embedding matrix containing the position information and time interval relationship is obtained. Specifically expressed as: E=[A1+C1+P1,A2+C2+P2,…,A n +C n +P n ]。 8. The ship motion trajectory prediction method based on ship track big data according to claim 1 is characterized in that: The decoder is provided with N layers of decoding layers with completely identical structures and different parameters, wherein the decoding layers include a masked multi-head self-attention module, two residual connections and layer normalization, and a feedforward network; The masked multi-head self-attention module includes an input linear layer, a scaled dot product attention layer, a merging layer, and an output linear layer; The input linear layer inputs the query vector Q, the key vector K and the value vector V respectively, wherein the query vector Q, the key vector K and the value vector V are the embedding vectors E of the trajectory point at time t respectively. t Multiply by the weight matrix W Q , weight matrix W K , weight matrix W V get; The scaled dot product attention layer performs a self-attention operation; The merging layer merges the results of multiple attention heads; The output linear layer performs another spatial mapping on the merged result and outputs the attention information, and its output is the output result of the masked multi-head self-attention module.

9. The ship motion trajectory prediction method based on ship track big data according to claim 1 is characterized in that: The scaled dot product attention layer performs a self-attention operation, specifically including: Calculate the similarity between the query vector Q and each key vector K, and use the dot product method to obtain the weight; Use the Softmax function to normalize the weights; The weight and the corresponding value vector V are weighted and summed to obtain the final self-attention vector. The formula for the self-attention operation is as follows: Among them, d K is the dimension of vector Q; The merging layer merges the results of multiple attention heads, expressed as: MultiHead(Q,K,V)=Concat(head1,head2,…,head h )W O where head i =Attention(QW i Q ,KW i K ,VW i V ) Where h is the number of attention heads, is the transformation parameter of the data; are the transformation parameters of the query vector Q, key vector K and value vector V respectively. Concat represents the concat function, which merges the generated results of each attention head.

10. The ship motion trajectory prediction method based on ship track big data according to claim 1, characterized in that: The residual and layer normalization are specifically expressed as: sub_layer_output=x+SubLayer(LayerNorm(x)) Among them, x represents the input, SubLayer is the residual operation, and LayerNorm is the layer normalization operation.

Images