Multi-ship navigation trajectory prediction method in complex navigation environment

Abstract

The invention discloses a multi-ship navigation trajectory prediction method in a complex navigation environment. The method comprises the following steps: acquiring AIS trajectory data of N ships; performing data cleaning operation on the AIS data of the N ships to obtain preprocessed AIS data; dividing a data set formed by the preprocessed AIS data into a training set and a test set; constructing an end-to-end graph convolutional neural network prediction model; based on the training set data, training an end-to-end graph convolutional neural network prediction model to obtain a trained end-to-end graph convolutional neural network prediction model; and inputting test set data into the trained end-to-end graph convolutional neural network prediction model to realize prediction of future trajectories of N ships. According to the method, feature representation of a graph structure is constructed based on AIS data of a ship, then a space-time graph convolutional network is used as an encoder to extract space-time features of a ship trajectory, and the space-time features are input into a decoder formed by connecting a convolutional neural network and a residual error, so that trajectory prediction of the inland ship is realized.

Description

Technical Field

[0001] The invention belongs to the field of fully automated products and relates to a trajectory prediction method for multiple ships sailing in a complex navigation environment. Background Art

[0002] The rapid development of the global trade economy is increasingly dependent on a large number of maritime transport exchanges and cooperation. With the increase in the number of maritime trade and ships used, the maritime regulatory system has also brought unprecedented pressure. In order to maintain regular maritime commercial activities, it is very important to compile a maritime safety management manual and build a safe, stable and effective maritime transport channel. Moreover, the safety of maritime transport has always been a topic of great concern to all countries. The navigation safety of inland ships in complex navigation environments is one of the important challenges facing the current intelligent ship system. However, due to a series of factors such as the changeable marine climate, the complex distribution of submarine reefs, and the large differences in ocean current changes, certain safety risks of standard navigation of maritime transport cannot be ignored. In order to ensure the safety of maritime traffic, it is very important to accurately predict the future trajectory of maritime ships and improve the situational awareness of ships in complex navigation environments. This can avoid a large number of collision avoidance accidents for ships, reduce the risk of maritime navigation, and thus improve the efficiency of maritime transportation.

[0003] In a crowded environment, being able to accurately predict the specific position of the ship itself and other ships can facilitate the adjustment of its trajectory to avoid collision. In the marine field, although the safe navigation of ships is governed by the rules for avoiding collisions at sea, it depends to a large extent on the navigation experience of the crew and their ability to imitate implicit rules. Therefore, the intention prediction of adjacent ships in complex and crowded waters is one of the important means to ensure the safety and efficiency of maritime transportation.

[0004] Early ship trajectory prediction was based on a physical model of ship motion, combining kinematic laws with mathematical equations, and taking into account all factors that may affect ship motion, such as ship mass, size, inertia, etc. The accuracy of this method relies on a very precise ideal representation of the environment and state assumptions, which is difficult to achieve in most actual ship trajectory prediction scenarios. Later, researchers used historical and real-time ship AIS data to model ship motion based on previous and current AIS data, implicitly integrating all factors that may affect the ship's movement behavior, thereby achieving the purpose of predicting the next position of the ship. With the development of artificial intelligence, the use of deep learning technology has become a popular trajectory prediction method, using LSTM or GRU to encode the ship's historical AIS trajectory into a context vector, and using a decoder to output the ship's future trajectory sequence.

[0005] In response to the navigation safety of inland ships, the ship autonomous navigation system has formulated a ship safety management system and navigation standard process specifications to improve the requirements for ship safety emergency operations and reduce the occurrence of marine ship accidents. In order to improve the navigation capacity of ships, the intelligent trajectory prediction of inland ships plays a vital role in the ship intelligent navigation and intelligent collision avoidance system. The current ship traffic flow prediction method only uses the ship position coordinates at each time step to form a feature vector, and uses the long short-term memory network (LSTM) to predict the next position of the ship. The prediction accuracy is poor, and the spatial relationship between each ship is not explicitly considered. Summary of the invention

[0006] In order to solve the problem that the current ship traffic flow prediction method only uses the ship position coordinates at each time step to form a feature vector, and uses a long short-term memory network (LSTM) to predict the next position of the ship, the prediction accuracy is poor, and the spatial relationship between each ship is not clearly considered, the technical solution adopted by the present invention is: a trajectory prediction method for multiple ships in a complex navigation environment, comprising the following steps:

[0007] Get the automatic identification system trajectory data of N ships;

[0008] Preprocessing the automatic identification system trajectory data of N ships to obtain preprocessed ship trajectory data;

[0009] The data set consisting of the preprocessed ship trajectory data is divided into a training set and a test set;

[0010] Build an end-to-end graph convolutional neural network prediction model;

[0011] Based on the training set data, the end-to-end graph convolutional neural network prediction model is trained to obtain a trained end-to-end graph convolutional neural network prediction model;

[0012] The test set data is input into the trained end-to-end graph convolutional neural network prediction model to predict the future trajectories of N ships.

[0013] Further: the end-to-end graph convolutional neural network prediction model includes:

[0014] Multi-graph representation module of ship trajectories: used to integrate the social interaction information of the position-based interaction graph and the speed-based interaction graph between ships into a multi-graph to construct a multi-graph representation of ship trajectories;

[0015] Spatiotemporal feature extraction module: used to extract high-level spatiotemporal features of ship trajectories based on multi-graph representation of ship trajectories;

[0016] Trajectory prediction module: It is used to predict the future estimates of N ships based on the high-level spatiotemporal features of the extracted ship trajectories using a bidirectional gated recurrent unit network.

[0017] Further: the spatiotemporal feature extraction module includes:

[0018] A spatiotemporal graph convolutional network is used to perform spatiotemporal convolution operations on the multi-graph representation of ship trajectories and extract the spatial social interaction features between ships;

[0019] A temporal convolutional network with residual connections is used to capture the temporal dependencies between the spatial features of ship trajectories based on the extracted spatial social interaction features between ships, and obtain a high-level feature representation of the historical trajectory data, which contains strong spatiotemporal correlations between ships.

[0020] Further: The process of obtaining the kernel function of the position-based interaction graph is as follows:

[0021] The relative positions of ships to each other are used to define position-based interactions. The movement of ships is more likely to be affected by nearby neighboring ships. The geometric interconnections of the scene are aggregated. First, a set of spatial graphs are created. represents the relative position of the ship in the scene at each time step t, Defined as Where V t =V={p i |i∈{1,...,N}} is the set of ships at time step t, is the adjacency matrix In order to describe the position-based interactions between ships and the intensity of the mutual influence between two ships at time step t, the following kernel function is used:

[0022]

[0023] Where i and j are the indicators of the two ships in the observation scene, is the position of ship i at time step t, is the position of ship j at time step t; ||·||2 is the L2 norm.

[0024] Further: the kernel function of the speed-based interaction graph is obtained as follows:

[0025] Use the instantaneous speed of the ship in the scene at time step t to define the speed-based interaction; use the ship speed feature to construct the second graph Where V t =V={p i |i∈{1,...,N}} is the set of ships at time step t, is a set of edges representing the speed correlation between graph nodes, represented by the adjacency matrix Definition: The kernel function used to represent velocity-based interactions is as follows,

[0026]

[0027] in is the instantaneous speed of ship i; is the instantaneous speed of ship j.

[0028] Further: The construction process of the spatiotemporal graph convolutional network is as follows:

[0029] The spatial social interaction characteristics of a single ship are represented by the adjacency matrix A and the identity matrix I. First, the adjacency matrix A is normalized, and the adjacency matrix A is {A1,...,A T}, using the symmetric normalized Laplacian operator to normalize each A t , which is used to maintain the value range of feature maps after graphics operations:

[0030]

[0031] in Λ t yes The diagonal node degree matrix of and Λ are defined respectively and Λ t The stacking of

[0032] Set the feature matrix of network layer l at time step t to V t (l) , assuming V (l) Yes V t (l) The stacking of , implements the spatiotemporal graph convolutional neural network layer defined in the above formula, as shown below:

[0033]

[0034] Where: W (l) is the trainable parameter matrix of layer l. After applying the spatiotemporal graph convolutional neural network, the spatial social interaction features of the ship are obtained. The obtained feature embedding vector is represented as

[0035] Further: the temporal convolutional network with residual connection is used to capture the temporal dependency between the spatial features of the ship trajectories based on the extracted spatial social interaction features between the ships, and obtain a high-level feature representation of the historical trajectory data, wherein the formula containing the strong spatiotemporal correlation between the ships is:

[0036]

[0037] Among them: as input, the motion trend of each ship is learned from the past behavior of each ship and the dynamic interaction of nearby ships, V f The shape is T P ×C f A tensor of size N × N.

[0038] Furthermore: the temporal convolutional network spatiotemporal graph convolutional network with residual connection is composed of a TCN network and a GCN network through residual connection.

[0039] Further: the trajectory prediction module adopts a bidirectional GRU network, which is used to predict the future estimates of N ships based on the extracted high-level spatiotemporal features of the ship trajectory and a bidirectional gated recurrent unit network as follows:

[0040] GRU network updates gate z t and reset gate r t Control and update, that is, update gate z t Determine the ratio of the state information of the previous moment to the current state, reset the gate r t Determines the degree of forgetting historical state information, thereby controlling the information flow and ensuring the transmission of important time series data; where x t is the current input state, h t-1 Define the previous hidden state, the hidden state of the current step h t and the output state y of the current step t Calculated by the following formula:

[0041]

[0042] where σ(·) is the sigmoid function, which serves as a gating signal to convert the intermediate state into a value between 0 and 1, and W xr , W hr , W xz , W hz , W xh' , W hh' and W o is the parameter matrix, ⊙ represents the Hadamard product; using the above formula, all hidden states and output states are obtained through cyclic update;

[0043] The bidirectional GRU network input is the high-level spatiotemporal features of the ship trajectory obtained from the spatiotemporal feature extraction module. In order to meet the input specification of the prediction model, it is reconstructed into a shape of (T p ×N×C f ), at the current time step t, the hidden state h of the bidirectional GRU network tBy input V f , the hidden state output of the forward layer and the hidden state output of the backward layer Determine, by connecting at time step t and To obtain the hidden layer state of the bidirectional GRU network, as shown below:

[0044]

[0045] Where [·] represents the connection operation between elements, and the output of the bidirectional GRU network is as follows:

[0046] y t =σ(h t ·W o ) (8)

[0047] All states of the bidirectional GRU network are concatenated into tensors Among them C hid Represents the hidden layer size of GRU, using tensor y out , the input parameter size is C hid *2, output size is C f The linear layer generates the probability distribution of future trajectories, that is, the final output of the trajectory prediction module is The future coordinates predicted by the trajectory prediction module Depend on Given.

[0048] A water traffic monitoring device in a complex navigation environment with multiple ships, comprising:

[0049] Acquisition module: used to obtain AIS track data of N ships;

[0050] Preprocessing module: used to perform data cleaning operations on the AIS data of N ships to obtain the preprocessed AIS data;

[0051] Partition module: used to divide the data set consisting of the pre-processed AIS data into a training set and a test set;

[0052] Building module: used to build an end-to-end graph convolutional neural network prediction model;

[0053] Training module: used to train the end-to-end graph convolutional neural network prediction model based on the training set data to obtain a trained end-to-end graph convolutional neural network prediction model;

[0054] Prediction module: used to input the test set data into the trained end-to-end graph convolutional neural network prediction model to predict the future trajectories of N ships.

[0055] The present invention provides a trajectory prediction method for multiple ships in a complex navigation environment, and develops a new end-to-end graph convolutional neural network prediction model based on AIS data to predict the future trajectory of inland ships. First, a feature representation of the graph structure is constructed based on the AIS data of the ship, and then the spatiotemporal graph convolutional network is used as an encoder to extract the spatiotemporal features of the ship trajectory, which are input into a decoder composed of a convolutional neural network and residual connections to achieve trajectory prediction of inland ships.

[0056] This application has the following advantages:

[0057] The present invention takes into account the social interaction between ships at sea, which is very necessary for ships in complex navigation environments. The target ship will be affected by the navigation position of adjacent ships during navigation. Therefore, the social interaction between ships is an important factor to be considered in the ship trajectory prediction process. We use graph convolutional neural networks to model the social interaction between ships. The nodes represent the ship positions, and the edge sets represent the social interactions between ships. The temporal and spatial features of the ship trajectories are extracted through the spatiotemporal feature extraction module, which contains more semantic information. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0059] Figure 1 It is a flow chart of the preprocessing process of AIS data;

[0060] Figure 2 It is the architecture of an end-to-end graph convolutional neural network prediction model;

[0061] Figure 3 This is an example of the execution process of a graph convolutional network;

[0062] Figure 4 It is the input and output structure of RT-CNN;

[0063] Figure 5 It is the structural diagram of GRU;

[0064] Figure 6 Execution structure diagram of BiGRU. DETAILED DESCRIPTION

[0065] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments may be combined with each other. The present invention will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0066] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. The following description of at least one exemplary embodiment is actually only illustrative and is by no means intended to limit the present invention and its application or use. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0067] A method for predicting the trajectories of multiple ships in a complex navigation environment comprises the following steps:

[0068] S1: Obtain AIS (Automatic Identification System) track data of N ships; N ≥ 1;

[0069] S2: Perform data cleaning operations on the AIS track data of N ships to obtain pre-processed AIS data;

[0070] S3: Divide the data set consisting of the preprocessed AIS trajectory data into a training set and a test set;

[0071] S4: Build an end-to-end graph convolutional neural network prediction model;

[0072] S5: Based on the training set data, the end-to-end graph convolutional neural network prediction model is trained to obtain a trained end-to-end graph convolutional neural network prediction model;

[0073] S6: Input the test set data into the trained end-to-end graph convolutional neural network prediction model to predict the future trajectories of N ships.

[0074] This method is applicable to the prediction of the future trajectory of a ship when sailing in an inland river or ocean;

[0075] Figure 1 It is a flow chart of the preprocessing process of AIS data;

[0076] The AIS trajectory data includes: extracting characteristic information of fields such as MMSI (unique ship ID), BaseDateTime (timestamp), LAT (latitude), LON (longitude), SOG (ground speed), and Heading (heading angle), and resampling the AIS data to 1 minute intervals.

[0077] The process of performing data cleaning operation on the AIS data of N ships to obtain the pre-processed AIS data is as follows:

[0078] For ships transmitting AIS information at a rate lower than one minute, linear interpolation is used to fill in missing data and duplicate timestamps are removed.

[0079] Then the data is preprocessed, such as Figure 2 As shown, firstly, outliers in AIS data are filtered out, AIS information of ships not at sea is removed, timestamps with AIS information from moored / anchored ships with recorded SOG values ​​less than 1 knot are removed, and concurrent AIS information with less than three ships at the timestamp is removed. Redundant AIS data are eliminated, and linear interpolation is performed on missing data.

[0080] Figure 2 It is the architecture of an end-to-end graph convolutional neural network prediction model;

[0081] The end-to-end graph convolutional neural network prediction model includes:

[0082] Multi-graph representation module of ship trajectories: used to integrate the social interaction information of the position-based interaction graph and the speed-based interaction graph between ships into a multi-graph to construct a multi-graph representation of ship trajectories;

[0083] Spatiotemporal feature extraction module: used to extract high-level spatiotemporal features of ship trajectories based on multi-graph representation of ship trajectories;

[0084] Trajectory prediction module: It is used to predict the future estimates of N ships based on the high-level spatiotemporal features of the extracted ship trajectories using a bidirectional gated recurrent unit network.

[0085] The spatiotemporal feature extraction module comprises:

[0086] The spatiotemporal graph convolutional network (ST-GCN) is used to perform spatiotemporal convolution operations on the multi-graph representation of ship trajectories and extract the spatial social interaction features between ships;

[0087] A temporal convolutional network with residual connections (RT-CNN) is used to capture the temporal dependencies between the spatial features of ship trajectories based on the extracted spatial social interaction features between ships, and obtain a high-level feature representation of the historical trajectory data, which contains strong spatiotemporal correlations between ships.

[0088] The process of constructing a multi-graph representation of a ship's trajectory is as follows:

[0089] Suppose G = (V, E) is an undirected multigraph, where V represents the vertex and E represents the edge set. Represents a collection of n ships. Vertices can store changes in information such as position, speed, and direction. Use multiple adjacency matrices to define the edge set E of the graph G. Each matrix represents a specific interaction between ships (vertices). In order to simulate how two ships affect each other, give each e ij ∈E assigns a value a ij , which is calculated by the kernel function. For each time step t, a graph G will be created.

[0090] In the overall end-to-end graph convolutional neural network prediction model, two types of interactions are used, the first is a position-based interaction graph, and the second is a speed-based interaction graph.

[0091] Position-based interaction graph: The relative positions of ships are used to define position-based interactions. The movement of ships is more likely to be affected by nearby neighboring ships, which aggregates the geometric interconnection of the scene. First, create a set of spatial graphs Represents the relative position of the ship in the scene at each time step t. Defined as Where V t =V={p i |i∈{1,...,N}} is the set of ships at time step t, is the adjacency matrix In order to describe the position-based interactions between ships and the strength of the mutual influence between two ships at time step t, the following kernel function is used,

[0092]

[0093] Among them: i and j are the indicators of the two ships in the observation scene, is the position of ship i at time step t, is the position of ship j at time step t. ||·||2 is the L2 norm.

[0094] Speed-based interaction graph: The speed-based interaction is defined using the instantaneous speed of the ships in the scene at time step t. The speed of the ship is a valuable information to understand whether the ships will interact. It can sense whether the two ships will collide at a certain point in the future. This feature is very important for predicting the future trajectory of the ship, so this feature is used to construct the second graph. Where V t =V={p i |i∈{1,...,N}} is the set of ships at time step t, is a set of edges representing the speed correlation between graph nodes, represented by the adjacency matrix Definition. The kernel function used to represent the velocity-based interaction is as follows,

[0095]

[0096] in: is the instantaneous speed of ship i; is the instantaneous speed of ship j.

[0097] In the ship trajectory spatiotemporal feature extraction module, the spatiotemporal graph convolutional network (ST-GCN) is used to perform spatiotemporal convolution operations on the graph representation of the ship trajectory, which is responsible for extracting the spatial social interaction features between ships. Then we use the temporal convolutional network with residual connection (RT-CNN) to capture the temporal dependency between the spatial interaction features, thereby obtaining a high-level spatiotemporal feature representation of the ship historical trajectory data. The specific process is as follows:

[0098] The spatial social interaction characteristics of a single ship are represented by the adjacency matrix A and the identity matrix I. In order to facilitate learning, the adjacency matrix is ​​first normalized. The adjacency matrix A is {A1,...,A T}, using the symmetric normalized Laplacian operator to normalize each A t , used to maintain the value range of feature maps after graphics operations.

[0099]

[0100] in Λ t yes The diagonal node degree matrix of . Use and Λ are defined respectively and Λ t The stacking and normalization of the adjacency matrix are crucial for the normal operation of graph convolutional neural networks, which can accelerate the learning process of GCN.

[0101] Set the feature matrix of network layer l at time step t to V t (l) , assuming V (l) Yes V t (l) According to the above definition, the spatiotemporal graph convolutional network defined in the above formula can be realized as follows:

[0102]

[0103] Where:W (l) is the trainable parameter matrix of layer l. After applying the spatiotemporal graph convolutional neural network, the spatial social interaction features of the ship are obtained, and the obtained feature embedding vector is represented as The execution process of the spatiotemporal graph convolutional network is as follows Figure 3 shown.

[0104] After capturing the neighboring node information of each vertex through the spatiotemporal graph convolutional network, a three-dimensional tensor containing the spatial social interaction features between ships is obtained. Among them C f =5 is the parameter dimension of the bivariate Gaussian distribution, T o is the length of the observation sequence sample, and N is the number of nodes in the graph. Then, we can get Contains T o feature maps of size C f × N. RT-CNN As input, the motion trend of each ship is learned from the past behavior of each ship and the dynamic interaction of nearby ships. The operation formula is as follows:

[0105]

[0106] Note that only the first layer in RT-CNN has no residual connections because it receives features from the encoder. The sizes of the observed sequence samples and the predicted sequence samples are different. Therefore, the output V of RT-CNN is f The shape is T P ×C f ×N tensor, such as Figure 4 As shown in Figure 2, RT-CNN can extract temporal correlation by learning the mapping relationship between input and future temporal features, so the spatiotemporal feature representation of the ship trajectory is obtained through the RT-CNN module.

[0107] The temporal convolutional network with residual connection is composed of a TCN network and a GCN network connected by residual connections, capturing the temporal dependency between the spatial features of the ship trajectory;

[0108] After the spatiotemporal feature extraction module captures the high-level spatiotemporal features of the ship trajectory, the trajectory prediction module can be used to generate the future trajectory distribution of the ship.

[0109] The trajectory prediction module adopts a bidirectional gated recurrent unit network, takes the advanced spatiotemporal feature embedding of the ship trajectory data as input, and outputs the future position distribution sequence of the ship, thereby realizing the future trajectory prediction of inland ships;

[0110] This is a typical sequence generation task. Because the traditional recurrent neural network (RNN) repeats the same operation at each time node, its calculation depth increases, and it faces problems such as gradient explosion, gradient vanishing, and inability to handle long-term dependencies. Inspired by the excellent performance and low computational cost of GRU in sequence-based generation tasks, the trajectory prediction module is designed as a bidirectional GRU network (BiGRU). The architecture of GRU is as follows: Figure 5 shown.

[0111] In BiGRU (Bidirectional Gated Recurrent Unit Network), the input state is updated by gate z t and reset gate r t Control and update, that is, update gate z t Determine the ratio of the state information of the previous moment to the current state, reset the gate r t Determines the degree of forgetting historical state information, thereby controlling the information flow and ensuring the transmission of important time series data. t is the current input state, h t-1 Define the previous hidden state, the hidden state of the current step h t and the output state y of the current step t It can be calculated by the following formula.

[0112]

[0113] Where σ(·) is the sigmoid function, which serves as a gating signal to convert the intermediate state into a value between 0 and 1. xr , W hr , W xz , W hz , W xh' , W hh' and W o is the parameter matrix. ⊙ represents the Hadamard product. Using the above formula, all hidden states and output states can be obtained through cyclic updates.

[0114] The input of the trajectory prediction module is the high-level spatiotemporal features of the ship trajectory obtained from the spatiotemporal feature extraction module. In order to meet the input specification of the prediction model, it is reconstructed into a shape of (T p ×N×C f ). Figure 6 As shown, at the current time step t, the hidden state h of BiGRU t By input V f , the hidden state output of the forward layer and the hidden state output of the backward layer Since BiGRU can be viewed as two independent GRUs, it can be connected at time step t and To obtain the hidden layer state of BiGRU, as shown below:

[0115]

[0116] Where [·] represents the connection operation between elements. The output of BiGRU is as follows:

[0117] y t =σ(h t ·W o ) (8)

[0118] All states of BiGRU are concatenated into a tensor Among them C hid Represents the hidden layer size of GRU. Using tensor y out , the input parameter size is C hid *2, output size is C f The linear layer generates the probability distribution of future trajectories, that is, the final output of the trajectory prediction module is The future coordinates predicted by the trajectory prediction module can be Given.

[0119] In order to verify the effectiveness of the proposed method, we collected AIS data downloaded from the National Oceanic and Atmospheric Administration (NOAA) in March 2022, with the latitude and longitude ranges of (21゜, 31゜) and (-95゜, -83゜), and compared them with the current prediction methods. Table 1 shows the comparison results of different models.

[0120] Table 1. Comparison of the results of our proposed method and other baseline methods.

[0121]

[0122]

[0123] From Table 1, it can be found that the prediction performance of Vanila_LSTM and GRU based on deep learning is not much different. The prediction performance of GRU is slightly higher than LSTM, which is significantly better than the current machine learning methods. The ST-Seq2seq model predicts that the ADE of the future trajectory of the ship within 5 minutes is 0.5226 nautical miles and the FDE is 0.7661 nautical miles, which is significantly better than Vanila_LSTM and GRU. Because the Vanila_LSTM and GRU models are equivalent to only applying the encoder part of ST-Seq2seq, and ST-Seq2seq can decode the ship sequence information captured by the encoder by adding the decoder part, so as to obtain a deeper internal relationship between the ship trajectories, and predict the distribution of the future trajectory of the ship based on the intermediate state vector, but does not consider the spatial social interaction between ships in complex scenarios. In fact, in the real maritime navigation environment, the social interaction between adjacent ships has an important impact on the future navigation intention of the target ship. The LSTM+spatial+temporal attention model extracts the temporal features of the ships through the temporal attention mechanism and the spatial features between the ships through the spatial attention mechanism, which makes the model have smaller ADE and FDE compared with the method that does not consider the interaction between ships, with an ADE value of 0.4514 nautical miles and an FDE value of 0.5820 nautical miles. CNN-Bi-LSTM uses CNN to obtain the spatial features between the ship trajectories, and then uses a bidirectional LSTM network to capture the temporal features of the trajectory data. Finally, a three-layer fully connected neural network is used to predict the future trajectory of the ship, and it has the smallest ADE value of 0.3860 nautical miles compared with other baseline models. Our proposed model achieves the best prediction results in both ADE and FDE indicators, with an ADE value of 0.1009 nautical miles and an FDE value of 0.1900 nautical miles. Compared with the best baseline model, ADE is improved by 73.86% and FDE is improved by 67.35%.

[0124] We chose the stochastic gradient descent (SGD) optimizer, which combines the advantages of the gradient descent algorithm with adaptive learning rate and the advantages of the momentum gradient descent algorithm, which can not only adapt to sparse gradients but also alleviate the gradient oscillation problem. The initial learning rate is set to 0.0001. The end-to-end graph convolutional neural network prediction model is trained for 200 epochs to minimize the loss function, the learning rate is adaptively reduced for each training epoch, and the batch size is set to 64. During inference, the bivariate distribution of future trajectories is predicted, including the mean, standard deviation, and correlation coefficient of each target at each timestamp. The position distribution of the ship can then be sampled from this distribution and the future trajectory of the ship can be predicted. All experiments were performed on an NVIDIA 3060 GPU under the PyTorch framework.

[0125] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for predicting the trajectory of multiple ships in a complex navigation environment, characterized by: The following steps are involved: Get the automatic identification system trajectory data of N ships; Preprocessing the automatic identification system trajectory data of N ships to obtain preprocessed ship trajectory data; The data set consisting of the preprocessed ship trajectory data is divided into a training set and a test set; Build an end-to-end graph convolutional neural network prediction model; Based on the training set data, the end-to-end graph convolutional neural network prediction model is trained to obtain a trained end-to-end graph convolutional neural network prediction model; The test set data is input into the trained end-to-end graph convolutional neural network prediction model to predict the future trajectories of N ships.

2. The method for predicting the trajectories of multiple ships in a complex navigation environment according to claim 1 is characterized by: The end-to-end graph convolutional neural network prediction model includes: Multi-graph representation module of ship trajectories: used to integrate the social interaction information of the position-based interaction graph and the speed-based interaction graph between ships into a multi-graph to construct a multi-graph representation of ship trajectories; Spatiotemporal feature extraction module: used to extract high-level spatiotemporal features of ship trajectories based on multi-graph representation of ship trajectories; Trajectory prediction module: It is used to predict the future estimates of N ships based on the high-level spatiotemporal features of the extracted ship trajectories using a bidirectional gated recurrent unit network.

3. The method for predicting the trajectories of multiple ships in a complex navigation environment according to claim 2 is characterized by: The spatiotemporal feature extraction module comprises: A spatiotemporal graph convolutional network is used to perform spatiotemporal convolution operations on the multi-graph representation of ship trajectories and extract the spatial social interaction features between ships; A temporal convolutional network with residual connections is used to capture the temporal dependencies between the spatial features of ship trajectories based on the extracted spatial social interaction features between ships, and obtain a high-level feature representation of the historical trajectory data, which contains strong spatiotemporal correlations between ships.

4. The method for predicting the trajectories of multiple ships in a complex navigation environment according to claim 1 is characterized by: The kernel function of the position-based interaction graph is obtained as follows: The relative positions of ships to each other are used to define position-based interactions. The movement of ships is more likely to be affected by nearby neighboring ships. The geometric interconnections of the scene are aggregated. First, a set of spatial graphs are created. represents the relative position of the ship in the scene at each time step t, Defined as Where V t =V={p i |i∈{1,...,N}} is the set of ships at time step t, is the adjacency matrix In order to describe the position-based interactions between ships and the intensity of the mutual influence between two ships at time step t, the following kernel function is used: Where i and j are the indicators of the two ships in the observation scene, is the position of ship i at time step t, is the position of ship j at time step t; ||·||2 is the L2 norm.

5. The method for predicting the trajectories of multiple ships in a complex navigation environment according to claim 1 is characterized by: The kernel function of the speed-based interaction graph is obtained as follows: Use the instantaneous speed of the ship in the scene at time step t to define the speed-based interaction; use the ship speed feature to construct the second graph Where V t =V={p i |i∈{1,...,N}} is the set of ships at time step t, is a set of edges representing the speed correlation between graph nodes, represented by the adjacency matrix Definition: The kernel function used to represent velocity-based interactions is as follows, in is the instantaneous speed of ship i; is the instantaneous speed of ship j.

6. The method for predicting the trajectories of multiple ships in a complex navigation environment according to claim 3 is characterized by: The construction process of the spatiotemporal graph convolutional network is as follows: The spatial social interaction characteristics of a single ship are represented by the adjacency matrix A and the identity matrix I. First, the adjacency matrix A is normalized, and the adjacency matrix A is {A1,...,A T }, using the symmetric normalized Laplacian operator to normalize each A t , used to maintain the value range of feature maps after graphics operations: in Λ t yes The diagonal node degree matrix of and Λ are defined respectively and Λ t The stacking of Set the feature matrix of network layer l at time step t to V t (l) , assuming V (l) Yes V t (l) The stacking of , implements the spatiotemporal graph convolutional neural network layer defined in the above formula, as shown below: Where: W (l) is the trainable parameter matrix of layer l. After applying the spatiotemporal graph convolutional neural network, the spatial social interaction features of the ship are obtained. The obtained feature embedding vector is represented as 7. The method for predicting the trajectories of multiple ships in a complex navigation environment according to claim 3 is characterized by: The temporal convolutional network with residual connection is used to capture the temporal dependency between the spatial features of ship trajectories based on the extracted spatial social interaction features between ships, and obtain a high-level feature representation of the historical trajectory data, wherein the formula containing the strong spatiotemporal correlation between ships is: Among them: as input, the motion trend of each ship is learned from the past behavior of each ship and the dynamic interaction of nearby ships, V f The shape is T P ×C f A tensor of size N × N.

8. The method for predicting the trajectory of multiple ships in a complex navigation environment according to claim 3 is characterized by: The temporal convolutional network with residual connection and the spatiotemporal graph convolutional network are composed of a TCN network and a GCN network through residual connections.

9. The method for predicting the trajectories of multiple ships in a complex navigation environment according to claim 1 is characterized by: The trajectory prediction module adopts a bidirectional GRU network. Based on the extracted high-level spatiotemporal features of the ship trajectories, the process of using the bidirectional GRU network to predict the future trajectories of N ships is as follows: The GRU network state is updated by gate z t and reset gate r t Control and update, that is, update gate z t Determine the ratio of the state information of the previous moment to the current state, reset the gate r t Determines the degree of forgetting historical state information, thereby controlling the information flow and ensuring the transmission of important time series data; where x t is the current input state, h t-1 Define the previous hidden state, the hidden state of the current step h t and the output state y of the current step t Calculated by the following formula: where σ(·) is the sigmoid function, which serves as a gating signal to convert the intermediate state into a value between 0 and 1, and W xr , W hr , W xz , W hz , W xh' , W hh' and W o is the parameter matrix, ⊙ represents the Hadamard product; using the above formula, all hidden states and output states are obtained through cyclic update; The bidirectional GRU network input is the high-level spatiotemporal features of the ship trajectory obtained from the spatiotemporal feature extraction module. In order to meet the input specification of the prediction model, it is reconstructed into a shape of (T p ×N×C f ), at the current time step t, the hidden state h of the bidirectional GRU network t By input V f , the hidden state output of the forward layer and the hidden state output of the backward layer Determine, by connecting at time step t and To obtain the hidden layer state of the bidirectional GRU network, as shown below: Where [·] represents the connection operation between elements, and the output of the bidirectional GRU network is as follows: y t =σ(h t ·W o ) (8) All states of the bidirectional GRU network are concatenated into tensors Among them C hid Represents the hidden layer size of GRU, using tensor y out , the input parameter size is C hid *2, output size is C f The linear layer generates the probability distribution of future trajectories, that is, the final output of the trajectory prediction module is The future coordinates predicted by the trajectory prediction module Depend on Given.

10. A water traffic monitoring device in a complex navigation environment with multiple ships, characterized by: include: Acquisition module: used to obtain AIS track data of N ships; Preprocessing module: used to perform data cleaning operations on the AIS track data of N ships to obtain the preprocessed AIS track data; Partition module: used to divide the data set consisting of the pre-processed AIS trajectory data into a training set and a test set; Building module: used to build an end-to-end graph convolutional neural network prediction model; Training module: used to train the end-to-end graph convolutional neural network prediction model based on the training set data to obtain a trained end-to-end graph convolutional neural network prediction model; Prediction module: used to input the test set data into the trained end-to-end graph convolutional neural network prediction model to predict the future trajectories of N ships.

Images