Dynamic spatial-temporal graph attention method for ship trajectory prediction

Abstract

Disclosed in the present invention is a dynamic spatial-temporal graph attention method for ship trajectory prediction, comprising the following steps: spatial graph learning: using a graph neural network to aggregate neighbor embeddings of a target entity to learn a local structure representation thereof, then calculating the effect of neighbor entities and a collaborative relationship on the target entity by means of a knowledge-aware graph attention mechanism, and finally fusing semantics and attributes from different neighbors by means of an information aggregation operation to refine the embedding of the target entity; temporal graph fusion: using a temporal graph attention layer to learn a global temporal dependency relationship of ship trajectory data, and providing interpretability for a model, then using a GRU-based sequence encoder to replace positional encoding in a conventional Transformer architecture, and capturing more local temporal dependency relationships while providing a more reasonable inductive bias for an input sequence; and feature decoding: decoding a trajectory vector having spatial-temporal attributes, generating future relative position coordinates of a ship, and achieving high-precision ship trajectory prediction.

Description

A dynamic spatiotemporal graph attention method for ship trajectory prediction Technical Field

[0001] The present invention relates to the field of ship trajectory prediction, and in particular to a dynamic spatiotemporal graph attention method for ship trajectory prediction. Background Art

[0002] Ship trajectory prediction uses an algorithmic model to predict a ship's trajectory over the next period of time by analyzing ship data and environmental factors. This is important for early warning of potential collisions and intelligent shipping management. However, accurate ship trajectory prediction is a challenging problem for the following reasons:

[0003] 1) Disturbances from the external environment and interactions between adjacent ships, such as ship suction effect, wave loads, structural stress, and wind, can cause significant deviations between the actual route and the intended direction.

[0004] 2) Small fluctuations in the observable trajectory distribution will be amplified during future voyages.

[0005] The reason is that compared to autonomous vehicles and pedestrians, the completion time of similar actions is longer, especially for large ships turning, accelerating, decelerating, berthing, and unberthing. This means that the reaction generated by a ship's actions may continue to affect the navigation of other nearby ships for some time to come, and the actions taken by other ships will also affect the trajectory of nearby ships.

[0006] Based on the dynamics and complexity of maritime traffic conditions mentioned above, it is difficult to predict the future position of a ship.

[0007] In order to solve the above problems, the common existing ship trajectory prediction methods are as follows:

[0008] Statistical model based on historical data: By analyzing historical ship trajectory data, extracting features, and using statistical methods to build models, commonly used methods include linear regression and ARIMA models.

[0009] Machine learning methods: Use machine learning algorithms such as decision trees, support vector machines, and random forests to build models based on historical trajectory data and environmental factors and predict future trajectories.

[0010] Deep learning methods: Use deep neural network models, such as recurrent neural networks (RNNs) or graph neural networks (GNNs), to learn the time series features of historical data and perform trajectory prediction.

[0011] Numerical simulation based on physical models: Based on physical principles and environmental factors, a dynamic model of ship motion is established, and predictions are made through numerical simulation methods combined with historical data.

[0012] Although the above ship trajectory prediction methods are constantly improving, they still have the following defects:

[0013] 1. Data sparsity: Ship trajectory data is usually obtained based on the historical records of ships, while ship activities are wide-ranging and diverse, resulting in data sparsity. This has a negative impact on the accuracy and generalization ability of the model.

[0014] 2. Lack of complete historical data: The historical records of ship trajectory data are usually limited and may only contain trajectory information for a short period of time. This limits the model's ability to fully understand and predict ship behavior.

[0015] 3. Complex environmental dynamics: Ship behavior is affected by a variety of environmental factors, such as ocean currents, wind speed, and tides. The uncertain and dynamic changes of these factors pose a challenge to the accuracy of trajectory prediction.

[0016] 4. Lack of ship characteristic information: Existing ship trajectory data typically only contains location and time information, but lacks detailed descriptions of the ship's characteristics. For example, characteristics such as ship type, size, and speed can have a significant impact on trajectory prediction, but are often difficult to obtain.

[0017] 5. Limited model generalization ability: Due to the particularity of ship trajectory data, some prediction models may show limited generalization ability when applied to other navigation environments or different types of ships.

[0018] As can be seen from the above, factors that need to be considered when predicting ship trajectories include environmental factors such as ship type, speed, wind speed, and currents, as well as ship characteristics and voyage objectives. These factors have a significant impact on the selection and development of prediction models. Furthermore, the quality and quantity of data also affect the accuracy of prediction results.

[0019] Therefore, it is necessary to improve the existing ship trajectory prediction method to overcome the above defects.

[0020] Summary of the Invention

[0021] The purpose of this invention is to provide a dynamic spatiotemporal graph attention method, which improves the accuracy of trajectory prediction by dynamically modeling ship navigation scenarios and applying it to ship track prediction tasks.

[0022] The above technical objectives of the present invention are achieved through the following technical solutions:

[0023] A dynamic spatiotemporal graph attention method for ship trajectory prediction includes the following steps:

[0024] Step 1: Spatial graph learning, using information propagation technology in graph neural networks to learn the local representation of the target ship in the situation awareness graph and obtain the target entity encoding;

[0025] Step 2, temporal graph fusion, uses a masked attention layer to learn long-term temporal dependencies from spatially dependent track vectors and a GRU-based sequence encoder-decoder to capture local temporal dependencies;

[0026] Step 3, feature decoding, decodes the trajectory vector with spatiotemporal attributes to generate the future relative position coordinates of the ship, achieving high-precision trajectory prediction tasks.

[0027] Furthermore, in step 1, the situation awareness graph G is set θ =(ε θ , R θ );

[0028] Where ε is the situation awareness graph G θ The vertex set of

[0029] R is the situational awareness map G θ The edge set in is represented as θ is the current moment of the ship, and the situation awareness map G θ is directed or undirected;

[0030] The local representation of the target ship in step 1 refers to the neighboring ship entities sailing around the target ship, which uses the adjacency matrix To model;

[0031] in, It represents the implicit communication of the target ship, Representation node and e j The strength of the relationship between the two ships is measured by the geographical proximity of the two ships.

[0032] Furthermore, the spatial graph learning in step 1 utilizes a knowledge-aware graph attention mechanism to calculate the influence of neighbor entities and collaborative relationships on the target entity, and then fuses the semantics and attributes from different neighbors through information aggregation operations to refine the embedding of the target entity.

[0033] Furthermore, the GRU-based sequence encoder-decoder in step 2 is a superposition of multiple GRU neural network modules.

[0034] Furthermore, the masked attention layer in step 2 is an operation for performing self-attention calculation between feature vectors.

[0035] Furthermore, the trajectory vector with spatiotemporal attributes in step 3 refers to the target ship trajectory vector generated after the spatial graph learning and time graph fusion operation.

[0036] Furthermore, the decoding operation in step 3 utilizes residual control and dimensionality adjustment of the fully connected layer to decode the encoded trajectory vector and generate the future position coordinates of the target ship.

[0037] A trajectory prediction device for ship situation awareness, comprising

[0038] A spatial graph learning encoder that integrates graph neural networks and attention to capture the dynamic interactions between ships;

[0039] A temporal graph fusion module that employs a masked time-aware attention layer to globally learn the temporal dependencies of vessel trajectories;

[0040] The feature decoding module is used to decode the trajectory vector with spatiotemporal attributes, generate the future relative position coordinates of the ship, and achieve high-precision trajectory prediction tasks.

[0041] An electronic device, characterized in that it includes a processor and a memory, the memory storing machine-executable instructions that can be executed by the processor, and the processor executing the machine-executable instructions to implement the method described in the claim.

[0042] A machine-readable storage medium stores machine-executable instructions. When the machine-executable instructions are called and executed by a processor, the machine-executable instructions prompt the processor to implement the method described in the claim.

[0043] In summary, the present invention has the following beneficial effects:

[0044] 1. This paper addresses the problem of dynamic modeling of ship navigation scenarios by designing a spatial graph learning encoder (S-GLE) that uses graph neural networks to capture the dynamic interactions between ships. In spatial graph learning, a knowledge-aware graph attention mechanism is proposed to calculate the impact of neighbor entities and collaborative relationships on the target entity. Finally, an information aggregation operation is used to fuse the semantics and attributes from different neighbors to refine the embedding of the target entity, addressing the criticism of the current research work that simply uses static spatial graphs to simulate ship interactions.

[0045] 2. We also design a Temporal Graph Fusion (TGF) module that employs a masked time-aware attention layer to learn the temporal dependencies of vessel trajectories globally. Specifically, we utilize a GRU-based sequential encoder-decoder to replace the positional encoding in the traditional attention architecture, allowing us to focus on more temporal dependencies locally, improving the model's predictive performance.

[0046] 3. The present invention reconstructs the feature decoding operation and designs a residual control block to decode the ship trajectory vector with spatiotemporal characteristics, effectively solving the noise interference in the track prediction task and providing ships with a more reasonable and accurate safe navigation route. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] FIG1 is a step diagram of the dynamic graph attention method described in the present invention.

[0048] FIG2 is a configuration procedure diagram of the dynamic image attention device according to the present invention.

[0049] FIG3 is a schematic diagram of the GRU encoder-decoder of the dynamic graph attention method provided by the present invention.

[0050] FIG4 is a flow chart of the dynamic graph attention method provided by the present invention.

[0051] Figure 5 is a schematic diagram of the spatial graph learning encoder of the dynamic graph attention method provided by the present invention.

[0052] Figure 6 is a schematic diagram of the residual control block of the dynamic graph attention method provided by the present invention. DETAILED DESCRIPTION

[0053] In order to make the technical means, creative features, objectives and effects achieved by the present invention easier to understand, the present invention is further described below with reference to diagrams and specific embodiments.

[0054] As shown in Figures 1 to 6, the present invention proposes a dynamic spatiotemporal graph attention method for ship trajectory prediction, comprising the following steps:

[0055] Step 1, spatial graph learning, uses graph neural networks to aggregate the neighbor embeddings of the target entity to learn its local structural representation; then a knowledge-aware graph attention mechanism is used to calculate the influence of neighbor entities and collaborative relationships on the target entity; finally, an information aggregation operation is used to fuse semantics and attributes from different neighbors to refine the embedding of the target entity.

[0056] Step 2, time graph fusion, uses a time graph attention layer to learn the global temporal dependencies of the ship trajectory data and provide interpretability for the model; secondly, a GRU-based sequence encoder is used to replace the positional encoding in the traditional Transformer architecture, providing a more reasonable inductive bias for the input sequence while capturing more local temporal dependencies.

[0057] Step 3, feature decoding,

[0058] The trajectory vector with spatiotemporal attributes is decoded to generate the future relative position coordinates of the ship, achieving high-precision ship trajectory prediction.

[0059] The present invention also provides a trajectory prediction device for ship situation awareness, which includes a spatial graph learning encoder 101, a time graph fusion module 102 and a feature decoding module 103.

[0060] Among them, the spatial graph learning encoder 101 is used to integrate graph neural networks and attention to capture the dynamic interactions between ships.

[0061] In the spatial graph learning encoder 101, a graph neural network is used to aggregate the neighbor embeddings of the target entity in the situation awareness graph to learn its local representation (information propagation process), and a knowledge-aware graph attention mechanism is used to calculate the influence of neighbor entities and collaborative relationships on the target entity. Finally, an information aggregation operation is used to fuse the semantics and attributes from different neighbors to refine the embedding of the target entity.

[0062] The situation awareness map G θ =(ε θ , R θ );

[0063] Where ε is the situation awareness graph G θ The vertex set of

[0064] R is the situational awareness map G θ The edge set in is represented as θ is the current moment of the ship, and the situation awareness map G θ is directed or undirected;

[0065] The local representation of the target ship in step 1 refers to the neighboring ship entities sailing around the target ship, which uses the adjacency matrix To model;

[0066] in, It represents the implicit communication of the target ship, Representation node and e jThe strength of the relationship between the two ships is measured by the geographical proximity of the two ships. The knowledge-aware graph attention mechanism refers to the attention operation between the ship entity vectors.

[0067] The information propagation process mentioned above refers to the representation of the ship entity at the l=0 layer. The refinement will be done through a layer-by-layer propagation procedure, i.e. from the neighbor entity e k To the target entity at layer l The knowledge diffusion can be expressed as:

[0068] Among them, a v ∈x v Represents the basic attribute information of the container. f is a feedforward neural network (this embodiment uses a single-layer MLP), which encodes nodes and pairwise relationships separately. Then, by inwardly aggregating its multi-hop neighbors Embedding of the target entity e v High-order representation in the figure. π v，k is the attenuation coefficient of each transmission on the edge, indicating how much information is transmitted from k to v. is the trainable weight matrix applied to each node. is the e of neighbors generated by layer (l-1) k Embedding. The information in the graph will be propagated to the target entity e v , to enrich its representation.

[0069] During the inward propagation of information from a single entity, some adjacent entities and collaborative relationships have different influences on the target entity. More specifically, if two ships with different attributes (such as COG, SOG, size, etc.) are located in close proximity, their maritime situation awareness and trajectories are more likely to influence each other. Therefore, this embodiment designs a knowledge-aware graph attention mechanism to reveal the different meanings of the target entity in collaboration with its adjacent entities and relationships.

[0070] The knowledge-aware graph attention mechanism specifically includes the following:

[0071] We first compute self-attention on the ship entity, a shared attention mechanism, and then inject the graph structure into the mechanism by performing masked attention computation.

[0072] It is reflected by the following formula: v,k =att(We v ;We k )

[0073] It is worth noting that only entity e is counted in the instance k ∈N v , where N v is a neighbor of node v in the graph. The attention mechanism att is a single-layer feedforward neural network consisting of a weight vector Parameterization. To make the coefficients easily comparable across different entities, we use the function softmax on e k All selections are normalized.

[0074] The graph attention operation is defined by the following formula:

[0075] Among them, e v is the target ship entity, e k is the neighbor of the target entity, f(*) is the ReLU nonlinear activation function, and the connection operation is represented by ||.

[0076] In the spatial graph learning encoder 101, the information aggregation operation specifically includes:

[0077] The information aggregation operation aggregates semantics and attributes from different neighbors and co-relations to refine the embedding of the target entity.

[0078] Specifically, given Update e in the propagation layer l v The embedding of is expressed as:

[0079] Among them, σ is the activation function of Sigmoid; Retained e v The historical embedding information of the ship node can be obtained, and it can effectively alleviate the over-smoothing problem of the residual connection. In the iterative process, multiple representations of the ship node can be obtained, namely And e v The semantics and attributes of its high-order (i.e., L-th order) neighbors can be combined to enrich its embedding after repeated propagation of l layers. Since the output of the l-th layer is an information aggregation of the tree structure depth of l roots, the outputs of different layers emphasize the connection information of different orders.

[0080] This embodiment adopts a layer aggregation mechanism to concatenate the representations of each step into a vector, e v The final entity embedding of is defined as:

[0081] in, is the output target entity embedding of the last propagation layer L. This embodiment can not only enhance the original embedding by performing the propagation operation, but also change the propagation strength by adjusting L. Similarly, other entity embeddings / spatial representations in the graph G can also be obtained through iteration. Therefore, the target entity The spatial representation sequence is generated by iterative calculation of the S-GLE layer at different moments.

[0082] The temporal graph fusion module 102 employs a masked time-aware attention layer to learn the temporal dependencies of the vessel trajectories globally. In particular, it utilizes a GRU-based sequential encoder-decoder to replace the positional encoding in the traditional attention structure, allowing for more temporal dependencies to be locally focused.

[0083] In the time graph fusion module 102, the input is the ship entity vector with spatial dependency Ship spatial representation from GNN output through GRU encoder-decoder The encoder is responsible for modeling the observable historical ship trajectory and mining the ship's motion patterns along the time evolution direction. The decoder takes other future features as input and combines them with the ship's historical state passed by the encoder to predict the ship's target value.

[0084] Specifically, this embodiment will Send to the encoder, Figure 3 is a schematic diagram of the ship information flow encoded by the GRU-based neural network. As shown in the figure, the GRU takes sequence data as input and recursively follows the progression of the sequence, enabling it to learn the local temporal relationships of the target ship.

[0085] Furthermore, this embodiment uses the space representation e of the ship node at each time step. *(θ) ,θ=1,2,...,θ pre and the ship's historical state h transmitted from the previous time step in the GRU (θ-1) , which contains the information of the previous node, can get two gate states: reset gate r and update gate z. And use a single target ship entity in the following formula As a demonstration of this example.

[0086] Where σ can be used as a gating signal, that is, σ(z)=1 / (1+e -z ), which converts the data to the 0-1 range, and W is a trainable weight matrix. The update gate z determines how much information from the previous time step to the current time step needs to be transferred. The reset gate r determines how much historical information needs to be forgotten or discarded.

[0087] Furthermore, the information propagation process in GRU is as follows:

[0088] h′ contains the input data for the current time step tanh(z)=(e z -e -z ) / (e z +e -z ). ⊙ represents the Hadamard product, which is the multiplication of corresponding elements in the operation matrix. is the output vector.

[0089] It is worth noting that the local state of the ship learned by the encoder-decoder at the current time step is not only closely related to the future moment, but will also continue to be passed to the upper layer through nonlinear transformation.

[0090] In the time graph fusion module 102, a masked time-aware attention is also involved, which refers to guiding the future position features of the target ship in a parallel manner, specifically including:

[0091] The masked time-aware attention layer transfers the local temporal feature vectors of the target vessel captured at different moments by the bottom GRU encoder-decoder to the global space and stacks them into a sequence. The attention mechanism then enables the model to consider interactions between nodes at different moments in the sequence and automatically learn relationships between features in the sequence that are relevant to the task at hand. These interaction patterns weight the importance of each input feature, enabling the model to learn dependencies between features that are far apart in the input sequence.

[0092] That is, the attention mechanism is based on the key and query The relationship between the logarithm Scaled and fused through the softmax function to become the following attention output:

[0093] Where T is the number of time steps into the attention layer.

[0094] Furthermore, this embodiment introduces a multi-head attention mechanism to allow the model to learn multiple different aspects of the relationship between input features. Assume that K sets of projection matrices are used to project the data onto different queries, keys, and values, thus forming a multi-head attention mechanism consisting of K independent attention modules connected in series.

[0095] Q v =Hv W Q ,K v =H v W K ,V=H v W V

[0096] in, and is the projection matrix to be learned, shared by all nodes. Multi-head attention in different layers enables the model to learn rich and useful representations by considering node information at different moments in the input sequence.

[0097] More importantly, in the ship trajectory prediction task, this embodiment extends it to masked multi-head attention to ensure that each time dimension can only pay attention to the features before it. We use the information of the unmasked nodes in the sequence to predict the masked target features, allowing the model to consider the entire input sequence, not just the previous frame.

[0098] The feature decoding module 103 is used to decode the trajectory vector with time and space attributes to generate the future relative position coordinates of the ship and achieve high-precision trajectory prediction tasks.

[0099] Furthermore, in the feature decoding module 103, a residual control block is designed to decode the ship trajectory vector with spatiotemporal features, which effectively solves the noise interference in the track prediction task and provides a more reasonable and accurate safe navigation route for the ship.

[0100] Furthermore, the feature decoding is divided into feature decoding during training and feature decoding during testing, specifically including:

[0101] During the training process, the spatial and temporal information of the target ship is first integrated to obtain the global observation features. Secondly, the feed-forward layer (FFL) Combining observed features and future features To generate the mean and covariance Among them, future features It is obtained by encoding the ground truth with S-GLE and GRU;

[0102] During the testing phase, the future ground truth of the target ship is set to zero and only the observed features Used by FFL generate and

[0103] Use Kullback-Leibler Distribution KL distribution to optimize and The decoded vector of the ship undergoes multiple layers of linear transformation operations to obtain the relative position coordinates.

[0104] The above description is reflected by the following formula:

[0105] in, is the relative position of the vessel node v in the future time calculated by the decoder, and ELU is the exponential linear unit activation function. Finally, the historical trajectory sequence is updated for the following time steps to include θ obs At the same time, this embodiment can easily convert the relative position into the absolute position through a simple inverse transformation operation.

[0106] The present invention also provides an electronic device, wherein the device includes a memory and a processor, the memory stores a configuration program system that can run the device of the present invention on the processor, and when the configuration program is executed by the processor, the track prediction method for ship situation awareness of the present invention can be implemented.

[0107] The present invention also provides a computer-readable storage medium, wherein the computer-readable storage medium stores a configuration program of the device described in the present invention, and the configuration program can be executed by one or more processors to implement the track prediction method for ship situation awareness as described in the present invention.

[0108] In this document, the directions or positional relationships indicated by terms such as "up", "down", "front", "back", "left", "right", "top", "bottom", "inside", "outside", "vertical", and "horizontal" are based on the directions or positional relationships shown in the accompanying drawings and are only for the clarity of the technical solution and the convenience of description, and therefore should not be understood as limiting the present invention.

[0109] As used herein, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion of elements other than the listed elements and may also include additional elements not specifically listed.

[0110] The basic principles, main features, and advantages of the present invention are shown and described above. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The above embodiments and descriptions are merely illustrative of the principles of the present invention. Various changes and modifications may be made to the present invention without departing from the spirit and scope of the present invention. Such changes and modifications are intended to fall within the scope of the present invention. The scope of protection claimed in the present invention is defined by the appended claims and their equivalents.

Claims

1. A dynamic spatiotemporal graph attention method for ship trajectory prediction, characterized in that: The steps include: Step 1: spatial graph learning, using the information propagation technology in the graph neural network to learn the local representation of the target ship in the situation awareness graph and obtain the target entity encoding; Step 2, temporal graph fusion, uses a masked attention layer to learn long-term temporal dependencies from spatially dependent track vectors and a GRU-based sequence encoder-decoder to capture local temporal dependencies; Step 3, feature decoding, decodes the trajectory vector with spatiotemporal attributes to generate the future relative position coordinates of the ship and achieve high-precision trajectory prediction tasks.

2. The dynamic spatiotemporal graph attention method for ship trajectory prediction according to claim 1, characterized in that: In step 1, the situation awareness graph G is assumed to be θ =(ε θ , R θ ); Where ε is the situation awareness graph G θ The vertex set of R is the situation awareness map G θ The edge set in is represented as θ is the current moment of the ship, and the situation awareness graph G θ is directed or undirected; The local representation of the target ship in step 1 refers to the neighboring ship entities sailing around the target ship, which adopts the adjacency matrix To model; in, It represents the implicit communication of the target ship, Representation Node and e j The strength of the relationship between the two ships is measured by the geographical proximity of the two ships.

3. The dynamic spatiotemporal graph attention method for ship trajectory prediction according to claim 1, characterized in that: The spatial graph learning in step 1 utilizes the knowledge-aware graph attention mechanism to calculate the influence of neighbor entities and collaborative relationships on the target entity, and then fuses the semantics and attributes from different neighbors through information aggregation operations to refine the embedding of the target entity.

4. The dynamic spatiotemporal graph attention method for ship trajectory prediction according to claim 1, characterized in that: The GRU-based sequence encoder-decoder in step 2 is a superposition of multiple GRU neural network modules.

5. The dynamic spatiotemporal graph attention method for ship trajectory prediction according to claim 1, characterized in that: The masked attention layer in step 2 is an operation used to perform self-attention calculation between feature vectors.

6. The dynamic spatiotemporal graph attention method for ship trajectory prediction according to claim 1, characterized in that: The trajectory vector with spatiotemporal attributes in step 3 refers to the target ship trajectory vector generated after the spatial graph learning and the time graph fusion operation.

7. The dynamic spatiotemporal graph attention method for ship trajectory prediction according to claim 1, characterized in that: The decoding operation in step 3 utilizes residual control and dimension adjustment of the fully connected layer to decode the encoded trajectory vector and generate the future position coordinates of the target ship.

8. A trajectory prediction device for ship situation awareness, characterized in that: include A spatial graph learning encoder (101) that integrates graph neural networks and attention to capture the dynamic interactions between ships; A temporal graph fusion module (102) which employs a masked time-aware attention layer to learn the temporal dependencies of vessel trajectories globally; The feature decoding module (103) is used to decode the trajectory vector with time and space attributes to generate the future relative position coordinates of the ship and realize the high-precision trajectory prediction task.

9. An electronic device, characterized in that: The method comprises a processor and a memory, wherein the memory stores machine executable instructions that can be executed by the processor, and the processor executes the machine executable instructions to implement the method according to any one of claims 1 to 7.

10. A machine-readable storage medium, characterized in that: The machine-readable storage medium stores machine-executable instructions. When the machine-executable instructions are called and executed by a processor, the machine-executable instructions prompt the processor to implement the method described in any one of claims 1 to 7.

Images