Maritime multi-maneuvering unit trajectory prediction method based on GNN-Transform
The interaction and spatiotemporal characteristics of multi-motor units at sea were extracted through the GNN-Transformer model to generate future trajectory prediction results, solving the problem of poor prediction effect of traditional methods in multi-motor unit interaction and confrontation scenarios, significantly improving prediction accuracy and sea The working efficiency of the police system.
Patent Information
- Application Number
- CN202411022915.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-29
- Publication Date
- 2024-12-06
AI Technical Summary
When traditional coastal security systems predict the trajectory of multiple maritime units, especially when interaction and confrontation between multiple units, the prediction effect is poor, making it difficult to effectively evaluate future motion trajectories, affecting the command and decision-making of the security system.
Using the GNN-Transformer-based method, the interaction and spatiotemporal characteristics of mobile units are extracted by constructing time-domain graph structure data, and combining the mask multi-head self-attention mechanism and the decoder of the multi-layer perceptron to generate future trajectory prediction results to improve Prediction accuracy.
It significantly improves the accuracy of the prediction of the trajectory of multiple maritime units, can effectively consider the interaction and confrontation factors between multiple units, and improves the working efficiency of the coast guard system.
Smart Images

Figure CN119090046A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of application of artificial intelligence coastal safety system, and in particular to a trajectory prediction method for multiple maneuverable units at sea based on GNN-Transformer. Background Art
[0002] In recent years, with the rapid development of artificial intelligence technology based on neural networks (NN), it aims to achieve feature extraction and intelligent processing of various information by simulating the learning process of the human brain. Trajectory prediction is an important research direction of time series analysis. By analyzing and reasoning the past motion trajectory and other related information of the target object, its possible future motion trajectory is predicted. Especially in coastal security tasks, trajectory prediction of unknown maneuvering targets can effectively evaluate their next motion trajectory, which will play an important role in the command decision-making of the security system. Traditional security trajectory prediction tasks are often driven by physical models, that is, the prediction of trajectories is achieved by constructing a priori formulas based on physical laws. This method has a certain effect on short-term motion without interference, but it is not effective in emergencies such as interception and sharp turns. In recent years, neural networks have also been used to predict the trajectory of maneuvering targets, which has alleviated the problem of poor long-term prediction effect to a certain extent. However, due to its lack of consideration of the interaction and confrontation factors between multiple units, when our coast guard command system dispatches an interception ship, when multiple opposing units confront our units, the current artificial intelligence model cannot make effective predictions. Summary of the invention
[0003] The purpose of the present invention is to solve the problems raised in the background technology and provide a method for predicting the trajectory of multiple mobile units at sea based on GNN-Transformer, which extracts the interactive features and spatiotemporal features of the confrontation between multiple mobile units in the past, realizes feature coupling and generates future trajectory predictions, which can greatly improve the prediction accuracy and promote the working efficiency of the maritime police system.
[0004] In order to achieve the purpose of the present invention, the present invention discloses a trajectory prediction method for multiple maneuvering units at sea based on GNN-Transformer, which specifically includes the following steps:
[0005] S1. Obtain the trajectory sequence data of the mobile units of our unmanned boats and the invading unknown units at sea through the maritime reconnaissance system and our maritime police headquarters;
[0006] S2, using the mobile unit as the graph node and the corresponding trajectory sequence data as the information in the node, constructing the time domain graph structure data as the input of the GNN module;
[0007] S3, extract unit interaction adversarial features in the spatial dimension through the GNN module;
[0008] S4, extracting trajectory motion features in the spatiotemporal dimension through the spatiotemporal Transformer module;
[0009] S5, outputs the unit's trajectory prediction results through a decoder that includes a masked multi-head self-attention mechanism and a multi-layer perceptron;
[0010] S6. Iteratively train and optimize the hyperparameters of the entire network to obtain the final network model and output the best prediction result.
[0011] Furthermore, in step S1, the maritime reconnaissance system collects information on the invading unit based on radar and visual technology to obtain specific information of the maritime mobile unit.
[0012] Furthermore, in step S2, the graph structure data is constructed using the trajectory sequences of all mobile units of both sides as nodes of the graph, and the node information includes position, speed, and camp label.
[0013] Furthermore, in step S3, the input trajectory sequence needs to first map the kinematic trajectory features to a high-level space through an MLP, and then input into a GNN module constructed by multiple graph neural layers for interactive feature extraction.
[0014] Furthermore, in step S4, the input features are simultaneously input into the time domain encoder and the spatial domain encoder to extract features, and finally connected and fused.
[0015] Furthermore, in step S4, the time domain encoder and the spatial domain encoder are two independent Transformer encoder modules, which respectively extract information in two specific domains based on the self-attention mechanism.
[0016] Furthermore, in step S5, the mask mechanism used is composed of a lower triangular matrix to ensure that when synchronously decoding future information of multiple time steps, the decoder only pays attention to the trajectory information before the current step to prevent information leakage and achieve synchronous generation of future trajectories.
[0017] Furthermore, in step S6, back propagation is performed using the Adam optimizer; during the training process, the optimal model selection indicator lies in the effect of the test set, and the optimal model parameters are selected based on the performance of the test set.
[0018] Furthermore, in step S6, the learning rate decreases exponentially during the learning process. As the model training progresses, the learning rate will gradually decrease from 0.01 to 0.001, taking into account both the early learning efficiency and the later learning accuracy.
[0019] Compared with the prior art, the significant improvements of the present invention are: 1) adding the consideration of the game confrontation and interaction between the two sides to predict the trajectory of the unit, thereby improving the prediction effect for dynamic maneuverable units; 2) integrating the confrontation interaction characteristics and spatiotemporal characteristics, which can greatly improve the model's accuracy in predicting the trajectory of maneuverable units.
[0020] In order to more clearly illustrate the functional characteristics and structural parameters of the present invention, further description is given below in conjunction with the accompanying drawings and specific implementation methods. BRIEF DESCRIPTION OF THE DRAWINGS
[0021] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:
[0022] Figure 1 This is the overall flow chart of the GNN-Transformer-based method for predicting the trajectory of multiple maneuverable units at sea.
[0023] Figure 2 This is a schematic diagram of the specific implementation process of the trajectory prediction model algorithm for multi-mobile units at sea based on GNN-Transformer;
[0024] Figure 3 This is a schematic diagram of the specific structure of the GNN-Transformer network model. DETAILED DESCRIPTION
[0025] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments; based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.
[0026] The present invention combines GNN and spatiotemporal Transformer models to realize the trajectory prediction of multiple maneuverable units in confrontation at sea. By analyzing the past information of both sides, the future trajectory information of the maneuverable units is predicted. Compared with the traditional method, the information extraction and information coupling of interactive features and spatiotemporal features are increased, and the generative model is used to achieve the future trajectory prediction effect, which can greatly improve the prediction accuracy of confrontation scenarios, promote the implementation of artificial intelligence technology in coastal security, and improve the working efficiency of the maritime police system. The specific implementation methods are as follows.
[0027] For the trajectory information of the mobile units at sea collected by the maritime reconnaissance system and the trajectory information of our mobile units extracted by our command system, we first construct the graph structure data with the trajectory time series of the mobile units as the nodes of the graph. The spatial graph data constructed by the n mobile units of both sides can be expressed as G n = {V n , E n , A n},in Represents the node information of n mobile units, including position, speed, and both camps’ labels. The edge set of the spatial graph is represented as side Represents whether there is interaction between one node and another node at time t. A represents the adjacency matrix of the graph. Each value in the matrix represents the strength of the interaction between nodes, that is, the weight of the directed edge. In this way, all trajectory data are constructed into a graph structure data.
[0028] After obtaining the graph structure data, it is input into the GNN module to extract interactive features. Since the directly input node trajectory information is a relatively low-level feature, it is not conducive to information aggregation and feature extraction in GNN. Therefore, the low-level trajectory information is first mapped to the high-level feature space through a multi-layer perceptron (MLP). The formula for extracting features in a single layer of a multi-layer perceptron is as follows:
[0029] F(x)=g(W 2 (W 1 X+b 1 )+b 2 )
[0030] Where W 1 , b 1 are the weights and biases of the hidden layer, W 2 , b 2 is the weight and bias of the activation layer. The multi-layer perceptron consists of multiple hidden layers, and can effectively obtain information in the high-level feature space through multi-layer information transmission and feature extraction.
[0031] After obtaining the high-level trajectory features, the features are input into the GNN module for interactive feature extraction. The GNN module has multiple graph neural layers for feature aggregation, so that the network can extract the interactive information between nodes from the aggregated multiple nodes. The operator in a single-layer graph neural network layer is shown in the following formula:
[0032]
[0033] Among them, i is the selected central node, Represents the feature expression of node i at layer L. Ni Represents the adjacent nodes of node i, including node i itself. Represents the feature weight parameter of the jth adjacent node in this layer. There are multiple operators designed as above in GNN, which can realize the extraction of interaction features between nodes.
[0034] The feature sequence obtained by the GNN module is mainly aimed at capturing the motion association between nodes due to interaction factors, but it is still insufficient for time series analysis, especially for the long-range dependency of the input sequence, which will disappear due to the transmission of hidden layer information. Therefore, a spatiotemporal Transformer is constructed to use the self-attention mechanism to capture the spatiotemporal feature association between each position in the input feature sequence.
[0035] In the proposed spatiotemporal Transformer, the input features are composite features that connect interaction features and trajectory features. The parallel encoder projects the input into two different high-level feature hidden spaces, fuses the acquired features and maps them into future trajectories using a generative encoder. The interaction feature sequence that will be used as input will be acquired by the GNN module in the previous part. The spatiotemporal trajectory feature sequence is input into the temporal encoder and the spatial encoder in batches of time and space, respectively, so as to perform feature modeling in parallel in the temporal and spatial dimensions. We design an edge weight masked self-attention mechanism for the interaction between nodes in the spatial encoder to embed edge information when information is passed between nodes. Each encoder has a corresponding self-attention module and a stack of convolutional layers for feature aggregation. The temporal feature sequence and the spatial feature sequence output by the two encoders are connected with the trajectory feature sequence, and feature aggregation and extraction are performed through a feature fusion module constructed by a multi-layer perceptron, so as to obtain the spatiotemporal trajectory feature sequence.
[0036] The spatiotemporal trajectory feature sequence and the interactive feature sequence are input into the decoder module, and the high-level features can be mapped to future trajectories. The masked self-attention decoder module is composed of a masked multi-head self-attention mechanism and a fully connected forward propagation layer stacked together. In the generative decoder, since the input of the decoder is a complete and continuous trajectory sequence, a trajectory is also predicted in parallel during decoding. However, in order to prevent information leakage and autoregression problems, a mask matrix is used to ensure that the information at each position only focuses on the part of the subsequent position, thus constructing a masked multi-head self-attention module. The masked multi-head self-attention mechanism is shown in the following formula:
[0037]
[0038] Where Q, K, and V represent the query matrix, key matrix, and value matrix of the input feature sequence mapped through three different feature spaces. M represents the mask matrix, which is essentially a lower triangular matrix. enc_out Mapped to the prediction trajectory space through a fully connected feedforward layer. Constructing this generative encoder can directly predict the trajectories of all time steps in parallel in a single inference, unlike other time series analysis models that require iterative forward operations step by step.
[0039] Finally, iterative training is performed according to the general deep learning process to back-propagate and update the model parameters until the best model prediction effect is obtained after testing, which is used as the final model for future trajectory prediction.
[0040] Example
[0041] In this example, the model can automatically adjust model parameters during training, without the need for prior model settings and detailed parameter adjustments, to ensure that the model-predicted trajectory fits the actual trajectory. Compared with traditional physical model-based methods, this approach has higher accuracy and avoids the trouble of manually setting hyperparameters a priori.
[0042] In this embodiment, the training and testing data sets will be simulated according to the simulation model, combined with expert knowledge to simulate the trajectory sequence of the confrontation nature of the mobile units at sea of both sides. The training set and the test set are divided into 80% and 20% of the total data set respectively, the trajectory sampling interval is 0.2s as a time step, the input past trajectory length is 20 time steps, and the prediction length is 10 time steps.
[0043] In this embodiment, the model development is based on python3.8 and the supporting pytorch2.0.0 as a deep learning development package. The GNN-Transformer trajectory prediction model based on graph structure two-way adversarial trajectory data is used as the trajectory prediction solution. The specific solution steps are as follows: Figure 1 shown.
[0044] like Figure 1 As shown in FIG. 1 , a method for predicting the trajectory of multiple mobile units at sea based on GNN-Transformer includes the following steps S1 to S6. The specific details of each step will be described below. The implementation flow chart of all steps is as follows: Figure 2 shown.
[0045] In this embodiment, in step S1, when collecting information, the coast guard reconnaissance system will detect the location information through radar and other systems to obtain the trajectory, and obtain other information through machine vision and other systems as information collection for unknown units at sea.
[0046] In this embodiment, in step S2, each node will contain a past trajectory sequence of a set data length (20 time steps in this example) and a camp number (1 for our side and 0 for the other side).
[0047] Steps S3 to S5 involve the network model of the system, which needs to be explained in conjunction with the specific model design. The designed GNN-Transformer network model is as follows: Figure 3 shown.
[0048] In this embodiment, in step S3, the GNN module inputs a graph structure trajectory information sequence, where each node in the graph represents the past trajectory information of a mobile unit. This step will achieve Figure 3 Enter the graph structure data section.
[0049] In this embodiment, in step S4, the input of the spatiotemporal transformer is a composite feature sequence connected with the interaction features and trajectory features obtained in S3, and the feature sequence is input into the temporal encoder and the spatial encoder respectively. Figure 3 The temporal encoder and spatial encoder shown in the figure can respectively use the self-attention mechanism to perform correlation modeling and feature extraction on the temporal features and spatial features, and perform feature fusion through cascading and fusion modules.
[0050] In this embodiment, in step S4, since the input feature dimension is A*T*F (unit*time step*feature), it is only suitable to extract the time domain features through the self-attention mechanism in the time domain encoder. Before inputting to the spatial domain encoder, it needs to be transposed into the form of T*A*F (time step*unit*feature), and the spatial domain features can be obtained through the self-attention mechanism in the spatial domain encoder. The specific implementation is as follows Figure 3 As shown in the figure, the features extracted by GNN interactive features are reconstructed into time domain and spatial domain features, which are input into the subsequent time domain encoder and spatial domain encoder respectively.
[0051] In this embodiment, in step S5, the mask mechanism is composed of a lower triangular matrix to ensure that when synchronously decoding future information of multiple time steps, the decoder only pays attention to the trajectory information before the current step to prevent information leakage and achieve synchronous generation of future trajectories. Figure 3 As shown in Figure 1, the mask mechanism will initially decode the motion features of the graph structure so that the motion features will not leak information during subsequent decoding. The encoded motion features and spatiotemporal features are fused and generatively decoded through the self-attention mechanism, and finally mapped to future trajectories through the fully connected layer.
[0052] In this embodiment, in step S6, the commonly used Adam optimizer is used for parameter back propagation, the initial learning rate is 0.01, the final learning rate is 0.001, a total of 200 rounds of training are performed, and the model parameters with the best test set effect in the 200 rounds are stored.
[0053] Based on the above steps, all the processes for model learning and training are constructed.
[0054] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.
[0055] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.
Claims
1. A GNN-Transformer-based trajectory prediction method for multiple maneuvering units at sea. It is characterized in that The specific steps include: S1. Obtain the trajectory sequence data of the mobile units of our unmanned boats and the invading unknown units at sea through the maritime reconnaissance system and our maritime police headquarters; S2, using the mobile unit as the graph node and the corresponding trajectory sequence data as the information in the node, constructing the time domain graph structure data as the input of the GNN module; S3, extract unit interaction adversarial features in the spatial dimension through the GNN module; S4, extracting trajectory motion features in the spatiotemporal dimension through the spatiotemporal Transformer module; S5, outputs the unit's trajectory prediction results through a decoder that includes a masked multi-head self-attention mechanism and a multi-layer perceptron; S6. Iteratively train and optimize the hyperparameters of the entire network to obtain the final network model and output the best prediction result.
2. According to claim 1, a GNN-Transformer-based trajectory prediction method for multiple mobile units at sea, It is characterized in that In step S1, the maritime reconnaissance system collects information on the invading unit based on radar and visual technology to obtain specific information of the maritime mobile unit.
3. According to claim 1, a GNN-Transformer-based trajectory prediction method for multiple mobile units at sea, It is characterized in that In step S2, the trajectory sequences of all mobile units of both sides are used as nodes of the graph to construct graph structure data, and the node information includes position, speed, and camp label.
4. According to claim 1, a GNN-Transformer-based trajectory prediction method for multiple mobile units at sea, It is characterized in that In step S3, the input trajectory sequence needs to first map the kinematic trajectory features to the high-level space through MLP, and then input into the GNN module constructed by multiple graph neural layers for interactive feature extraction.
5. According to claim 1, a GNN-Transformer-based trajectory prediction method for multiple mobile units at sea, It is characterized in that In step S4, the input features are simultaneously input into the time domain encoder and the spatial domain encoder to extract features, and finally connected and fused.
6. A method for predicting trajectories of multiple mobile units at sea based on GNN-Transformer according to claim 5, It is characterized in that In step S4, the time domain encoder and the spatial domain encoder are two independent Transformer encoder modules, which respectively extract information in two specific fields based on the self-attention mechanism.
7. According to claim 1, a GNN-Transformer-based trajectory prediction method for multiple mobile units at sea, It is characterized in that In step S5, the mask mechanism used is composed of a lower triangular matrix to ensure that when synchronously decoding future information of multiple time steps, the decoder only pays attention to the trajectory information before the current step to prevent information leakage and achieve synchronous generation of future trajectories.
8. According to claim 1, a GNN-Transformer-based trajectory prediction method for multiple mobile units at sea, It is characterized in that In step S6, back propagation is performed using the Adam optimizer. During the training process, the optimal model selection indicator lies in the effect of the test set, and the optimal model parameters are selected based on the performance of the test set.
9. A method for predicting trajectories of multiple mobile units at sea based on GNN-Transformer according to claim 8, It is characterized in that In step S6, the learning rate decreases exponentially during the learning process. As the model training progresses, the learning rate will gradually decrease from 0.01 to 0.001, taking into account both the early learning efficiency and the later learning accuracy.
Citation Information
Cited By
Method for predicting residual service life of rotating machine based on GBIT-GAN
CN119962111A
A rotating machinery residual service life prediction method based on GBIT-GAN
CN119962111B
Maritime traffic network mining method
CN121255792A
A maritime traffic network mining method
CN121255792B
Multi-data-source driving safety evaluation method and system
CN121350947A
