Ship trajectory prediction method and system based on multi-modal multi-scale time attention

Abstract

The application discloses a ship track prediction method and system based on multi-modal multi-scale time sequence attention. The method obtains ship AIS historical track data, and after abnormal filtering, track segmentation, resampling, coordinate conversion, kinematic feature calculation and standardization, a historical state sequence and a future relative displacement label are constructed; a prediction model including a shared GRU encoder, an adaptive multi-scale time sequence attention module, a multi-modal decoder and a mode probability classification head is established, multi-scale track dependencies are captured through short-term, medium-term and global attention, and multiple future candidate tracks and corresponding probabilities are output. During training, Winner-Takes-All joint loss is adopted, and classification and diversity constraints are combined. The system can be used for ship traffic management, collision risk early warning, route planning and maritime situation awareness.

Description

Technical Field

[0001] This invention relates to the fields of intelligent shipping, maritime traffic monitoring, spatiotemporal sequence prediction, and deep learning technologies, specifically to a method and system for predicting ship trajectories based on multimodal, multi-scale temporal attention. This invention is particularly applicable to short- and medium-term trajectory prediction of ships in port waters, near-shore channels, and open sea areas based on AIS data. Background Technology

[0002] Automatic Identification System (AIS) continuously broadcasts information such as a vessel's position, speed, course, and type, serving as a crucial data source for maritime supervision, port scheduling, collision risk assessment, and autonomous navigation decision-making. Accurately predicting a vessel's trajectory for the next few minutes helps identify potential risks in advance, optimize route planning, and improve traffic organization efficiency.

[0003] Existing methods for ship trajectory prediction mainly include uniform extrapolation, Kalman filtering, recurrent neural networks, Transformer and its variants. Uniform extrapolation and Kalman filtering rely on simple motion assumptions and are difficult to handle complex maneuvers such as turning, deceleration, and collision avoidance. Although traditional LSTM or GRU can learn time dependencies, they usually only output a single deterministic trajectory, which is difficult to cover the multiple maneuvering modes that ships may encounter in the future. General Transformer-type time series prediction methods also face the problems of high coupling of variables such as position, speed, and heading in AIS scenarios, as well as the coexistence of short-term local maneuvers and long-term route priors.

[0004] Ship motion exhibits significant multi-scale and multi-modal characteristics: changes in speed and heading within the last few tens of seconds determine short-term maneuvering trends, while track patterns on a timescale of several minutes reflect channel constraints and route intentions. Furthermore, factors such as ports, channels, and collision avoidance allow for multiple plausible future trajectories. Therefore, this field requires a ship track prediction method that can adaptively integrate short-term, medium-term, and global time dependencies, and output multiple candidate tracks and their corresponding probabilities. Summary of the Invention

[0005] The purpose of this invention is to overcome the problems of insufficient single-time-scale modeling, difficulty in covering multiple future maneuvers with deterministic prediction, and insufficient characterization of multivariate coupling relationships in the existing technology, and to provide a ship trajectory prediction method and system based on multimodal and multi-scale temporal attention.

[0006] The technical solution adopted in this invention is as follows:

[0007] A ship trajectory prediction method based on multimodal and multi-scale temporal attention includes the following steps:

[0008] S1: Obtain the historical AIS track data of the target vessel; the AIS track data includes timestamp, latitude and longitude, speed over land, heading over land, vessel identifier, and vessel type information;

[0009] S2: Preprocess historical AIS track data to obtain a standardized multidimensional ship motion state sequence;

[0010] S3: Construct supervised learning samples based on the preprocessed multidimensional ship state sequence;

[0011] S4: Construct a multimodal, multi-scale temporal attention GRU trajectory prediction model, including a shared GRU encoder, an adaptive multi-scale temporal attention module, a multimodal decoder, and a pattern probability classification head;

[0012] The shared GRU encoder is used to receive continuous... The multidimensional ship state sequence at each time point outputs include... Hidden state sequence at each time step ;in, Constrained by the model structure, This represents the final hidden state corresponding to the last historical moment.

[0013] The adaptive multi-scale temporal attention module is used to divide the hidden state sequence output by the shared GRU encoder into three scales: short-term window, medium-term window, and global window. For each scale, the scale context is calculated through an independent scaling dot product attention mechanism, where the current hidden state of the multimodal decoder is used as the query vector, and the corresponding historical hidden state subsequence of the shared GRU encoder is used as the key vector and value vector. Then, a learnable gating network adaptively outputs the fusion weights of the contexts of each scale according to the current hidden state of the multimodal decoder, and the weighted sum is used to obtain the fused temporal context. The historical hidden state subsequence of the shared GRU encoder is composed of the hidden state sequences that fall into the corresponding time window from the hidden state sequence output by the shared GRU encoder.

[0014] The multimodal decoder contains K learnable mode embedding vectors, used to embed the final hidden state from the shared GRU encoder output. Each path is fused with the embedding vector of each mode to obtain the independent initial decoding state of K candidate paths. At each prediction time step in the future prediction window, the future path is generated stepwise through K parallel GRUCells, conditioned on the fused temporal context. A two-dimensional displacement sequence at time points; where K is a set value. Constrained by the model structure;

[0015] The pattern probability classification head is used to classify the final hidden state based on the output of the shared GRU encoder. The normalized probability distribution of K candidate tracks is obtained, which characterizes the probability of each motion mode;

[0016] S5: Train a multimodal, multi-scale temporal attention GRU trajectory prediction model using supervised learning samples, to continuously... Using a multidimensional ship state sequence at a given moment as input, and future... The two-dimensional displacement of each time point relative to the position of the last historical time point is used as the prediction label; during training, the parameters are optimized using the Winner-Takes-All joint loss based on the optimal candidate trajectory to obtain the trained trajectory prediction model; the Winner-Takes-All joint loss includes position regression loss, velocity continuity loss, heading smoothing loss, pattern classification loss and diversity regularization loss;

[0017] S6: The nearest ship to be predicted, whether in real-time or offline. The multidimensional ship motion state sequence at each moment is input into the trained trajectory prediction model, and the model outputs K future candidate trajectories and their corresponding probabilities.

[0018] S7: Generate the prediction results required for ship future motion risk assessment, route planning, or traffic situation display based on K future candidate tracks and their corresponding probabilities.

[0019] Furthermore, in S4, the short-term window corresponds to the hidden state of the most recent first proportional step, used to capture recent maneuver trends; the medium-term window corresponds to the hidden state of the most recent second proportional step, used to capture stable speed and heading changes; and the global window corresponds to all... The hidden state is used to capture the complete historical track pattern; where the first ratio is smaller than the second ratio.

[0020] Furthermore, for each scale The adaptive multi-scale temporal attention module adjusts the hidden state based on the current hidden state of the multimodal decoder. Generate query vectors and with a shared GRU encoder at scale Corresponding hidden state Generate key vectors Sum value vector Calculate attention weights And obtain the scale context. Subsequently, through the gating network Obtain the fusion weights at each scale and calculate the fusion context. ;in, For any time scale in a short-term window, medium-term window, or global window, The dimension of the key vector. For scale The attention weights corresponding to each historical moment are as follows: For scale The corresponding context vector, The fusion weight vector for each scale context, For scale The corresponding fusion weights, For the learnable weight matrix of the gated network, To fuse temporal context vectors, For normalized exponential functions, Both ∑ and simplify represent summation operations.

[0021] Furthermore, the multimodal decoder in S4 is for the first... The k-th candidate pattern is obtained by fusing the final hidden state of the shared GRU encoder with the embedding vector of the k-th pattern. The initial decoding state of each candidate track; within the future prediction window, the multimodal decoder takes the fused temporal context and the current decoding state as input, updates the decoding state step by step through the corresponding GRUCell, and maps the decoding state to two-dimensional displacement through the displacement prediction layer, thereby generating the future... Two-dimensional displacement at time t; where =1,2,...K; The pattern probability classification head maps the final hidden state of the shared GRU encoder and normalizes it using softmax to obtain the probability distribution corresponding to the K candidate tracks.

[0022] Furthermore, the Winner-Takes-All loss based on the optimal candidate trajectory in S5 is as follows: For each training sample, calculate the average displacement error between K candidate tracks and the real track, select the candidate track with the smallest average displacement error as the optimal mode, calculate the position regression loss, velocity continuity loss, and heading smoothing loss only for the candidate track corresponding to the optimal mode, and use the index of the optimal mode as the mode classification label, set the mode classification loss as the cross-entropy loss to constrain the probability distribution output by the mode probability classification head; and add the diversity regularization loss to the joint loss for parameter optimization.

[0023] Furthermore, the diversity regularization loss in S5 is calculated based on the pairwise distance between different candidate track endpoints. A penalty is applied when the distance between any two candidate track endpoints is less than an adaptive threshold, which is determined by the displacement magnitude of the candidate track endpoints, in order to suppress mode collapse in multimodal prediction.

[0024] A ship trajectory prediction system based on multimodal and multi-scale temporal attention includes:

[0025] The data acquisition module acquires the historical AIS track data of the target vessel; the AIS track data includes timestamp, latitude and longitude, speed over land, heading over land, vessel identifier, and vessel type information;

[0026] The data preprocessing module is used to perform abnormal speed filtering, track segmentation by time interval, equal-interval resampling, latitude and longitude conversion to local coordinates, kinematic feature calculation, heading encoding and standardization on historical AIS track data to obtain a standardized multidimensional ship motion state sequence.

[0027] The sample construction module is used to construct supervised learning samples based on the preprocessed multidimensional ship state sequence.

[0028] The model building module is used to build a multimodal, multi-scale temporal attention GRU trajectory prediction model, including a shared GRU encoder, an adaptive multi-scale temporal attention module, a multimodal decoder, and a pattern probability classification head;

[0029] The model training module is used to train a multimodal, multi-scale temporal attention GRU trajectory prediction model using supervised learning samples, and obtain a trained trajectory prediction model.

[0030] The trajectory prediction module is used to input the multi-dimensional ship motion state sequence of the ship to be predicted in real time or offline into the trained trajectory prediction model, and output the future candidate trajectories and their corresponding probabilities.

[0031] The results application module is used to generate the prediction results required for ship future motion risk assessment, route planning, or traffic situation display based on K future candidate tracks and their corresponding probabilities.

[0032] Compared with the prior art, the present invention has the following significant advantages:

[0033] High prediction accuracy: The multi-scale temporal attention of this invention can simultaneously capture recent maneuver trends, medium-term motion states and complete historical trajectory patterns, and improve the accuracy of short- and medium-term trajectory prediction by combining with the GRU encoding and decoding structure.

[0034] Strong multimodal representation capability: Through pattern embedding and multi-candidate decoder, the model can simultaneously provide multiple reasonable future trajectories such as straight flight, turning, and deceleration, and represent the credibility of each future by probability.

[0035] Low risk of pattern collapse: This invention combines Winner-Takes-All training with diversity regularization to maintain sufficient differentiation among different candidate trajectories, thereby improving the coverage of uncertain maneuvers in complex port and waterway environments.

[0036] Good generalization ability: The model of this invention takes general AIS kinematic features such as position, velocity, heading, acceleration and time interval as input, and can be trained and deployed in different geographical areas and traffic density scenarios.

[0037] Highly practical for engineering applications: The model of this invention takes input from widely available AIS data and outputs multiple future trajectories and probabilities, which can directly serve VTS traffic management, ship collision avoidance warning, route planning and maritime situation visualization. Attached Figure Description

[0038] Figure 1 The flowchart illustrates the ship trajectory prediction method based on multimodal and multi-scale temporal attention provided in this embodiment of the invention.

[0039] Figure 2 This is a schematic diagram of AIS data preprocessing and sliding window sample construction provided in an embodiment of the present invention.

[0040] Figure 3 This is a schematic diagram of the structure of the multimodal, multi-scale temporal attention GRU trajectory prediction model constructed in an embodiment of the present invention.

[0041] Figure 4 This is a schematic diagram of the Winner-Takes-All training and multi-candidate track output process in an embodiment of the present invention. Detailed Implementation

[0042] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described below with reference to the accompanying drawings and specific embodiments. It should be understood that the following embodiments are only for explaining the invention and are not intended to limit the scope of protection of the invention.

[0043] Example 1:

[0044] See Figure 1 This embodiment provides a ship trajectory prediction method based on multimodal and multi-scale temporal attention, specifically including the following steps:

[0045] S1: Data Acquisition. Acquire historical AIS track data of the target vessel. The AIS track data includes timestamps, latitude and longitude, speed over land, heading over land, vessel identification, and vessel type information. This embodiment can use AIS data from the Danish Maritime Authority, US NAIS data, or AIS data collected by the port VTS system as the data source, and can aggregate continuous observations of the same target vessel according to the vessel identification.

[0046] S2: Data preprocessing. See also Figure 2Following the process of "original AIS sequence, abnormal speed filtering, track segmentation by time interval, 15-second resampling at equal intervals, latitude and longitude to local coordinate conversion, kinematic feature calculation, heading encoding, and standardization," historical AIS track data is processed to obtain a standardized multidimensional ship motion state sequence. Specifically, firstly, abnormal speed filtering is performed on the original AIS sequence to remove records with ground speeds exceeding a preset threshold, and records with missing latitude and longitude, abnormal timestamps, or unreasonable heading values ​​can also be removed. Then, the track is segmented according to the time interval between adjacent AIS reports. When the time interval exceeds a preset threshold, the original track is segmented into different track segments. Subsequently, 15-second resampling is performed, retaining track segments with lengths that meet the requirements of subsequent historical and prediction windows. When processing the coordinates, latitude and longitude are converted to local metric coordinates with the reference point of the study area as the origin to obtain local coordinates. and Further calculations Directional velocity components , Directional velocity components , Directional acceleration components , Directional acceleration components Sine and cosine values ​​of the heading relative to the ground, and the rate of change of the heading angle. and the time interval between adjacent sampling points Isokinetic characteristics. After standardization, a multi-dimensional ship state vector is formed at each moment; in Figure 2 In the example shown, the input feature dimension is 11.

[0047] S3: Sample Construction. See also... Figure 2 Supervised learning samples are constructed based on preprocessed multidimensional ship state sequences. A sliding window approach is used to extract samples from each cleaned track segment, ensuring continuous... Using a multidimensional ship state sequence from a historical moment as input, and the future... The two-dimensional displacement of each time point relative to the position of the last historical time point is used as the prediction label. As one specific implementation method, A value of 30 can be selected, corresponding to 7.5 minutes of historical observations at a 15-second sampling interval; A value of 20 can be used, corresponding to the future trajectory 5 minutes in advance; the sliding step size can be set to 25. The above values ​​are merely examples. , The sliding step size can be adjusted according to the prediction duration, sampling frequency, and application scenario.

[0048] S4: Model Building. See also Figure 3A multimodal, multi-scale temporal attention GRU trajectory prediction model is constructed, which includes a shared GRU encoder, an adaptive multi-scale temporal attention module, a multimodal decoder, and a pattern probability classification head.

[0049] Shared GRU encoder for receiving continuous The system outputs a multidimensional ship state sequence at each time step and a hidden state sequence. ,in For the first The hidden state corresponding to each historical moment This represents the final hidden state corresponding to the last historical moment. Hidden state sequence. Used to provide a complete historical representation for subsequent multi-scale attention, the final hidden state It is used to characterize the overall motion state of the history window and can be used for multimodal decoder initialization and pattern probability classification.

[0050] The adaptive multi-scale temporal attention module is used to process the hidden state sequence output by the shared GRU encoder. The time window is divided into three scales: short-term window, medium-term window, and global window. The short-term window corresponds to the hidden state of the most recent first proportion or first order of magnitude, used to capture recent maneuver trends; the medium-term window corresponds to the hidden state of the most recent second proportion or second order of magnitude, used to capture stable speed and heading changes; the global window corresponds to all... Step hidden state, used to capture complete historical track patterns. In Figure 3 In the illustrated embodiment, the most recent 3 steps can be used as a short-term window, the most recent 10 steps as a medium-term window, and all 30 steps as a global window.

[0051] For each scale The adaptive multi-scale temporal attention module computes the scale context through independent scaling dot product attention mechanisms. Specifically, at any prediction step, the current hidden state of the multimodal decoder is used as the basis for the calculation. As a source of query, by scale The corresponding shared GRU encoder history hidden state subsequence As the source of keys and values; the From the hidden state sequence The hidden states that fall within the corresponding time window constitute the query vector. Key vector Sum value vector Calculate attention weights And obtain the scale context. ,in The dimension is the key vector. Subsequently, a learnable gating network is used. The fusion weights of short-term, medium-term, and global scale contexts are obtained, and the fusion temporal context is calculated. Therefore, the model can adaptively select a mode that relies more on short-term maneuvers, medium-term trends, or global trajectory based on the current decoding state.

[0052] The multimodal decoder includes A learnable pattern embedding vector is used to represent Candidate motion patterns. For the first... The candidate patterns will share the final hidden state of the GRU encoder. With the The pattern embedding vectors are added together or fused through mapping to obtain the first pattern embedding vector. The initial decoding state corresponding to each candidate track; within the future prediction window, the multimodal decoder fuses the temporal context. Taking the current decoding state as input, the decoding state is updated progressively through the corresponding GRUCell, and the decoding state is mapped to a two-dimensional displacement by the displacement prediction layer, thereby generating future... A two-dimensional displacement sequence at time points. , Candidate patterns are generated in parallel. A candidate trajectory for the future.

[0053] The pattern probability classification head is based on the final hidden state of the shared GRU encoder. Output The normalized probability distribution of candidate trajectories is used to characterize the likelihood of each motion mode. The normalized probability distribution can be obtained through... The function is obtained, and its components correspond to... Confidence level of candidate trajectories. Figure 3 middle" Candidate future flight paths+ The output of "the probability of each mode" corresponds to the joint output of the multimodal decoder and the mode probability classification head.

[0054] S5: Model Training. See also Figure 4 A multimodal, multi-scale temporal attention GRU trajectory prediction model was trained using supervised learning samples. During training, a Winner-Takes-All joint loss based on the optimal candidate trajectory was employed. Specifically, for each training sample, the average displacement error (ADE) between K candidate trajectories and the actual trajectory was calculated, and the candidate trajectory with the smallest average displacement error was selected as the optimal model. Only for the aforementioned optimal mode The corresponding candidate trajectory is calculated using position regression loss, velocity continuity loss, and heading smoothing loss. Position regression loss is used to constrain the optimal candidate trajectory to be close to the true future position; velocity continuity loss is used to constrain the displacement changes between adjacent predicted points to be consistent with the true trajectory; heading smoothing loss is used to constrain the heading angle changes calculated from adjacent displacements, reducing unreasonable jitter.

[0055] At the same time, in the optimal mode The index is used as the pattern classification label, and cross-entropy loss is used as the pattern classification loss to constrain the probability distribution output by the pattern probability classification head. The diversity regularization loss is calculated based on the pairwise distances between different candidate track endpoints. A penalty is applied when the distance between any two candidate track endpoints is less than an adaptive threshold, which can be determined by the displacement magnitude of the candidate track endpoints to suppress pattern collapse in multimodal prediction. The total training loss can be obtained by weighted summation of position regression loss, velocity continuity loss, heading smoothing loss, pattern classification loss, and diversity regularization loss. The training optimizer can be AdamW, and the initial learning rate, weight decay, batch size, and number of early stopping rounds can be set according to hardware conditions and data scale. During training, the optimal model can be selected based on the sum of the validation set ADE and FDE.

[0056] S6: Track Prediction. In practical applications, the system continuously receives real-time or offline AIS data and maintains the latest track prediction for each vessel to be predicted. A multi-dimensional cache of ship motion states at each time step. When the input length requirement is met, the input tensor is constructed using the same preprocessing and normalization methods as in the training phase, and then input into the trained trajectory prediction model to obtain... A sequence of future candidate relative displacements and their corresponding probabilities. Based on the probability distribution output by the pattern probability classification head, [the following is done / is performed / etc.]. The candidate future paths are sorted from highest to lowest probability to obtain multiple future paths after probability sorting, and all candidate paths can be retained for subsequent risk assessment.

[0057] S7: Output Results. The system denormalizes the normalized relative displacement output by the model to meter-level displacement and superimposes it onto the last historical position to obtain future local metric coordinates; if necessary, it then calculates latitude and longitude from the local metric coordinates based on the reference point. For ship traffic management or situational awareness, all candidate tracks can be plotted simultaneously and their credibility can be indicated by probability; for collision risk warning, the highest probability track can be used or the minimum safe distance, encounter time, and risk level can be calculated for all candidate tracks; for route planning, multiple candidate prediction results can be used as planning constraints or risk cost inputs.

[0058] Example 2:

[0059] This embodiment provides a ship trajectory prediction system corresponding to the method described above. See also Figures 1 to 4 The system adopts a modular design, including a data acquisition module, a data preprocessing module, a sample construction module, a model construction module, a model training module, a trajectory prediction module, and a result application module. Each module corresponds to steps S1 to S7 in Example 1.

[0060] The data acquisition module is used to read the real-time AIS track data stream or offline historical AIS track data of the target vessel, and aggregate continuous observations of the same target vessel by vessel identifier to obtain timestamp, latitude and longitude, speed above ground, heading above ground, vessel identifier and vessel type information.

[0061] The data preprocessing module performs abnormal speed filtering, track segmentation by time interval, 15-second resampling, latitude and longitude conversion to local coordinates, kinematic feature calculation, heading encoding, and standardization on historical AIS track data to obtain a standardized multidimensional ship motion state sequence; its processing flow is similar to... Figure 2 The process is consistent with the one shown.

[0062] The sample building module is used to build samples based on the historical window length. Prediction window length The sliding step size generates training, validation, and test samples, where the input is continuous. A multidimensional sequence of ship states at a historical moment, labeled as future. The two-dimensional relative displacement at each moment.

[0063] The model building module is used to configure the input dimension, hidden dimension, number of GRU layers, short-term window, medium-term window, global window, number of candidate patterns K, and loss weights, and to build the model. Figure 3 The multimodal, multi-scale temporal attention GRU trajectory prediction model shown includes a shared GRU encoder, an adaptive multi-scale temporal attention module, a multimodal decoder, and a pattern probability classification head.

[0064] The model training module performs forward computation, Winner-Takes-All optimal mode selection, joint loss calculation, backpropagation, parameter updates, validation set evaluation, and model saving; its training process is similar to... Figure 4 The process is consistent with the one shown.

[0065] The trajectory prediction module is used to load the trained model parameters, output K candidate future trajectories and their corresponding probabilities for real-time or batch multi-dimensional ship motion state sequences of ships to be predicted, and can sort the candidate trajectories by probability.

[0066] The results application module is used to provide the prediction results to the ship traffic management system, collision risk warning system, route planning system, or maritime situation visualization platform, and based on... The project generates the prediction results needed for future ship motion risk assessment, route planning, or traffic situation display by identifying candidate future tracks and their corresponding probabilities.

[0067] Experimental Results and Analysis:

[0068] To verify the effectiveness of the method of this invention, experiments were conducted on two datasets with different geographical regions and traffic patterns: DMA and NAIS. After preprocessing, the input features were all 11-dimensional, with a historical window of 30 steps and a prediction window of 20 steps. The evaluation metrics used were average displacement error (ADE), endpoint displacement error (FDE), root mean square error (RMSE), and mean absolute error (MAE).

[0069] On the DMA test set, the model of this invention achieves an ADE of approximately 21.93m and an FDE of 47.26m, outperforming the uniform model, Kalman filter style extrapolation, LSTM, Transformer, deterministic MSTA-GRU, and comparative methods such as iTransformer, PatchTST, TrAISformer, and N-HiTS. Compared to deterministic MSTA-GRU, the ADE is reduced by approximately 15.5%, and the FDE by approximately 20.8%; compared to the stronger N-HiTS baseline, the ADE is reduced by approximately 32.4%, and the FDE by approximately 35.5%.

[0070] On the NAIS dataset, this invention also maintains good cross-regional generalization performance, indicating that the multi-scale temporal attention and multi-modal decoding structure can adapt to different sea areas, different traffic densities and different ship motion patterns.

[0071] Industrial applicability:

[0072] This invention can be widely applied to scenarios such as port vessel traffic management systems (VTS), marine intelligent monitoring platforms, ship collision risk warning systems, assisted driving and autonomous navigation systems, route planning systems, and maritime search and rescue and emergency command systems, to improve the lead time and reliability of maritime traffic situational awareness.

Claims

1. A ship trajectory prediction method based on multimodal and multi-scale temporal attention, characterized in that, Includes the following steps: S1: Obtain the historical AIS track data of the target vessel; the AIS track data includes timestamp, latitude and longitude, speed over land, heading over land, vessel identifier, and vessel type information; S2: Preprocess historical AIS track data to obtain a standardized multidimensional ship motion state sequence; S3: Construct supervised learning samples based on the preprocessed multidimensional ship state sequence; S4: Construct a multimodal, multi-scale temporal attention GRU trajectory prediction model, including a shared GRU encoder, an adaptive multi-scale temporal attention module, a multimodal decoder, and a pattern probability classification head; The shared GRU encoder is used to receive continuous... The multidimensional ship state sequence at each time point outputs include... Hidden state sequence at each time step ;in, Constrained by the model structure, This represents the final hidden state corresponding to the last historical moment. The adaptive multi-scale temporal attention module is used to divide the hidden state sequence output by the shared GRU encoder into three scales: short-term window, medium-term window, and global window. For each scale, the scale context is calculated through an independent scaling dot product attention mechanism, where the current hidden state of the multimodal decoder is used as the query vector, and the corresponding historical hidden state subsequence of the shared GRU encoder is used as the key vector and value vector. Then, a learnable gating network adaptively outputs the fusion weights of the contexts of each scale according to the current hidden state of the multimodal decoder, and the weighted sum is used to obtain the fused temporal context. The historical hidden state subsequence of the shared GRU encoder is composed of the hidden state sequences that fall into the corresponding time window from the hidden state sequence output by the shared GRU encoder. The multimodal decoder contains K learnable mode embedding vectors, used to embed the final hidden state from the shared GRU encoder output. Each path is fused with the embedding vector of each mode to obtain the independent initial decoding state of K candidate paths. At each prediction time step in the future prediction window, the future path is generated stepwise through K parallel GRUCells, conditioned on the fused temporal context. A two-dimensional displacement sequence at time points; where K is a set value. Constrained by the model structure; The pattern probability classification head is used to classify the final hidden state based on the output of the shared GRU encoder. The normalized probability distribution of K candidate tracks is obtained, which characterizes the probability of each motion mode; S5: Train a multimodal, multi-scale temporal attention GRU trajectory prediction model using supervised learning samples, to continuously... Using a multidimensional ship state sequence at a given moment as input, and future... The two-dimensional displacement of each time point relative to the position of the last historical time point is used as the prediction label; during training, the parameters are optimized using the Winner-Takes-All joint loss based on the optimal candidate trajectory to obtain the trained trajectory prediction model; the Winner-Takes-All joint loss includes position regression loss, velocity continuity loss, heading smoothing loss, pattern classification loss and diversity regularization loss; S6: The nearest ship to be predicted, whether in real-time or offline. The multidimensional ship motion state sequence at each moment is input into the trained trajectory prediction model, and the model outputs K future candidate trajectories and their corresponding probabilities. S7: Generate the prediction results required for ship future motion risk assessment, route planning, or traffic situation display based on K future candidate tracks and their corresponding probabilities.

2. The ship trajectory prediction method based on multimodal multi-scale temporal attention according to claim 1, characterized in that, The short-term window in S4 corresponds to the hidden state of the most recent first proportional step and is used to capture recent maneuver trends; the medium-term window corresponds to the hidden state of the most recent second proportional step and is used to capture stable speed and heading changes; the global window corresponds to all... The hidden state is used to capture the complete historical track pattern; where the first ratio is smaller than the second ratio.

3. The ship trajectory prediction method based on multimodal multi-scale temporal attention according to claim 2, characterized in that, For each scale The adaptive multi-scale temporal attention module adjusts the hidden state based on the current hidden state of the multimodal decoder. Generate query vectors and with a shared GRU encoder at scale Corresponding hidden state Generate key vectors Sum value vector Calculate attention weights And obtain the scale context. Subsequently, through the gating network Obtain the fusion weights at each scale and calculate the fusion context. ;in, For any time scale in a short-term window, medium-term window, or global window, The dimension of the key vector. For scale The attention weights corresponding to each historical moment are as follows. For scale The corresponding context vector, The fusion weight vector for each scale context, For scale The corresponding fusion weights, For the learnable weight matrix of the gated network, To fuse temporal context vectors, For normalized exponential functions, Both ∑ and simplify represent summation operations.

4. The ship trajectory prediction method based on multimodal multi-scale temporal attention according to claim 1, characterized in that, The multimodal decoder in S4 is for the first... The k-th candidate pattern is obtained by fusing the final hidden state of the shared GRU encoder with the embedding vector of the k-th pattern. The initial decoding state of the candidate tracks; Within the future prediction window, the multimodal decoder takes the fused temporal context and the current decoding state as input, updates the decoding state step by step through the corresponding GRUCell, and maps the decoding state to two-dimensional displacement through the displacement prediction layer, thus generating the future... Two-dimensional displacement at time t; where =1,2,...K; The pattern probability classification head maps the final hidden state of the shared GRU encoder and normalizes it using softmax to obtain the probability distribution corresponding to the K candidate tracks.

5. The ship trajectory prediction method based on multimodal multi-scale temporal attention according to claim 1, characterized in that, The Winner-Takes-All loss based on the optimal candidate trajectory in S5 is as follows: For each training sample, calculate the average displacement error between K candidate tracks and the real track, select the candidate track with the smallest average displacement error as the optimal mode, calculate the position regression loss, velocity continuity loss and heading smoothing loss only for the candidate track corresponding to the optimal mode, and use the index of the optimal mode as the mode classification label, set the mode classification loss as cross-entropy loss to constrain the probability distribution output by the mode probability classification head; and add the diversity regularization loss to the joint loss for parameter optimization.

6. The ship trajectory prediction method based on multimodal multi-scale temporal attention according to claim 5, characterized in that, The diversity regularization loss in S5 is calculated based on the pairwise distance between different candidate track endpoints. A penalty is applied when the distance between any two candidate track endpoints is less than an adaptive threshold, which is determined by the displacement magnitude of the candidate track endpoints, in order to suppress mode collapse in multimodal prediction.

7. A ship trajectory prediction system based on multimodal and multi-scale temporal attention, characterized in that, include: The data acquisition module acquires the historical AIS track data of the target vessel; the AIS track data includes timestamp, latitude and longitude, speed over land, heading over land, vessel identifier, and vessel type information; The data preprocessing module is used to perform abnormal speed filtering, track segmentation by time interval, equal-interval resampling, latitude and longitude conversion to local coordinates, kinematic feature calculation, heading encoding and standardization on historical AIS track data to obtain a standardized multidimensional ship motion state sequence. The sample construction module is used to construct supervised learning samples based on the preprocessed multidimensional ship state sequence. The model building module is used to build a multimodal, multi-scale temporal attention GRU trajectory prediction model, including a shared GRU encoder, an adaptive multi-scale temporal attention module, a multimodal decoder, and a pattern probability classification head; The model training module is used to train a multimodal, multi-scale temporal attention GRU trajectory prediction model using supervised learning samples, and obtain a trained trajectory prediction model. The trajectory prediction module is used to input the multi-dimensional ship motion state sequence of the ship to be predicted in real time or offline into the trained trajectory prediction model, and output the future candidate trajectories and their corresponding probabilities. The results application module is used to generate the prediction results required for ship future motion risk assessment, route planning, or traffic situation display based on K future candidate tracks and their corresponding probabilities.

Images