Search-enhanced ship trajectory prediction method and system based on frozen traisformer and hierarchical adaptive knn
By combining the frozen TrAISformer model with hierarchical adaptive kNN retrieval, the problems of insufficient generalization ability and catastrophic forgetting in ship trajectory prediction are solved, achieving efficient and accurate trajectory prediction. It supports online updates and multi-condition combined retrieval, improving the robustness and accuracy of prediction.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- GUANGDONG POLICE COLLEGE (GUANGDONG PROVINCIAL PUBLIC SECURITY JUDICIAL MANAGEMENT CADRE COLLEGE)
- Filing Date
- 2026-03-26
- Publication Date
- 2026-06-23
AI Technical Summary
Existing ship trajectory prediction technologies suffer from problems such as insufficient generalization ability, high cost of updating parameters of pre-trained large models and susceptibility to catastrophic forgetting, lack of effective use of historical similar trajectories by purely data-driven methods, low retrieval efficiency, and weak integration with prediction models.
The frozen TrAISformer model is used to extract general trajectory features, and a hierarchical adaptive kNN retrieval module is used to retrieve highly correlated neighbor trajectories from the historical database. Weighted neighbor predictions are generated through aggregation and scoring modules, and finally, the weighted fusion prediction module is used to adaptively fuse the prediction results based on scene complexity and confidence.
It effectively improves the accuracy and robustness of the prediction algorithm, reduces computational costs, improves retrieval efficiency, supports online dynamic updates, and solves the problem of insufficient generalization ability for new routes or seasonal changes.
Smart Images
Figure CN122262811A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the fields of maritime traffic safety supervision and intelligent shipping technology, specifically to a retrieval-enhanced ship trajectory prediction method and system based on frozen TrAISformer and hierarchical adaptive k-nearest neighbor (kNN) retrieval. This invention combines a pre-trained deep learning model with retrieval enhancement technology. By freezing the TrAISformer model parameters and using a hierarchical adaptive k-nearest neighbor (kNN) retrieval database, it achieves efficient and accurate prediction of future ship trajectories. This technical solution is applicable to scenarios such as ship traffic management systems, intelligent shipping platforms, maritime regulatory systems, and decision support systems for autonomous vessels, providing reliable technical support for maritime traffic safety supervision. Background Technology
[0002] Ship trajectory prediction is one of the core technologies in maritime traffic safety supervision and intelligent shipping. Accurate ship trajectory prediction can provide decision-making basis for maritime management departments, effectively prevent ship collisions, optimize shipping scheduling, and improve the efficiency of maritime traffic operations. With the rapid development of the global shipping industry, ship traffic density continues to increase, especially in key waterways such as ports and waterways, which places higher demands on the accuracy and real-time performance of ship trajectory prediction.
[0003] Automatic Identification System (AIS) is currently the primary means of acquiring ship trajectory data. AIS systems can automatically broadcast dynamic information such as a ship's position, speed, and heading, as well as static information such as its name and type. With the global proliferation of AIS equipment, the accumulation of massive amounts of AIS data provides a crucial data foundation for ship trajectory prediction research.
[0004] Existing ship trajectory prediction technologies can be mainly classified into the following categories:
[0005] The first category is traditional prediction methods based on kinematic models. These methods assume that the ship maintains uniform linear motion or a constant turning rate for a short period of time, and use state estimation techniques such as Kalman filtering to predict the trajectory. However, while these methods are computationally simple, they struggle to capture complex navigation patterns, and their prediction accuracy is limited for scenarios with frequent changes in heading and complex navigation behavior.
[0006] The second category is prediction methods based on statistical learning. These methods acquire ship position information using surface radar, cluster the trajectory data, and then use Hidden Markov Models (HMMs) for parameter training and prediction. While these methods can model navigation patterns to a certain extent, their expressive power is limited, making it difficult to handle high-dimensional, long-term trajectory data.
[0007] The third category is prediction methods based on deep learning. In recent years, sequence modeling methods such as recurrent neural networks (RNNs), long short-term memory networks (LSTMs), and gated recurrent units (GRUs) have been widely used in ship trajectory prediction. With the emergence of the Transformer architecture, pre-trained models based on attention mechanisms, such as TrAISformer, have shown stronger sequence modeling capabilities. However, these deep learning methods have the following shortcomings: model training requires a large amount of computational resources; their generalization ability to new routes, new ship types, or seasonal changes is insufficient; and model parameter updates may lead to catastrophic forgetting, i.e., forgetting previously acquired knowledge when learning new knowledge.
[0008] The fourth category is prediction methods based on similarity retrieval. This involves dividing the predicted waterway into a regular grid, calculating the density map of ship trajectory distribution, and using similarity search to obtain the set of departure state points for trajectory prediction. However, existing retrieval methods generally suffer from problems such as a simple retrieval database structure, low retrieval efficiency, difficulty in handling multi-dimensional conditional retrieval, and insufficient integration mechanisms with deep learning prediction models.
[0009] Retrieval-Augmented Generation (RAG) technology has achieved significant success in the field of natural language processing in recent years, enhancing the generative capabilities of models by retrieving external knowledge bases. However, research on applying retrieval-augmented generation techniques to ship trajectory prediction is still in its early stages, lacking an effective scheme that combines frozen pre-trained models with hierarchical adaptive kNN retrieval.
[0010] In summary, existing ship trajectory prediction technologies suffer from the following technical problems: First, traditional deep learning models lack generalization ability when faced with new routes, new ship types, or seasonal changes; second, updating parameters of pre-trained large models is costly and may lead to catastrophic forgetting; third, purely data-driven methods lack effective utilization of historical similar trajectories; and fourth, existing retrieval enhancement methods have low retrieval efficiency and are not sufficiently integrated with the prediction model. Therefore, there is an urgent need for a ship trajectory prediction method that can fully utilize the knowledge of pre-trained models, efficiently retrieve historical similar trajectories, and support online dynamic updates. Summary of the Invention
[0011] To overcome the shortcomings and deficiencies of existing technologies, this invention provides a retrieval-enhanced ship trajectory prediction method and system based on a frozen TrAISformer and hierarchical adaptive kNN. This invention aims to address the limitations of generalization ability in traditional deep learning models and the high cost and catastrophic forgetting inherent in pre-trained large models when updating parameters. First, a parameter-frozen TrAISformer model is used to extract general trajectory features. Then, a hierarchical adaptive kNN retrieval module retrieves highly correlated neighbor trajectories from a historical database. An aggregation and scoring module generates weighted neighbor predictions. Finally, a weighted fusion prediction module adaptively fuses the dual-branch prediction results based on scene complexity and confidence level. The resulting predictions retain the sequence modeling capabilities of deep models while utilizing prior knowledge from historical data, effectively improving the accuracy and robustness of the prediction algorithm.
[0012] To achieve the above objectives, the present invention adopts the following technical solution:
[0013] A retrieval-enhanced ship trajectory prediction method based on frozen TrAISformer and hierarchical adaptive kNN, characterized by the following steps:
[0014] Preprocess the ship's AIS data stream to generate a standardized trajectory sequence;
[0015] The standardized trajectory sequence is fed into the parameter-frozen TrAISformer model to extract trajectory representation vectors and preliminary prediction results.
[0016] The trajectory representation vector and the current trajectory attributes are fed into the hierarchical adaptive kNN retrieval module to obtain the top k most similar neighbor trajectories in the four-level hierarchical index library;
[0017] The most similar neighbor trajectory is fed into the aggregation and scoring module, and the comprehensive weight of each neighbor trajectory is calculated using multi-attribute similarity to generate a weighted neighbor prediction result.
[0018] The weighted fusion prediction module adaptively fuses the preliminary prediction results and the weighted neighbor prediction results, dynamically adjusts the fusion weights according to the scene complexity, and outputs the final trajectory prediction results.
[0019] Model testing involves loading the TrAISformer model with frozen parameters and a hierarchical retrieval library, and outputting the trajectory prediction results for the vessel under test.
[0020] As a preferred technical solution, the step of preprocessing the ship's AIS data stream to generate a standardized trajectory sequence includes the following steps:
[0021] Data cleaning is performed on the AIS data stream to remove outlier and noisy data points, and missing data points are filled in using linear interpolation or cubic spline interpolation.
[0022] The geographic coordinates were converted into plane rectangular coordinates using the Mercator projection, and the latitude and longitude, speed (SOG), and heading (COG) were normalized.
[0023] The trajectory is segmented according to a preset time window or distance threshold to obtain a standardized trajectory sequence. ,in , These are sequence length and feature dimension, respectively.
[0024] The hierarchical adaptive kNN retrieval module supports online dynamic updates, including: receiving new ship trajectory data, extracting feature vectors by freezing TrAISformer, determining the storage location and inserting nodes according to the four-level index structure, and aging out historical data according to a preset strategy.
[0025] As a preferred technical solution, before feeding the standardized trajectory sequence into the parameter-frozen TrAISformer model, a model pre-training step is also included:
[0026] Construct the initial TrAISformer model using historical ship AIS trajectory datasets as the training set;
[0027] The model is pre-trained using self-supervised learning or sequence prediction tasks, and the model parameters are updated by minimizing the position prediction error.
[0028] The loss function for training the model The mean squared error loss (MSE Loss) is used, and the specific formula is as follows:
[0029]
[0030] in, This represents the number of samples in the training batch. The length of the trajectory sequence. For the first The trajectory at time True normalized coordinates These are the predicted coordinates output by the model;
[0031] Once the loss function converges or reaches the preset number of training rounds, the model parameters are saved and the state of all trainable parameters is set to frozen, thus obtaining the TrAISformer model with the parameters frozen.
[0032] As a preferred technical solution, the steps of feeding the standardized trajectory sequence into the parameter-frozen TrAISformer model to extract the trajectory representation vector and preliminary prediction results include:
[0033] The standardized trajectory sequence is fed into the input embedding layer and the position encoding layer to obtain an embedding vector with position information.
[0034] The embedded vectors are fed into a multi-layered stacked Transformer encoding layer, each layer containing a multi-head self-attention mechanism module Attention(Q,K,V) and a feedforward neural network module to extract deep temporal features;
[0035] Take the output of the last layer of the Transformer encoder as the trajectory representation vector. And generate preliminary prediction results through the output layer. ;
[0036] The TrAISformer model employs a parameter freezing strategy during prediction, setting all trainable parameters to a non-updateable state and performing only forward inference calculations without gradient backpropagation to avoid catastrophic forgetting.
[0037] As a preferred technical solution, the hierarchical adaptive kNN retrieval module adopts a four-level hierarchical index structure, specifically including:
[0038] The first-level index layer categorizes and indexes historical trajectory data according to ship type.
[0039] The second-level index layer, under the vessel type classification, partitions and indexes the historical trajectory data according to the navigation region.
[0040] The third-level index layer, under the navigation area partition, groups and indexes historical trajectory data according to seasonal factors;
[0041] The fourth-level index layer further subdivides the historical trajectory data into indexes based on time periods under the seasonal grouping, and stores the historical trajectory data and its feature vectors in the leaf nodes.
[0042] As a preferred technical solution, the step of calculating the comprehensive weight of each neighbor's trajectory using multi-attribute similarity to generate a weighted neighbor prediction result includes the following specific steps:
[0043] Calculate the similarity between the current trajectory and candidate trajectories in terms of vector space, vessel type, navigation area, seasonal factors, and time period to obtain a comprehensive similarity score. ;
[0044] The The calculation formula is as follows:
[0045]
[0046] in , , , , These are the hyperparameters for the weights of each dimension, and the sum of the weights is 1. Let the trajectory represent the cosine similarity between vectors;
[0047] The overall weight of each neighbor's trajectory is calculated based on the overall similarity. and utilize We obtain the weighted aggregation of neighbor trajectories. .
[0048] As a preferred technical solution, the comprehensive weight and weighted neighbor prediction results The calculation formula is as follows:
[0049]
[0050]
[0051] in This indicates normalization processing. For the first Confidence factor of each neighbor, As a time-sensitive factor, For the first A neighbor at any time The location.
[0052] As a preferred technical solution, the step of adaptively fusing the preliminary prediction result and the weighted neighbor prediction result using the weighted fusion prediction module to output the final trajectory prediction result includes the following steps:
[0053] Assess the complexity of the current prediction scenario. And calculate the prediction confidence of the TrAISformer model respectively. Confidence of kNN search results ;
[0054] The fusion weights of the TRAISformer branches are dynamically calculated based on scene complexity and confidence level. ;
[0055] The The calculation formula is as follows:
[0056]
[0057] in The range of values is A larger value indicates a more complex scenario.
[0058] 9. The retrieval-enhanced ship trajectory prediction method based on frozen TrAISformer and hierarchical adaptive kNN as described in claim 8, characterized in that the output of the final trajectory prediction result... The specific calculation formula is as follows:
[0059]
[0060] in The initial predicted location is output by the TrAISformer model. To predict the location of weighted neighbors, For the predicted time.
[0061] A retrieval-enhanced ship trajectory prediction system based on frozen TrAISformer and hierarchical adaptive kNN based on any of the above detection methods includes: a data preprocessing module, a frozen TrAISformer encoding module, a hierarchical adaptive kNN retrieval module, an aggregation and scoring module, and a weighted fusion prediction module.
[0062] The data preprocessing module is used to clean, interpolate, and standardize the ship AIS data stream to generate a standardized trajectory sequence.
[0063] The frozen TrAISformer encoding module is used to feed the standardized trajectory sequence into the parameter-frozen TrAISformer model to extract the trajectory representation vector and preliminary prediction results;
[0064] The hierarchical adaptive kNN retrieval module is used to maintain a four-level hierarchical index library and perform adaptive retrieval based on trajectory representation vectors and attribute information to obtain the top k most similar neighbor trajectories.
[0065] The aggregation and scoring module is used to calculate the multi-attribute comprehensive weight of each neighbor's trajectory and generate weighted neighbor prediction results;
[0066] The weighted fusion prediction module is used to dynamically adjust the weights according to the scene complexity and prediction confidence, fuse the preliminary prediction results and the weighted neighbor prediction results, and output the final trajectory prediction results.
[0067] Compared with the prior art, the present invention has the following advantages and beneficial effects:
[0068] (1) The present invention adopts a parameter freezing strategy. The parameters of the TrAISformer model are kept completely frozen throughout the prediction process, which preserves the general trajectory modeling knowledge learned in the pre-training stage, effectively avoids the catastrophic forgetting problem common in traditional fine-tuning methods, and significantly reduces the computational cost.
[0069] (2) The present invention designs a hierarchical adaptive kNN retrieval library, which adopts a four-level hierarchical index structure of ship type, region, season and time, supports multi-condition combination retrieval and pruning optimization. Compared with the traditional flat index structure, it significantly reduces the retrieval time complexity and improves the retrieval efficiency.
[0070] (3) The present invention is designed to support an online dynamic update retrieval mechanism. New ship trajectory data can be added to the corresponding index node in the retrieval database in real time, enabling the system to continuously learn and quickly adapt to new scenarios, effectively solving the problem of insufficient generalization ability of the model to new routes or seasonal changes.
[0071] (4) The present invention designs an adaptive weighted fusion mechanism that comprehensively utilizes the deep feature extraction capability of the TrAISformer pre-trained model and the historical experience information of kNN retrieval. The weights are dynamically adjusted according to the scene complexity and prediction confidence, achieving complementary advantages and effectively improving the accuracy and robustness of trajectory prediction. Attached Figure Description
[0072] Figure 1 This is a schematic diagram of the overall architecture of the retrieval-enhanced ship trajectory prediction system based on frozen TrAISformer and hierarchical adaptive kNN provided in an embodiment of the present invention.
[0073] Figure 2 A flowchart of the data preprocessing module provided in an embodiment of the present invention. Figure 2 In the diagram, 201 is the AIS raw data input, 202 is the data cleaning unit, 203 is the interpolation and completion unit, 204 is the coordinate standardization unit, 205 is the trajectory segmentation unit, and 206 is the standardized trajectory sequence output.
[0074] Figure 3 This is a schematic diagram of the frozen TrAISformer encoding module structure provided in an embodiment of the present invention. Figure 3 In the diagram, 301 is the input embedding layer, 302 is the position encoding layer, 303 is the multi-head self-attention layer, 304 is the Add&Norm layer, 305 is the feedforward neural network layer, and 306 is the output layer.
[0075] Figure 4 This is a schematic diagram of the hierarchical adaptive kNN retrieval library index structure provided in an embodiment of the present invention. Figure 4In the table, 401 is the first-level ship type index, 402 is the second-level region index, 403 is the third-level season index, 404 is the fourth-level time period index, and 405 is the trajectory data storage node.
[0076] Figure 5 This is a flowchart of multi-attribute similarity calculation provided in an embodiment of the present invention. Figure 5 In the table, 501 is for ship type similarity calculation, 502 is for navigation area similarity calculation, 503 is for seasonal factor similarity calculation, 504 is for time period similarity calculation, 505 is for weighted comprehensive calculation, and 506 is for comprehensive similarity output.
[0077] Figure 6 This is a schematic diagram illustrating the working principle of the weighted fusion prediction module provided in an embodiment of the present invention. Figure 6 In the diagram, 601 is the input for the TrAISformer prediction result, 602 is the input for the kNN weighted prediction result, 603 is the scene complexity evaluation unit, 604 is the confidence calculation unit, 605 is the fusion weight adjustment unit, 606 is the result fusion unit, and 607 is the final prediction result output.
[0078] Figure 7 The flowchart of the ship trajectory prediction method provided in this embodiment of the invention illustrates the complete processing flow from AIS data acquisition to the final prediction result output.
[0079] Figure 8 The figure shows the comparison results of the retrieval enhancement fusion model proposed in this invention and the benchmark model in terms of trajectory prediction error. (a) is a comparison of the average error trend with the prediction time, and (b) is a comparison of the overall average performance and the improvement ratio.
[0080] Figure 9 The image shows a comparison of the trajectory prediction visualization effects in specific ship navigation scenarios provided by the embodiments of the present invention. Detailed Implementation
[0081] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.
[0082] Example 1:
[0083] This embodiment uses a real ship AIS trajectory dataset for training, validation, and testing. The dataset contains navigation data for various types of ships (such as cargo ships, tankers, and passenger ships) in different sea areas, seasons, and time periods. This embodiment is simulated on a Linux system, implemented using the deep learning framework PyTorch, and utilizes high-performance GPUs for accelerated computation.
[0084] like Figure 1 shown (Note: Figure 1 (This is a schematic diagram of the overall system architecture provided in an embodiment of the present invention.) This embodiment provides a retrieval-enhanced ship trajectory prediction method based on frozen TrAISformer and hierarchical adaptive kNN, including the following steps:
[0085] S1: Acquire the ship's AIS data stream, perform data preprocessing on the AIS data stream, and generate a standardized trajectory sequence;
[0086] In this embodiment, the original message is first received through the AIS data interface, and then parsed to extract the information including MMSI and longitude. ),latitude( ), ground speed ( ), ground heading ( ) and timestamp ( Data streams with characteristics such as )
[0087] The data is then cleaned to remove outlier points and noisy data. For data gaps that occur during transmission, linear interpolation or spline interpolation is used to fill in the gaps. Let two known adjacent data points be points. (time ,Location ) and points (time ,Location (Time stamps need to be filled in) Data points ( Then the position of the completion point. The calculation formula is:
[0088]
[0089] Finally, the coordinate data is subjected to Mercator projection and normalization, and the continuous data stream is divided into a fixed-length standardized trajectory sequence according to a preset time window or distance threshold, which serves as the input to the model.
[0090] S2: Encode the TrAISformer model with the input parameters of the standardized trajectory sequence to obtain the trajectory representation vector and preliminary prediction results;
[0091] In this embodiment, the standardized trajectory sequence is fed into the TrAISformer encoding module. This module adopts a GPT-style Transformer architecture, which includes an input embedding layer, a position encoding layer, multiple stacked Transformer encoding layers, and an output layer.
[0092] To avoid catastrophic forgetting during online learning, this embodiment adopts a parameter freezing strategy: load the pre-trained TrAISformer model weights and set all trainable parameters to a non-updateable state (requires_grad=False) to keep the network structure and parameter weights fixed.
[0093] After embedding, the input sequence enters a multi-head self-attention layer to calculate the temporal dependencies within the sequence. The calculation formula is as follows:
[0094]
[0095] in, These are the query matrix, key matrix, and value matrix, respectively. is the dimension of the key vector.
[0096] The module ultimately outputs two parts of data: first, the output of the last layer of the Transformer encoder is used as a high-dimensional trajectory representation vector. The first is for subsequent retrieval; the second is based on preliminary prediction results generated by autoregression. .
[0097] In this embodiment, before feeding the standardized trajectory sequence into the parameter-frozen TrAISformer model, a model pre-training step is also included:
[0098] Construct the initial TrAISformer model using historical ship AIS trajectory datasets as the training set;
[0099] The model is pre-trained using self-supervised learning or sequence prediction tasks, and the model parameters are updated by minimizing the position prediction error.
[0100] The loss function for training the model The mean squared error loss is used, and the specific formula is as follows:
[0101]
[0102] in, This represents the number of samples in the training batch. The length of the trajectory sequence. For the first The trajectory at time True normalized coordinates These are the predicted coordinates output by the model;
[0103] Once the loss function converges or reaches the preset number of training rounds, the model parameters are saved and the state of all trainable parameters is set to frozen, thus obtaining the parameter-frozen TrAISformer model.
[0104] S3: Input the trajectory representation vector and the current trajectory attribute into the hierarchical adaptive kNN retrieval module for adaptive retrieval, and obtain the top k most similar neighbor trajectories and their corresponding similarity scores in the four-level hierarchical index library;
[0105] In this embodiment, the retrieval database (hierarchical adaptive kNN retrieval module) adopts a four-level hierarchical index structure of "ship type-navigation area-seasonal factors-time period". The retrieval process first quickly locates the leaf node in the index tree based on the static attributes of the current trajectory, and filters out the candidate set, thereby significantly reducing the time complexity of the retrieval.
[0106] Subsequently, the current trajectory representation vector is calculated. With candidate trajectory representation vector Cosine similarity between them:
[0107]
[0108] Based on the calculated similarity scores, the top scorers are selected. Each trajectory serves as the set of most similar neighbors. Furthermore, this retrieval library (hierarchical adaptive kNN retrieval module) supports online dynamic updates, including: receiving new ship trajectory data, extracting feature vectors by freezing the TrAISformer, determining storage locations and inserting nodes according to the four-level index structure, and simultaneously aging historical data according to a preset strategy. It can receive new trajectory data in real time and insert corresponding index nodes, enabling continuous model learning.
[0109] S4: Aggregate and score the most similar neighbor trajectories, calculate the comprehensive weight of each neighbor trajectory, and generate a weighted neighbor prediction result;
[0110] In this embodiment, to more accurately evaluate the reference value of neighbor trajectories, multi-attribute similarity calculation is introduced in addition to vector similarity. The similarity of ship types is calculated separately (…). ), navigation area similarity ( ), seasonal similarity ( ) and time period similarity ( ) .
[0111] Overall similarity score The calculation formula is as follows:
[0112]
[0113] in, These are the weight parameters for each dimension.
[0114] Based on comprehensive similarity, combined with confidence factor and timeliness factor Calculate the first Normalized composite weight of each neighbor :
[0115]
[0116] Finally, the future paths of each neighbor's trajectory are weighted and aggregated using a comprehensive weighting method to generate a weighted neighbor prediction result. :
[0117]
[0118] S5: Fuse the preliminary prediction result with the weighted neighbor prediction result, adaptively adjust the fusion weight according to the scene, and output the final trajectory prediction result.
[0119] In this embodiment, the complexity factor of the current scene is calculated by the scene evaluation unit. And combined with the prediction confidence of the TrAISformer model Confidence of kNN search results The fusion weights are dynamically adjusted.
[0120] The calculation of the fusion weights uses a linear interpolation strategy, for example, for the weights of TrAISformer. :
[0121]
[0122] The corresponding kNN fusion weights are .
[0123] Final trajectory prediction location The result is obtained by weighted fusion of the two:
[0124]
[0125] This fusion mechanism automatically increases the weight of the retrieval results when the scenario complexity is high or the confidence of the pre-trained model is low, thereby improving the robustness of the prediction.
[0126] S6: Model validation and testing, outputting the final prediction metric.
[0127] In this embodiment, the average displacement error (ADE) is used as the main evaluation metric. To verify the effectiveness of the method, experiments were conducted on the AIS historical trajectory dataset. The experiments compared the average prediction errors of the three methods:
[0128] method Average error (km) Basic TrAISformer 7.9061 kNN model 8.7594 This invention fusion model 7.5246
[0129] Experimental results show that the fusion model proposed in this invention reduces the average prediction error by 4.83% compared with the basic TrAISformer model.
[0130] The formula for calculating the relative improvement rate is:
[0131]
[0132] in The average error of the basic TrAISformer This represents the average error of the fusion model. Substitute the data to calculate:
[0133]
[0134] Experimental results validate the effectiveness of the method presented in this invention. By combining the frozen TrAISformer model with hierarchical adaptive kNN retrieval and employing an adaptive weighted fusion mechanism, the method of this invention can effectively improve the accuracy of ship trajectory prediction. To more intuitively verify the above-mentioned improvement effect, Figure 8 The results of the visualization comparison are further presented. For example... Figure 8 As shown in (a), the growth trend of the average prediction error (ADE) of each model is illustrated as the prediction duration changes from 0 to 4 hours. It can be seen that the error of all models increases with the duration of prediction, but the proposed fusion model (solid green line) consistently remains below the baseline TrAISformer model (dashed blue line) and the traditional kNN-LM model (solid red line). The green shaded area in the figure visually reflects the reduction in prediction error compared to the baseline model, demonstrating that introducing frozen high-dimensional semantic features for retrieval enhancement can effectively suppress error accumulation in long-term prediction.
[0135] like Figure 8 As shown in (b), in the overall average performance comparison, the overall average error of the fusion model of this invention is reduced to 7.5246 km (shown in the figure as the relative proportional optimization corresponding to 8.15 km), which clearly indicates an improvement of approximately 4.83% compared to the basic TrAISformer model. This quantitative result is consistent with the above calculations, fully demonstrating that this invention, through an adaptive weighted fusion mechanism, successfully combines the sequence modeling capability of deep models with the historical prior advantages of retrieval mechanisms, significantly improving prediction accuracy.
[0136] To further verify the predictive performance of the method of the present invention at the microscopic level, Figure 9 A representative example of ship navigation was selected for visualization. Figure 9The horizontal and vertical axes in the figure represent the normalized longitude and latitude, respectively.
[0137] In the figure, the dark blue solid line (labeled as "historical observation trajectory") represents the historical ship navigation path of the input model; the green dashed line (labeled as "true future trajectory") represents the actual future navigation path of the ship, serving as a Ground Truth reference; and the orange solid line marked with "x" (labeled as "predicted trajectory (method of this paper)") represents the output result of the fusion model of this invention.
[0138] from Figure 9 The visualization results show that the predicted trajectory generated by the fusion model proposed in this invention highly overlaps with the actual future trajectory. It maintains high accuracy not only during the straight-line navigation phase but also accurately captures the motion trend during nonlinear changes in the trajectory (where the curve bends in the figure). This intuitively demonstrates that the model, by combining the general features of the frozen TrAISformer with the historical prior information of the hierarchical adaptive kNN, can effectively cope with complex navigation scenarios and possesses excellent fitting capabilities.
[0139] Example 2:
[0140] This embodiment provides a retrieval-enhanced ship trajectory prediction system based on the frozen TrAISformer and hierarchical adaptive kNN based on the ship trajectory prediction method in Embodiment 1. The system includes: a data preprocessing module, a frozen TrAISformer encoding module, a hierarchical adaptive kNN retrieval module, an aggregation and scoring module, and a weighted fusion prediction module.
[0141] The data preprocessing module is used to receive ship AIS data streams and perform data cleaning and interpolation completion (such as linear interpolation formulas). ), coordinate standardization and trajectory segmentation processing, outputting a standardized trajectory sequence;
[0142] The frozen TrAISformer encoding module is used to load pre-trained weights and freeze parameters, and to perform multi-head self-attention calculation on the standardized trajectory sequence. The process involves a feedforward network to process the data, outputting a trajectory representation vector. and preliminary forecast results ;
[0143] The hierarchical adaptive kNN retrieval module is used to maintain a four-level index library containing "type-region-season-time", and calculates cosine similarity based on the trajectory representation vector. And combined with attribute information retrieval to obtain the previous The most similar neighbor;
[0144] The aggregation and scoring module is used to calculate the multi-attribute comprehensive similarity. Normalized weights are generated by combining timeliness and confidence level. Based on this, a weighted neighbor prediction result is generated. ;
[0145] The weighted fusion prediction module is used to evaluate the complexity of the scenario. Dynamically calculate fusion weights and The preliminary prediction results are then fused with the weighted neighbor prediction results. ), outputting the final trajectory prediction result.
[0146] This invention addresses the problems of high update costs, catastrophic forgetting, and weak generalization ability to new scenarios in existing ship trajectory prediction technologies. It proposes a retrieval-enhanced prediction method based on a frozen TrAISformer and hierarchical adaptive kNN. This invention utilizes a pre-trained model with frozen parameters to extract general features, combines this with a four-level hierarchical index to achieve efficient historical trajectory retrieval, and fuses model predictions and retrieval results through an adaptive weighting mechanism. Experiments demonstrate that this method significantly improves the accuracy and robustness of trajectory prediction while reducing computational costs, showing promising application prospects.
[0147] The above description is only a preferred embodiment of the present invention and does not limit the patent scope of the present invention. All equivalent structural transformations made under the inventive concept of the present invention using the contents of the present invention specification and drawings, or direct or indirect applications in other related technical fields, are included within the patent protection scope of the present invention.
Claims
1. A retrieval-enhanced ship trajectory prediction method based on frozen TrAISformer and hierarchical adaptive kNN, characterized in that, Includes the following steps: Preprocess the ship's AIS data stream to generate a standardized trajectory sequence; The standardized trajectory sequence is fed into the parameter-frozen TrAISformer model to extract trajectory representation vectors and preliminary prediction results. The trajectory representation vector and the current trajectory attributes are fed into the hierarchical adaptive kNN retrieval module to retrieve the previous trajectory from the four-level hierarchical index library. The most similar neighbor trajectory; The most similar neighbor trajectory is fed into the aggregation and scoring module, and the comprehensive weight of each neighbor trajectory is calculated using multi-attribute similarity to generate a weighted neighbor prediction result. The weighted fusion prediction module adaptively fuses the preliminary prediction results and the weighted neighbor prediction results, dynamically adjusts the fusion weights according to the scene complexity, and outputs the final trajectory prediction results. Model testing involves loading the TrAISformer model with frozen parameters and a hierarchical retrieval library, and outputting the trajectory prediction results for the vessel under test.
2. The retrieval-enhanced ship trajectory prediction method based on frozen TrAISformer and hierarchical adaptive kNN as described in claim 1, characterized in that, The step of preprocessing the ship AIS data stream to generate a standardized trajectory sequence includes: Data cleaning is performed on the AIS data stream to remove outlier and noisy data points, and missing data points are filled in using linear interpolation or cubic spline interpolation. The Mercator projection is used to convert geographic coordinates into Cartesian coordinates, and latitude, longitude, and speed are then compared. ,course Perform normalization processing; The trajectory is segmented according to a preset time window or distance threshold to obtain a standardized trajectory sequence. ,in , These are sequence length and feature dimension, respectively. The hierarchical adaptive kNN retrieval module supports online dynamic updates, including: receiving new ship trajectory data, extracting feature vectors by freezing TrAISformer, determining the storage location and inserting nodes according to the four-level index structure, and aging out historical data according to a preset strategy.
3. The retrieval-enhanced ship trajectory prediction method based on frozen TrAISformer and hierarchical adaptive kNN as described in claim 1, characterized in that, Before feeding the standardized trajectory sequence into the parameter-frozen TrAISformer model, a model pre-training step is also included: Construct the initial TrAISformer model using historical ship AIS trajectory datasets as the training set; The model is pre-trained using self-supervised learning or sequence prediction tasks, and the model parameters are updated by minimizing the position prediction error. The loss function for training the model The mean squared error loss is used, and the specific formula is as follows: in, This represents the number of samples in the training batch. The length of the trajectory sequence. For the first The trajectory at time True normalized coordinates These are the predicted coordinates output by the model; Once the loss function converges or reaches the preset number of training rounds, the model parameters are saved and the state of all trainable parameters is set to frozen, thus obtaining the parameter-frozen TrAISformer model.
4. The retrieval-enhanced ship trajectory prediction method based on frozen TrAISformer and hierarchical adaptive kNN as described in claim 1, characterized in that, The process of feeding the standardized trajectory sequence into the parameter-frozen TrAISformer model to extract trajectory representation vectors and preliminary prediction results includes the following steps: The standardized trajectory sequence is fed into the input embedding layer and the position encoding layer to obtain an embedding vector with position information. The embedding vector is fed into a multi-layered stacked Transformer encoding layer, each layer containing a multi-head self-attention mechanism module. Combined with a feedforward neural network module, it extracts deep temporal features; Take the output of the last layer of the Transformer encoder as the trajectory representation vector. And generate preliminary prediction results through the output layer. ; The TrAISformer model employs a parameter freezing strategy during prediction, setting all trainable parameters to a non-updateable state and performing only forward inference calculations without gradient backpropagation to avoid catastrophic forgetting.
5. The retrieval-enhanced ship trajectory prediction method based on frozen TrAISformer and hierarchical adaptive kNN as described in claim 1, characterized in that, The hierarchical adaptive kNN retrieval module adopts a four-level hierarchical index structure, specifically including: The first-level index layer categorizes and indexes historical trajectory data according to vessel type; The second-level index layer, under the vessel type classification, partitions and indexes the historical trajectory data according to the navigation area; The third-level index layer groups and indexes historical trajectory data according to seasonal factors under the navigation area partition; The fourth-level index layer, under the seasonal grouping, further subdivides the historical trajectory data into indexes according to time periods, and stores the historical trajectory data and its feature vectors in the leaf nodes.
6. The retrieval-enhanced ship trajectory prediction method based on frozen TrAISformer and hierarchical adaptive kNN as described in claim 1, characterized in that, The step of calculating the comprehensive weight of each neighbor's trajectory using multi-attribute similarity to generate a weighted neighbor prediction result includes: Calculate the similarity between the current trajectory and candidate trajectories in terms of vector space, vessel type, navigation area, seasonal factors, and time period to obtain a comprehensive similarity score. ; The The calculation formula is as follows: in , , , , These are the hyperparameters for the weights of each dimension, and the sum of the weights is 1. Let the trajectory represent the cosine similarity between vectors; The overall weight of each neighbor's trajectory is calculated based on the overall similarity. and utilize We obtain the weighted aggregation of neighbor trajectories. .
7. The retrieval-enhanced ship trajectory prediction method based on frozen TrAISformer and hierarchical adaptive kNN as described in claim 6, characterized in that, The comprehensive weight and weighted neighbor prediction results The calculation formula is as follows: in This indicates normalization processing. For the first Confidence factor of each neighbor, As a time-sensitive factor, For the first A neighbor at any time The location.
8. The retrieval-enhanced ship trajectory prediction method based on frozen TrAISformer and hierarchical adaptive kNN as described in claim 1, characterized in that, The step of using a weighted fusion prediction module to adaptively fuse the preliminary prediction results and the weighted neighbor prediction results to output the final trajectory prediction result includes the following steps: Assess the complexity of the current prediction scenario. And calculate the prediction confidence of the TrAISformer model respectively. Confidence of kNN search results ; The fusion weights of the TRAISformer branches are dynamically calculated based on scene complexity and confidence level. ; The The calculation formula is as follows: in The range of values is A larger value indicates a more complex scenario.
9. The retrieval-enhanced ship trajectory prediction method based on frozen TrAISformer and hierarchical adaptive kNN as described in claim 8, characterized in that, The output final trajectory prediction result The specific calculation formula is as follows: in The initial predicted location is output by the TrAISformer model. To predict the location of weighted neighbors, For the predicted time.
10. A retrieval-enhanced ship trajectory prediction system based on frozen TrAISformer and hierarchical adaptive kNN, based on the detection method of any one of claims 1-9, characterized in that, include: Data preprocessing module, frozen TrAISformer encoding module, hierarchical adaptive kNN retrieval module, aggregation and scoring module, weighted fusion prediction module; The data preprocessing module is used to clean, interpolate, and standardize the ship AIS data stream to generate a standardized trajectory sequence. The frozen TrAISformer encoding module is used to feed the standardized trajectory sequence into the parameter-frozen TrAISformer model to extract the trajectory representation vector and preliminary prediction results; The hierarchical adaptive kNN retrieval module is used to maintain a four-level hierarchical index library and perform adaptive retrieval based on trajectory representation vectors and attribute information to obtain the previous... The most similar neighbor trajectory; The aggregation and scoring module is used to calculate the multi-attribute comprehensive weight of each neighbor's trajectory and generate weighted neighbor prediction results; The weighted fusion prediction module is used to dynamically adjust the weights according to the scene complexity and prediction confidence, fuse the preliminary prediction results and the weighted neighbor prediction results, and output the final trajectory prediction results.