A lightweight ship attitude prediction method based on TokenLite-Former

Abstract

This invention proposes a lightweight ship attitude prediction method based on TokenLite-Former, belonging to the field of ship attitude prediction technology. The method includes: firstly, acquiring the heave, roll, and pitch three-degree-of-freedom attitude response sequences within a historical time window, performing linear mapping and position encoding after standardization; then, using the TokenLite module to evaluate the importance of historical time steps, compressing long sequences into compact token representations through segmented weighted convergence to reduce the computational overhead of Transformer self-attention; next, inputting the lightweight Transformer encoder to extract global temporal dependency features; finally, predicting the change in future attitude response relative to the current moment based on a temporal residual mechanism, and superimposing it with the current attitude to recover the absolute prediction result. Furthermore, a pre-training-fine-tuning strategy based on similar sea states is combined to improve the model's adaptability to real sea states. This invention achieves high-precision, lightweight, and extremely short-term ship attitude prediction, providing reliable support for wave compensation control, safety early warning, and real-time deployment.

Description

Technical Field

[0001] This invention relates to the fields of deep learning time series prediction and ship attitude prediction, specifically to a lightweight ship attitude prediction method based on TokenLite-Former. Background Technology

[0002] With the development of sea-based space launch technology, sea-based space launch ships have gradually become an important support platform for maritime space launch missions due to their advantages such as flexible launch site selection, high mission deployment mobility, and less restriction by the geographical conditions of land-based launch sites. However, during operation, sea-based space launch ships are subject to the coupled effects of marine environmental loads such as wind, waves, and currents, as well as the ship's motion response, resulting in attitude response exhibiting obvious nonlinear, non-stationary, and multi-degree-of-freedom coupled characteristics. Especially in critical stages such as rocket transport, erection, launch preparation, and launch window determination, changes in the ship's heave, roll, and pitch directly affect the stability of the launch platform, rocket attitude safety, equipment docking accuracy, and mission execution reliability. Therefore, conducting short-term attitude response prediction for sea-based space launch ships and obtaining the ship's motion change trends in advance within future time windows has significant engineering value for maritime launch safety assessment, launch window assistance, motion compensation control, and mission decision support.

[0003] Existing ship attitude prediction methods mainly include model-driven and data-driven methods. Model-driven methods typically rely on hydrodynamic theory, response amplitude operators, and other models to establish ship motion response prediction relationships. However, under complex wind and wave coupled excitation, ship attitude responses exhibit strong nonlinear and non-stationary characteristics, and traditional methods still have insufficient adaptability and prediction stability to complex sea state changes. In recent years, deep learning models have been used for ship motion prediction, capable of learning nonlinear mapping relationships from historical motion sequences and improving attitude prediction accuracy to some extent. However, ship attitude response sequences usually have long historical windows and multi-step prediction requirements. Transformers perform global self-attention calculations for each time step, resulting in high computational complexity, large memory consumption, and insufficient inference efficiency, making it difficult to meet the real-time deployment requirements of engineering projects. Furthermore, in the multi-step attitude prediction process, if the absolute attitude response value at future moments is directly predicted, the model needs to simultaneously fit complex components such as low-frequency drift, baseline offset, phase changes, and local disturbances, which can easily lead to the accumulation and amplification of prediction errors as the time steps increase, affecting long-term prediction stability. Meanwhile, there are usually distribution differences between simulated sea state data and real sea state data. The sample size of real sea state data is limited. Models trained directly using simulated data may have insufficient generalization ability and reduced prediction accuracy under real sea conditions.

[0004] Therefore, it is necessary to design a lightweight ship attitude prediction method based on TokenLite-Former. This method reduces the computational cost of long-sequence modeling by compressing time tokens, extracts global temporal dependency features using a lightweight Transformer, mitigates the accumulation of multi-step prediction errors by combining a time residual mechanism, and improves the model's adaptability to real sea conditions through a pre-training and fine-tuning strategy based on similar sea conditions. This enables high-precision, lightweight, and extremely short-term prediction of ship heave, roll, and pitch attitude responses. Summary of the Invention

[0005] In view of this, the present invention proposes a lightweight ship attitude prediction method based on TokenLite-Former, which aims to solve the problems of existing ship attitude prediction methods under complex wind and wave coupled excitation, such as large computational cost of long-term series modeling, easy accumulation of multi-step prediction errors, insufficient adaptability to real sea conditions, and limited real-time deployment capability in engineering.

[0006] This invention proposes a lightweight ship attitude prediction method based on TokenLite-Former, comprising: The ship's three-degree-of-freedom attitude response sequence within a historical time window is obtained. The three-degree-of-freedom attitude response sequence includes heave (Z), roll (RX), and pitch (RY). The attitude response sequence is standardized and used to construct model input samples. The input samples of the model are linearly mapped and positionally encoded, and the original pose response sequence is projected onto a high-dimensional feature space to obtain an input embedding representation for temporal feature extraction. The importance of each historical time step in the input embedding representation is evaluated using the TokenLite module, a time weight distribution is generated, and a segmented weighted aggregation method is used based on the time weight distribution to compress the long-term attitude response sequence into a compact token representation. The compact token representation is input into the lightweight Transformer encoder, and global temporal dependency features in the ship's attitude response are extracted through a multi-head self-attention mechanism and a feedforward network to obtain a temporal feature representation that reflects the evolution of the ship's historical motion. Based on the time residual mechanism, the temporal features are used to represent the change in the future attitude response relative to the current attitude response. The change is then superimposed with the current attitude response to recover the absolute attitude prediction result for the future time. By combining a pre-training-fine-tuning strategy based on similar sea conditions, the TokenLite-Former model is pre-trained using simulated sea state data and then fine-tuned using real sea state data to improve the model's adaptability under real sea conditions. Finally, the model outputs the attitude prediction results of ship heave Z, roll RX and pitch RY.

[0007] Furthermore, the three-degree-of-freedom attitude response sequence is composed of the ship's motion response data within a continuous historical time window, and the motion response data includes attitude response components in three directions: heave (Z), roll (RX), and pitch (RY). The three-degree-of-freedom attitude response sequence is arranged according to a preset sampling time interval to form a historical input sequence of length T, and the future attitude response prediction sequence is determined according to a preset prediction step size. The historical input sequence is standardized.

[0008] Furthermore, the standardized historical input sequence is used as the model input sample, and it is projected onto the high-dimensional feature space through a linear mapping layer to obtain a high-dimensional feature representation; Positional encoding is superimposed on the high-dimensional feature representation to obtain the input embedding representation.

[0009] Furthermore, when using the TokenLite module to evaluate the importance of historical time steps and generate a compact token representation, the process includes: Based on the input embedding representation, the importance of each historical time step is evaluated using the ScoreNet time scoring network to obtain the original importance score corresponding to each time step. The time scoring network adopts a lightweight temporal convolutional structure and is used to extract local time-related features, short-term oscillation features and abrupt fragment features from the ship attitude response sequence. Based on the original importance scores, the corresponding time weight distribution is generated by normalization using the Softmax function. A temperature coefficient is introduced during the Softmax normalization process; The original historical timeline is divided into multiple continuous sub-intervals, and the input embedding representation is segmented and weighted and aggregated according to the time weight distribution within each sub-interval to generate the corresponding compressed token. The compressed tokens generated in each sub-interval are combined in chronological order to obtain a compact token representation with a length shorter than the original historical input sequence.

[0010] Furthermore, the construction of the compact token representation based on the time weight distribution includes: Based on the input embedding representation, the importance score of each historical time step is output using the ScoreNet time scoring network, and then normalized using the Softmax function to obtain the time weight distribution, which has the following form: in, Indicates the first The importance score of each historical time step Represents the temperature coefficient. Indicates the length of the historical input sequence. Indicates the first The time weight corresponding to each historical time step; Divide the timeline of the historical input sequence into For the nth continuous subinterval, for the nth Sub-intervals The time weights within this sub-interval are then renormalized, taking the following form: in, Indicates the first Within each sub-interval, the first Locally normalized weights at each time step, This represents a small constant used to prevent the denominator from being zero; Based on the aforementioned locally normalized weights, for the first... The input embeddings within each sub-interval are weighted and aggregated to generate the corresponding compressed token, which can be represented as follows: in, Indicates the first Input embedding features for each historical time step, Indicates the first One compressed token; The compressed tokens generated in each sub-interval are combined in chronological order to obtain a compact token sequence: in, .

[0011] Furthermore, the extraction of global temporal dependency features using a lightweight Transformer encoder includes: Input the compact token sequence into the lightweight Transformer encoder; Based on the compact token sequence, query vector, key vector, and value vector are generated respectively, and attention weights between each compressed token are calculated, in the form of: in, This represents the compact token sequence. , , These represent the query vector, key vector, and value vector, respectively. , , For learnable parameter matrix, The dimension of the key vector; A multi-head self-attention mechanism is used to extract temporal dependencies in parallel in different feature subspaces, and the outputs of each attention head are concatenated and linearly transformed to obtain the multi-head attention output features. The multi-head attention output features are sequentially passed through residual connections, normalization layers, and feedforward networks to obtain a temporal feature representation that reflects the evolution of the ship's historical attitude response. The lightweight Transformer encoder retains only the encoder structure, removes the decoder and mask self-attention structure, and controls the model size by reducing the number of encoder layers, compressing the embedding dimension, and reducing the number of attention heads.

[0012] Furthermore, the prediction of future attitude response based on the time residual mechanism includes: The ship's current attitude response is used as the baseline for residual prediction; Let the attitude response corresponding to the last time step of the historical input sequence be... , the future No. The true attitude response at each time step is Then the true residual change in the future attitude response is: in, Indicates the prediction step size. Indicates the future number The change in attitude response at each time step relative to the current moment; Output the predicted residual change using the TokenLite-Former model And construct the residual prediction loss function, which has the following form: in, This represents the time residual prediction loss. This represents the amount of change in future attitude response predicted by the model. This represents the actual change in attitude response; During the model inference phase, the predicted residual change is superimposed with the current attitude response to recover the absolute attitude prediction result for the future time step, which takes the following form: in, Indicates the future number The absolute attitude prediction results for each time step.

[0013] Furthermore, the future attitude response prediction result is a multi-step prediction result, which includes three attitude response components of the ship: heave Z, roll RX, and pitch RY at multiple future time steps. The TokenLite-Former model employs a multi-input multi-output prediction method, which outputs the changes in the three-degree-of-freedom attitude response within the future prediction time window based on the three-degree-of-freedom attitude response sequence within the historical time window.

[0014] Furthermore, the combined pre-training-fine-tuning strategy based on similar sea conditions includes: Obtain simulation data of ship attitude response under different typical sea state conditions; Based on the motion response characteristics of the target's real sea state data, simulated sea state data that is similar to the target's real sea state is selected from the simulated data as pre-training data. The TokenLite-Former model is pre-trained using the pre-trained data to obtain the initial parameters of the model; The pre-trained TokenLite-Former model was fine-tuned using real sea state data, and the model parameters were updated.

[0015] Furthermore, the method performs very short-term predictions of the ship's heave (Z), roll (RX), and pitch (RY) attitude responses within a future time window. The attitude prediction results are used for ship stability analysis, equipment operating status assessment, operation window determination, compensation control system input, and safety auxiliary decision-making for maritime aerospace launch operations.

[0016] Compared with existing technologies, the advantages of this invention are as follows: It evaluates the importance of historical time steps using the TokenLite module and compresses long-term sequences into compact token representations using a segmented weighted aggregation method, reducing the computational overhead of Transformer self-attention while retaining key dynamic information; it extracts global temporal dependency features from ship attitude responses using a lightweight Transformer encoder, improving the model's ability to model nonlinear and non-stationary motion responses under complex wind and wave coupled excitation; it predicts the change in future attitude responses relative to the current moment using a time residual mechanism, reducing the difficulty of directly predicting absolute attitude values ​​and mitigating error accumulation during multi-step prediction; and it improves the model's adaptability, generalization ability, and prediction stability under real sea conditions through a pre-training-fine-tuning strategy based on similar sea states. Therefore, this invention can achieve high-precision, lightweight, and extremely short-term prediction of ship heave (Z), roll (RX), and pitch (RY) attitude responses, providing reliable technical support for motion compensation control, safety early warning, offshore wind power operation and maintenance, and engineering auxiliary decision-making. Attached Figure Description

[0017] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings: Figure 1 A flowchart of a lightweight ship attitude prediction method based on TokenLite-Former provided in an embodiment of the present invention; Figure 2 This is a schematic diagram of the structure of the prediction method based on TokenLite-Former of the present invention; Figure 3 This is a schematic diagram of the TokenLite portion of the present invention; Figure 4 This is a schematic diagram of the pre-training-fine-tuning strategy structure based on similar sea conditions of the present invention. Detailed Implementation

[0018] Exemplary embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the disclosure to those skilled in the art. It should be noted that, unless otherwise specified, embodiments and features in the embodiments of the present invention can be combined with each other. The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0019] Reference Figure 1-3 As shown in some embodiments of this application, a lightweight ship attitude prediction method based on TokenLite-Former includes: Step S100: Obtain the three-degree-of-freedom attitude response sequence of the ship within the historical time window. The three-degree-of-freedom attitude response sequence includes heave Z, roll RX and pitch RY. Standardize the three-degree-of-freedom attitude response sequence and construct model input samples. Step S200: Perform linear mapping and position encoding on the model input samples, project the original pose response sequence onto the high-dimensional feature space, and obtain the input embedding representation for temporal feature extraction; Step S300: Use the TokenLite module to evaluate the importance of each historical time step in the input embedding representation, generate a time weight distribution, and compress the long-term pose response sequence into a compact token representation by using a segmented weighted aggregation method based on the time weight distribution. Step S400: Input the compact token representation into the lightweight Transformer encoder, and extract the global temporal dependency features in the ship's attitude response through a multi-head self-attention mechanism and a feedforward network to obtain a temporal feature representation that reflects the evolution of the ship's historical motion. Step S500: Based on the time residual mechanism, use temporal features to represent the change in future attitude response relative to the current attitude response, and superimpose the change with the current attitude response to recover the absolute attitude prediction result for the future time. Step S600: Combining a pre-training-fine-tuning strategy based on similar sea conditions, the TokenLite-Former model is pre-trained using simulated sea state data, and the model is fine-tuned using real sea state data to improve the model's adaptability under real sea conditions. Finally, the ship's heave Z, roll RX, and pitch RY attitude prediction results are output.

[0020] Specifically, in this embodiment, the ship is subjected to the coupling effects of wind, waves, currents, and the mooring system over a long period, resulting in attitude responses exhibiting significant nonlinearity, non-stationarity, and multi-degree-of-freedom coupling characteristics. To meet the real-time requirements of wave compensation control, safety early warning, and maritime space launches for short-term prediction results, this embodiment constructs a complete technical framework: "historical attitude sequence input—time token compression—lightweight Transformer temporal modeling—temporal residual prediction—pre-training fine-tuning to adapt to real sea conditions." This framework can improve the prediction accuracy and stability of the ship's heave, roll, and pitch attitude responses under complex sea conditions while reducing the computational overhead of long-sequence modeling.

[0021] In some embodiments of this application, the three-degree-of-freedom attitude response sequence is composed of motion response data of the ship within a continuous historical time window. The motion response data includes attitude response components in three directions: heave (Z), roll (RX), and pitch (RY). Specifically, heave (Z) characterizes the overall heave and sway change of the ship under wave excitation, roll (RX) characterizes the rotational response of the ship about its longitudinal axis, and pitch (RY) characterizes the rotational response of the ship about its transverse axis. These three degrees of freedom effectively reflect the ship's key attitude changes and are closely related to the ship's deck clearance, the operational stability of superstructure equipment, the stress changes in the mooring system, and the safety of offshore operations.

[0022] Specifically, let the ship's attitude response sequence within a continuous historical time window be: in, Indicates the length of the historical input sequence. In this embodiment, the input feature dimension is represented. These correspond to the three attitude response components: heave (Z), roll (RX), and pitch (RY). For example, the length of the historical input sequence can be set to... This corresponds to a 30-second historical window with a sampling interval of 0.1 seconds; the prediction step size can be set to... This corresponds to an extremely short-term prediction window of 1 second in the future. The above parameters are merely examples and do not constitute a limitation on the scope of protection of this invention.

[0023] To improve the numerical stability of the model training process, the historical input sequences are standardized. The standardization process can be expressed as: in, and Let represent the mean and standard deviation of the historical input sequence, respectively. This represents a small constant used to prevent the denominator from being zero. Standardization reduces the differences in dimensions and amplitude ranges of heave, roll, and pitch, allowing the model to learn the coupling relationships between different degrees of freedom more stably.

[0024] Understandably, by selecting heave (Z), roll (RX), and pitch (RY) as the prediction targets, the computational burden of full multi-degree-of-freedom modeling can be reduced while focusing on the ship's key attitude responses. This approach retains core motion information closely related to ship stability and maritime operational safety, while also improving the model's applicability in real-time prediction and engineering deployment.

[0025] In some embodiments of this application, step S200, which involves linear mapping and position encoding of the model input samples, includes: inputting the standardized historical input sequence into a linear mapping layer, projecting the low-dimensional pose response sequence onto a high-dimensional feature space, and obtaining an input embedding representation. Its form can be expressed as: in, This represents the input embedding representation. and These represent the weight matrix and bias term of the linear mapping layer, respectively. Indicates the embedding dimension.

[0026] Specifically, positional encoding is superimposed on the input embedding representation to preserve the temporal order between different historical time steps. The position-encoded input embedding representation can be expressed as: in, This represents the positional encoding matrix. Through positional encoding, the model can distinguish the relative positions of different time steps in the historical sequence, providing temporal positional information for subsequent time importance assessment and global temporal dependency modeling.

[0027] Understandably, the linear mapping and position encoding process provides a unified high-dimensional feature input for the TokenLite module and the lightweight Transformer encoder, enabling the model to not only express the coupling relationship between different attitude response components, but also to preserve the chronological order of historical time series, thereby improving the ability to model the evolution of ship motion.

[0028] In some embodiments of this application, step S300, which uses the TokenLite module to evaluate the importance of historical time steps and generate compact token representations, includes: based on the input embedding representation, using the ScoreNet time scoring network to evaluate the importance of each historical time step and obtain the original importance score corresponding to each time step. The time scoring network adopts a lightweight temporal convolutional structure to extract local time-related features, short-term oscillation features, and abrupt fragment features from the ship attitude response sequence.

[0029] Specifically, ScoreNet can be composed of a one-dimensional temporal convolutional layer, a non-linear activation function, and an output convolutional layer, with its input being the position-encoded embedding representation. The output is a time importance score: in, This represents the original importance score corresponding to each historical time step. Indicates the first The importance score of each historical time step.

[0030] To convert the original importance scores into an interpretable time-weighted distribution, this embodiment uses the Softmax function for normalization: in, Indicates the first The time weight corresponding to each historical time step This represents the temperature coefficient, used to adjust the sharpness of the time-weighted distribution. When When the time step is small, the model tends to focus on a few key time steps; when the time step is small, the model tends to focus on a few key time steps. When the weight is larger, the time weight distribution is smoother, which is beneficial for preserving the overall time series information.

[0031] Understandably, using ScoreNet to evaluate the importance of historical time steps enables the model to automatically identify key time segments that contribute more to predicting future pose responses, such as local peaks, phase transition regions, and short-term perturbation regions. Compared to uniform compression or fixed patch partitioning, this mechanism can more fully preserve key dynamic information while compressing the sequence length.

[0032] In some embodiments of this application, after obtaining the time weight distribution, a segmented weighted aggregation method is further used to construct the compressed token. Specifically, the original historical timeline is divided into... Let the n continuous subintervals be: in, This represents the number of tokens after compression, and satisfies... For the first Sub-intervals The time weights within that sub-interval are then renormalized: in, Indicates the first Within each sub-interval, the first Locally normalized weights at each time step, This represents a small constant used to prevent the denominator from being zero.

[0033] Based on locally normalized weights, for the th The input embeddings within each sub-interval are weighted and aggregated to generate the corresponding compressed token: in, Indicates the first Input embedding features for each historical time step, Indicates the first Each sub-interval generates a compressed token. The compressed tokens generated in each sub-interval are combined in chronological order to obtain a compact token sequence. Understandably, the TokenLite module is used to convert the original length of... Long-time sequences are compressed into sequences of length 1000. The compact token sequence reduces the self-attention computation length of the subsequent Transformer encoder from... Reduced to Since the computational complexity of the self-attention mechanism is related to the square of the sequence length, this compression process can significantly reduce the computational and storage overhead in long-term sequence pose prediction, while retaining key dynamic information through time-importance weighting, avoiding the loss of important motion segments caused by simple downsampling.

[0034] In some embodiments of this application, the lightweight Transformer encoder in step S400 is used to perform global temporal modeling on the compressed token sequence. Specifically, it compresses the token sequence... Input a lightweight Transformer encoder and establish global relationships between different compressed tokens through a multi-head self-attention mechanism.

[0035] During the self-attention computation process, the query vector, key vector, and value vector are first generated from the compact token sequence: in, , , These represent the query vector, key vector, and value vector, respectively. , , This is the learnable parameter matrix. Attention weights are calculated based on the query vector and key vector, and the value vectors are weighted and summed, which can be expressed as: in, This represents the dimension of the key vector. Specifically, a multi-head self-attention mechanism is used to extract temporal dependencies in parallel in different feature subspaces, and the outputs of each attention head are concatenated and linearly transformed to obtain the multi-head attention output features.

[0036] Subsequently, the multi-head attention output features are sequentially passed through residual connections, normalization layers, and feedforward networks to obtain a temporal feature representation reflecting the evolution of the ship's historical attitude response. The lightweight Transformer encoder retains only the encoder structure, removing the decoder and mask self-attention structures, and controls the model size by reducing the number of encoder layers, compressing the embedding dimension, and reducing the number of attention heads.

[0037] Understandably, the lightweight Transformer encoder can establish global temporal dependencies on a compact token sequence, enabling the model to effectively capture multi-scale motion features such as periodic oscillations, slow drift, and local perturbations in the ship's attitude response. Compared to the standard Transformer directly processing complete long sequences, this invention significantly reduces the attention computation burden through pre-processing TokenLite compression, improving the model's engineering deployment capability in real-time prediction scenarios.

[0038] In some embodiments of this application, step S500, which predicts the future attitude response based on the time residual mechanism, includes: taking the attitude response corresponding to the last time step of the historical input sequence as the attitude response benchmark at the current moment, and using the temporal features output by the lightweight Transformer encoder to represent and predict the change in attitude response at multiple future time steps relative to the current moment.

[0039] Let the attitude response corresponding to the last time step of the historical input sequence be... , the future No. The true attitude response at each time step is Then the true residual change in the future attitude response can be expressed as: in, Indicates the prediction step size. Indicates the future number The change in attitude response at each time step relative to the current moment.

[0040] The TokenLite-Former model outputs the predicted residual change. The training loss function is constructed using the error between the predicted residual change and the actual residual change. in, This represents the time residual prediction loss. This represents the amount of change in future attitude response predicted by the model.

[0041] During the model inference phase, the predicted residual change is superimposed with the current attitude response to recover the absolute attitude prediction result for the future time step: in, Indicates the future number The absolute attitude prediction results for each time step.

[0042] Understandably, compared to directly predicting the absolute attitude response value at future moments, the time residual mechanism transforms the prediction target into a short-term change relative to the current moment, allowing the model to learn the local evolution trend of the ship's attitude response more comprehensively. This approach can reduce the impact of low-frequency drift, baseline shift, and amplitude changes in non-stationary sequences on model training, alleviate the error accumulation problem in the multi-step prediction process, and improve the stability and accuracy of ship attitude prediction.

[0043] In some embodiments of this application, the future attitude response prediction result is a multi-step prediction result, which includes three attitude response components of the ship: heave (Z), roll (RX), and pitch (RY) at multiple future time steps. The TokenLite-Former model adopts a multi-input multi-output prediction method, which outputs the change in the three-degree-of-freedom attitude response within the future prediction time window at once, based on the three-degree-of-freedom attitude response sequence within the historical time window.

[0044] Specifically, the model input is the three-degree-of-freedom attitude response matrix within the historical time window, and the model output is the attitude residual matrix within the future prediction time window. in, This indicates the number of future prediction time steps, and 3 represents the three attitude response components: heave (Z), roll (RX), and pitch (RY). By outputting the prediction results for multiple future time steps at once, the problem of error propagation during stepwise recursive prediction can be avoided, thus improving the efficiency and stability of multi-step prediction.

[0045] Understandably, the multi-input multi-output (MIMO) prediction method is better suited to the task of ship attitude prediction in the very short term. While ensuring prediction accuracy, it reduces the number of inferences, improves the online prediction efficiency of the model, and provides continuous future attitude response information for motion compensation control and safety early warning.

[0046] In some embodiments of this application, step S600, which combines a pre-training-fine-tuning strategy based on similar sea states to improve the model's adaptability under real sea state conditions, includes: acquiring simulated ship attitude response data under different typical sea state conditions, where typical sea states include ship motion response data under different wind speeds, significant wave heights, and characteristic periods; selecting simulated sea state data similar to the target real sea state as pre-training data based on the motion response characteristics of the target real sea state data; pre-training the TokenLite-Former model using the pre-training data to obtain initial model parameters; and fine-tuning the pre-trained TokenLite-Former model using real sea state data to update the model parameters, so that the model adapts to the distribution differences between real sea state data and simulated sea state data.

[0047] Specifically, during the simulation phase, ship attitude response data under typical sea states (Levels 3, 4, and 5) can be obtained through hydrodynamic simulation. Under different sea state conditions, the ship's geometric and mass parameters remain consistent; only the wind and wave load conditions are changed to highlight the impact of environmental excitation changes on the ship's motion response. Subsequently, based on the significant wave height, characteristic period, motion response amplitude, or spectral characteristics of the real sea state data, the simulated sea state data that most closely resembles the target real sea state is selected as pre-training data.

[0048] Understandably, a pre-training-fine-tuning strategy based on similar sea states can alleviate the problem of distribution discrepancies between simulated and real data. By first learning the basic evolution of ship motion response on similar simulated sea states, and then making minor parameter adjustments using real sea state data, the model's generalization ability, prediction stability, and small-sample adaptability under real complex sea states can be improved.

[0049] In some embodiments of this application, the training and parameter optimization of the TokenLite-Former model includes: constructing sample pairs of historical input sequences and future predicted sequences based on the training dataset, and using the change in the future pose response residual as the model training objective; constructing a training loss function using the error between the predicted residual change and the actual residual change, and updating the learnable parameters in the TokenLite module, the lightweight Transformer encoder, and the prediction output layer through backpropagation.

[0050] Specifically, the model structure parameters and training parameters are optimized. Model structure parameters include one or more of the following: embedding dimension, number of Transformer encoder layers, number of attention heads, feedforward network dimension, number of compressed tokens, ScoreNet hidden dimension, convolutional kernel size, and temperature coefficient. Training parameters include one or more of the following: learning rate, batch size, weight decay coefficient, number of training epochs, and dropout ratio. The model parameter configuration is determined through parameter search and validation set performance evaluation to achieve a balance between prediction accuracy, computational efficiency, and model complexity.

[0051] For example, the embedding dimension can be set to 128, the number of compressed tokens can be set to 64, the number of Transformer encoder layers can be set to 3, the number of attention heads can be set to 4, the feedforward network dimension can be set to 256, the ScoreNet hidden dimension can be set to 64, the convolutional kernel size can be set to 3, and the temperature coefficient can be set to 1.0. The above parameters are only examples, and those skilled in the art can adjust them according to the specific ship type, sampling frequency, prediction stride, and the performance of the deployed equipment.

[0052] Understandably, by jointly optimizing the structural and training parameters of the TokenLite-Former model, a better balance can be achieved between prediction accuracy, model complexity, and inference efficiency, making the method suitable for the real-time deployment requirements of ship prediction.

[0053] Example of implementation effect: Under complex wind and wave coupled excitation, ship attitude response typically exhibits a superposition of periodic oscillations, local abrupt changes, and non-stationary drift. Using the method in this embodiment, the TokenLite module can automatically increase the temporal weights of key dynamic segments and compress long historical sequences into compact token representations; the lightweight Transformer encoder can extract global temporal dependency features with lower computational overhead; the temporal residual mechanism can reduce the difficulty of predicting future absolute attitude values ​​and decrease the accumulation of multi-step prediction errors; and the pre-training-fine-tuning strategy can improve the model's adaptability to real sea state data. Therefore, this invention provides a stable, efficient, and reliable technical solution for very short-term ship attitude prediction.

[0054] In some embodiments of this application, the method is applicable to scenarios such as ship attitude prediction, motion compensation control, safety early warning, and maritime space launch. The method is suitable for very short-term prediction of the ship's heave (Z), roll (RX), and pitch (RY) attitude responses under complex wind and wave coupled excitation. The attitude prediction results are used to characterize the ship's heave, roll, and pitch changes within a future time window, and can serve as input to the compensation control system, operational window judgment system, and safety early warning system.

[0055] Specifically, in scenarios such as offshore wind power operation and maintenance, equipment transfer, wave compensation control, and offshore space launch, the method in this embodiment can quickly predict the trend of ship attitude change in the near future based on the ship's historical attitude response, providing data support for ship stability analysis, equipment operation status assessment, operation window judgment, and offshore operation safety decision-making.

[0056] It is understandable that this invention, through the overall technical route of "time token compression - lightweight Transformer temporal modeling - time residual prediction - pre-training fine-tuning to adapt to real sea conditions", solves the problems of existing attitude prediction methods in terms of large computational overhead for long sequence modeling, easy accumulation of multi-step prediction errors and insufficient adaptability to real sea conditions. It can meet the comprehensive requirements of ship attitude prediction for high precision, lightweight and real-time performance.

[0057] Understandably, the future attitude response prediction results are multi-step predictions, including the ship's heave (Z), roll (RX), and pitch (RY) attitude response components at multiple future time steps. The TokenLite-Former model adopts a multi-input multi-output prediction method, outputting the changes in the three degrees of freedom attitude response within the future prediction time window based on the three-degree-of-freedom attitude response sequence within the historical time window. The time residual mechanism uses the attitude response of the last time step of the historical input sequence as a benchmark to restore the future attitude response changes output by the model to the absolute attitude response values ​​at future moments. By jointly predicting the three key degrees of freedom—heave (Z), roll (RX), and pitch (RY)—the computational burden of complete multi-degree-of-freedom modeling is reduced, and the real-time performance and engineering deployment adaptability of the ship attitude prediction model are improved.

[0058] Understandably, step six, which combines a pre-training-fine-tuning strategy based on similar sea states to improve the model's adaptability to real sea states, includes: acquiring simulated ship attitude response data under different typical sea states, including ship motion response data under different wind speeds, significant wave heights, and characteristic periods; selecting simulated sea state data similar to the target real sea state as pre-training data based on the motion response characteristics of the target real sea state data; pre-training the TokenLite-Former model using the pre-training data to obtain initial model parameters, enabling the model to learn the evolution law of ship attitude response under similar sea state conditions; fine-tuning the pre-trained TokenLite-Former model using real sea state data to update model parameters, allowing the model to adapt to the distribution differences between real and simulated sea state data; and improving the model's generalization ability, prediction stability, and small-sample adaptability under real sea state conditions through the pre-training-fine-tuning strategy.

[0059] Understandably, the training and parameter optimization of the TokenLite-Former model includes: constructing sample pairs of historical input sequences and future predicted sequences based on the training dataset, and using the change in the future pose response residual as the model training objective; constructing a training loss function using the error between the predicted residual change and the actual residual change, and updating the learnable parameters in the TokenLite module, the lightweight Transformer encoder, and the prediction output layer through backpropagation; optimizing the model structure parameters and training parameters, including one or more of the following: embedding dimension, number of Transformer encoder layers, number of attention heads, feedforward network dimension, number of compressed tokens, ScoreNet hidden dimension, convolutional kernel size, and temperature coefficient; and training parameters including one or more of the following: learning rate, batch size, weight decay coefficient, number of training epochs, and dropout ratio; and determining the model parameter configuration through parameter search and validation set performance evaluation to achieve a balance between prediction accuracy, computational efficiency, and model complexity.

[0060] Understandably, the method is applicable to scenarios such as ship attitude prediction, motion compensation control, safety early warning, and offshore wind power operation and maintenance auxiliary decision-making; the method is applicable to the very short-term prediction of ship heave Z, roll RX, and pitch RY attitude responses under complex wind and wave coupled excitation; the attitude prediction results are used to characterize the heave, lateral tilt, and longitudinal tilt changes of floating offshore vessels within future time windows; the attitude prediction results are used to provide data support for ship stability analysis, equipment operating status assessment, operation window judgment, compensation control system input, and offshore operation safety decision-making.

[0061] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program goods. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program goods embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0062] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program goods according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0063] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0064] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0065] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the specific implementation of the present invention. Any modifications or equivalent substitutions that do not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A lightweight ship attitude prediction method based on TokenLite-Former, characterized in that, include: The ship's three-degree-of-freedom attitude response sequence within a historical time window is obtained. The three-degree-of-freedom attitude response sequence includes heave (Z), roll (RX), and pitch (RY). The attitude response sequence is standardized and used to construct model input samples. The input samples of the model are linearly mapped and positionally encoded, and the original pose response sequence is projected onto a high-dimensional feature space to obtain an input embedding representation for temporal feature extraction. The importance of each historical time step in the input embedding representation is evaluated using the TokenLite module, a time weight distribution is generated, and a segmented weighted aggregation method is used based on the time weight distribution to compress the long-term attitude response sequence into a compact token representation. The compact token representation is input into the lightweight Transformer encoder, and global temporal dependency features in the ship's attitude response are extracted through a multi-head self-attention mechanism and a feedforward network to obtain a temporal feature representation that reflects the evolution of the ship's historical motion. Based on the time residual mechanism, the temporal features are used to represent the change in the future attitude response relative to the current attitude response. The change is then superimposed with the current attitude response to recover the absolute attitude prediction result for the future time. By combining a pre-training-fine-tuning strategy based on similar sea conditions, the TokenLite-Former model is pre-trained using simulated sea state data and then fine-tuned using real sea state data to improve the model's adaptability under real sea conditions. Finally, the model outputs the attitude prediction results of ship heave Z, roll RX and pitch RY.

2. The lightweight ship attitude prediction method based on TokenLite-Former according to claim 1, characterized in that, The three-degree-of-freedom attitude response sequence is composed of the ship's motion response data within a continuous historical time window. The motion response data includes attitude response components in three directions: heave (Z), roll (RX), and pitch (RY). The three-degree-of-freedom attitude response sequence is arranged according to a preset sampling time interval to form a historical input sequence of length T, and the future attitude response prediction sequence is determined according to a preset prediction step size. The historical input sequence is standardized.

3. The lightweight ship attitude prediction method based on TokenLite-Former according to claim 2, characterized in that, The standardized historical input sequence is used as the model input sample, and it is projected onto the high-dimensional feature space through a linear mapping layer to obtain a high-dimensional feature representation. Positional encoding is superimposed on the high-dimensional feature representation to obtain the input embedding representation.

4. The lightweight ship attitude prediction method based on TokenLite-Former according to claim 3, characterized in that, The process of using the TokenLite module to evaluate the importance of historical time steps and generate compact token representations includes: Based on the input embedding representation, the importance of each historical time step is evaluated using the ScoreNet time scoring network to obtain the original importance score corresponding to each time step; The time scoring network adopts a lightweight temporal convolutional structure and is used to extract local time-related features, short-term oscillation features and abrupt fragment features from the ship attitude response sequence. Based on the original importance scores, the corresponding time weight distribution is generated by normalization using the Softmax function. A temperature coefficient is introduced during the Softmax normalization process; The original historical timeline is divided into multiple continuous sub-intervals, and the input embedding representation is segmented and weighted and aggregated according to the time weight distribution within each sub-interval to generate the corresponding compressed token. The compressed tokens generated in each sub-interval are combined in chronological order to obtain a compact token representation with a length shorter than the original historical input sequence.

5. The lightweight ship attitude prediction method based on TokenLite-Former according to claim 4, characterized in that, The construction of a compact token representation based on time weight distribution includes: Based on the input embedding representation, the importance score of each historical time step is output using the ScoreNet time scoring network, and then normalized using the Softmax function to obtain the time weight distribution, which has the following form: in, Indicates the first The importance score of each historical time step Represents the temperature coefficient. Indicates the length of the historical input sequence. Indicates the first The time weight corresponding to each historical time step; Divide the timeline of the historical input sequence into For the nth continuous subinterval, for the nth Sub-intervals The time weights within this sub-interval are then renormalized, taking the following form: in, Indicates the first Within each sub-interval, the first Locally normalized weights at each time step, This represents a small constant used to prevent the denominator from being zero; Based on the aforementioned locally normalized weights, for the first... The input embeddings within each sub-interval are weighted and aggregated to generate the corresponding compressed token, which can be represented as follows: in, Indicates the first Input embedding features for each historical time step, Indicates the first One compressed token; The compressed tokens generated in each sub-interval are combined in chronological order to obtain a compact token sequence: in, .

6. The lightweight ship attitude prediction method based on TokenLite-Former according to claim 5, characterized in that, The extraction of global temporal dependency features using a lightweight Transformer encoder includes: Input the compact token sequence into the lightweight Transformer encoder; Based on the compact token sequence, query vector, key vector, and value vector are generated respectively, and attention weights between each compressed token are calculated, in the form of: in, This represents the compact token sequence. , , These represent the query vector, key vector, and value vector, respectively. , , For learnable parameter matrix, The dimension of the key vector; A multi-head self-attention mechanism is used to extract temporal dependencies in parallel in different feature subspaces, and the outputs of each attention head are concatenated and linearly transformed to obtain the multi-head attention output features. The multi-head attention output features are sequentially passed through residual connections, normalization layers, and feedforward networks to obtain a temporal feature representation that reflects the evolution of the ship's historical attitude response. The lightweight Transformer encoder retains only the encoder structure, removes the decoder and mask self-attention structure, and controls the model size by reducing the number of encoder layers, compressing the embedding dimension, and reducing the number of attention heads.

7. The lightweight ship attitude prediction method based on TokenLite-Former according to claim 6, characterized in that, The prediction of future attitude response based on the time residual mechanism includes: The ship's current attitude response is used as the baseline for residual prediction; Let the attitude response corresponding to the last time step of the historical input sequence be... , the future No. The true attitude response at each time step is Then the true residual change in the future attitude response is: in, Indicates the prediction step size. Indicates the future number The change in attitude response at each time step relative to the current moment; Output the predicted residual change using the TokenLite-Former model And construct the residual prediction loss function, which has the following form: in, This represents the time residual prediction loss. This represents the amount of change in future attitude response predicted by the model. This represents the actual change in attitude response; During the model inference phase, the predicted residual change is superimposed with the current attitude response to recover the absolute attitude prediction result for the future time step, which takes the following form: in, Indicates the future number The absolute attitude prediction results for each time step.

8. The lightweight ship attitude prediction method based on TokenLite-Former according to claim 7, characterized in that, The future attitude response prediction result is a multi-step prediction result, which includes three attitude response components of the ship: heave Z, roll RX and pitch RY at multiple future time steps. The TokenLite-Former model employs a multi-input multi-output prediction method, which outputs the changes in the three-degree-of-freedom attitude response within the future prediction time window based on the three-degree-of-freedom attitude response sequence within the historical time window.

9. The lightweight ship attitude prediction method based on TokenLite-Former according to claim 1, characterized in that, The combined pre-training-fine-tuning strategy based on similar sea conditions includes: Obtain simulation data of ship attitude response under different typical sea state conditions; Based on the motion response characteristics of the target's real sea state data, simulated sea state data that is similar to the target's real sea state is selected from the simulated data as pre-training data. The TokenLite-Former model is pre-trained using the pre-trained data to obtain the initial parameters of the model; The pre-trained TokenLite-Former model was fine-tuned using real sea state data to update the model parameters.

10. The lightweight ship attitude prediction method based on TokenLite-Former according to claim 9, characterized in that, The method provides very short-term predictions of the ship's heave (Z), roll (RX), and pitch (RY) attitude responses within a future time window. The attitude prediction results are used for ship stability analysis, equipment operating status assessment, operation window determination, compensation control system input, and safety auxiliary decision-making for maritime aerospace launch operations.

Images