Hybrid method and system for multi-step real-time prediction of ship motion based on reinforcement learning
By combining real-time decomposition of ship motion data with reinforcement learning and error correction models, the problem of low accuracy in multi-step prediction of ship attitude has been solved, achieving high-precision and stable prediction results, and improving the safety and efficiency of maritime operations.
Patent Information
- Application Number
- CN202210089496.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-01-25
- Publication Date
- 2024-11-08
- Estimated Expiration
- 2042-01-25
AI Technical Summary
Existing technologies struggle to achieve high-precision, multi-step, real-time prediction of ship motion attitudes, especially with low prediction accuracy within 10–30 seconds, and the model prediction results are unstable.
The ship motion data is decomposed into high- and low-frequency subsequences using real-time decomposition technology. The prediction accuracy and stability are further improved by combining the ORELM basic prediction model with a multi-input multi-output strategy and the AdaBoost.MRT algorithm for reinforcement learning, and then corrected by the LSSVM error correction model.
It improves the accuracy and stability of ship motion attitude prediction, achieving high-precision prediction within 10 to 30 seconds, providing real-time and reliable ship attitude guidance, and improving the safety and efficiency of offshore operations.
Smart Images

Figure CN114528638B_ABST
Abstract
Description
Technical Field
[0001] The invention relates to a method for predicting a ship motion attitude, and in particular to a multi-step prediction scheme for a ship attitude based on real-time decomposition, reinforcement learning and error correction. Background Art
[0002] During the operation of ships at sea, six degrees of freedom motions will occur, namely roll, sway, pitch, surge, heave and bow pitch. Among them, roll and pitch have the greatest impact on the safety of marine operations and the ship's operating performance. If the changes in the roll and pitch of the ship can be predicted 10 to 30 seconds in advance, the corresponding crew and ship operation system can make reasonable decisions in advance, improve the stability and reliability of the ship's navigation at sea, and improve the safety and work efficiency of marine operations. In addition, the real-time ship attitude prediction model can provide reliable guidance for obtaining shaking information, thereby reducing the negative impact of roll and pitch on the ship to a greater extent. Therefore, it is of great significance to establish an accurate multi-step real-time prediction model for ship attitude.
[0003] In order to realize the multi-step real-time prediction model of ship attitude, many scholars have conducted a lot of research. With the advent of the computer age, neural networks can better fit the nonlinear mapping of ship attitude and have good performance in the multi-step real-time prediction of ship attitude. Due to the complexity of the mapping relationship of ship attitude data, short-term real-time prediction (10 to 30 seconds) is more difficult and complicated than very short-term real-time prediction (within 10 seconds). The prediction performance of a single model is slightly insufficient. In order to better predict the multi-step ship attitude in real time, hybrid models combining decomposition technology, integration technology, etc. have been widely used. Among them, strong predictors can learn the non-significant correlation of time series better than basic predictors. Due to the characteristics of ship attitude such as instability, large fluctuations and high randomness, it is difficult for the prediction model to fully learn the law of ship attitude changes, and a small part of the regular information is implicit in the prediction error, so the error correction model is also widely used in the multi-step time series hybrid prediction model. Therefore, the present invention hopes to improve the prediction ability of the neural network model and realize high-precision multi-step real-time prediction of ship motion data. Summary of the invention
[0004] The purpose of the present invention is to provide a multi-step prediction scheme for ship attitude based on real-time decomposition, reinforcement learning and error correction to address the problem of low accuracy in current prediction of ship motion attitude.
[0005] The present invention provides a hybrid method for multi-step real-time prediction of ship motion based on reinforcement learning, comprising the following steps:
[0006] Compared with the prior art, the present invention has the following advantages and beneficial effects:
[0007] 1. Perform WPD real-time decomposition on the training data set and the prediction data set, decompose the high-frequency and low-frequency components of the ship motion data into subsequences with a fixed number of layers, and improve the model's ability to learn the mapping relationship between the subsequence change rules and the ship's attitude;
[0008] 2. Establish a multi-input and multi-output ORELM basic prediction model to avoid the error accumulation of multi-step prediction. And introduce the AdaBoost.MRT algorithm to overcome the problem of unstable prediction results of the ORELM model, and enhance the learning ability of a single ORELM for ship posture sequences in multi-step prediction;
[0009] 3. An LSSVM error correction model is established for the multi-step prediction error. The error correction model can further extract the changing rules of a small part of the ship motion data contained in the multi-step error sequence, and realize more complete learning and multi-step prediction of the ship motion data. BRIEF DESCRIPTION OF THE DRAWINGS
[0010] Figure 1 It is a complete structural diagram of the prediction model proposed in the embodiment of the present invention. DETAILED DESCRIPTION
[0011] The technical solution of the present invention will be clearly and completely described below in conjunction with the embodiments and drawings.
[0012] See also Figure 1 The steps of the multi-step prediction method of ship attitude provided by the embodiment of the present invention based on real-time decomposition, reinforcement learning and error correction are as follows:
[0013] Step 1: The original ship motion data is obtained by the attitude sensor installed on the ship, and the original ship motion data is divided into a training data set and a prediction data set;
[0014] 70% of the data set is used as a training set to train the initial ORELM model and the AdaBoost.MRT algorithm to enhance the ORELM; 30% is used as a prediction set, and the prediction data set is further divided into a validation set and a test set, each accounting for 15%. The prediction error of the validation set is used to establish an error correction model; the test set is used to verify the prediction performance of the proposed model. In the embodiment, the total length of the data set is 1800 seconds.
[0015] Step 2: Perform real-time wavelet packet decomposition (RTWPD) on the training data set and prediction data set divided in step 1) separately, and decompose the high-frequency component and low-frequency component of the ship motion data into subsequences with a fixed number of layers. Preferably, the fixed number of layers of the high-frequency component subsequence is 4, and the fixed number of layers of the low-frequency component subsequence is 4; in this way, the volatility of each subsequence of the low-frequency component is smaller, and the mapping relationship between the change rules of the high-frequency component is easier to be learned by the subsequent enhanced model;
[0016] The real-time wavelet packet decomposition (RTWPD) method proposed in the present invention is an improvement on the existing wavelet packet decomposition (WPD) technology: first, the original data is divided into a data set, and the division process includes a training set, a validation set, and a test set; wherein, the training set is subjected to wavelet packet decomposition (WPD); and when the attitude sensor acquires new data, it is the validation set, and the validation set is subjected to wavelet packet decomposition, and the decomposed subsequences are sent to each prediction model that has been trained; and the above operations are repeated to complete the real-time wavelet packet decomposition (RTWPD).
[0017] In the embodiment, the decomposition steps of real-time wavelet packet decomposition RTWPD are as follows:
[0018] Step 1: Set the decomposition level to 3 and the wavelet basis function to db10; Build 2 3 wavelet packet subspaces, and construct the corresponding high-pass filter group and low-pass filter group, which are specifically expressed as follows:
[0019]
[0020] Wherein, h(k) and g(k) are respectively a high-pass filter group and a low-pass filter group with a translation parameter of k, and k is a translation parameter; represents the scale function at time t, θ(t) represents the wavelet transform function at time t, express and The inner product operation of θ(t),θ(2t-k)> represents the inner product operation of θ(t) and θ(2t-k).
[0021] Step 2: RTWPD maps the training part S(t) of the ship motion sequence to the above subspace, and obtains the subsequences S(3,0), S(3,1), S(3,2), S(3,3), S(3,4), S(3,5), S(3,6), S(3,7). The length of each subsequence is 1260 seconds. The specific decomposition algorithm is as follows:
[0022]
[0023] Where i is the decomposition layer, i∈{3,2,1,0}; j is the frequency parameter, j∈{6,4,2,0}, W k (i,j) represents the wavelet coefficients at the i-th decomposition layer, the j-th decomposition band, and the translation parameter is k;
[0024] Step 3: Get new data, update the training part S(t), and repeat Step 1-2 until all 1800 seconds of data are obtained.
[0025] Existing one-step decomposition technologies, including WPD, cannot obtain future data in actual situations, resulting in data leakage. However, the real-time wavelet packet decomposition proposed in the present invention can continuously update future data and only uses known data during training, which is in line with the actual application of ship prediction.
[0026] Step 3: For each subsequence decomposed in step 2), an outlier robust extreme learning machine (ORELM) basic prediction model is established, and the AdaBoost.MRT reinforcement learning algorithm is introduced. By continuously iteratively training ORELM, multiple ORELM models with trained initial weights and biases are combined together to obtain a reinforced ORELM model, which overcomes the problem of unstable prediction results of the original basic ORELM model and enhances the learning ability of a single ORELM for ship attitude sequences in multi-step predictions;
[0027] For the sake of reference, the following introduces the principle of ORELM prediction model:
[0028] Based on the extreme learning machine (ELM), ORELM combines the l1 norm with the ELM model to reduce the interference of outliers. By introducing the adjustment parameter C, the training error and output weight are weighed to improve the generalization performance of the ORELM model and its robustness to outliers.
[0029] The loss function of ORELM is as follows:
[0030]
[0031] st Y-Hβ=e
[0032]
[0033] Among them, Y is the target matrix, H is the output matrix, β is the output weight, e is the training error, ||·||0 is the l0 norm, ||·||1 is the l1 norm, is the l2 norm, λ is the Lagrange multiplier, μ is the error function, and L(e,β,λ) is the augmented Lagrangian function. In the augmented Lagrangian multiplier, the variables e, β, and λ are optimally estimated by repeatedly minimizing the complete Lagrangian function process. In summary, the augmented Lagrangian multiplier becomes the following form:
[0034]
[0035] Among them, τ is the number of iterations, β τ is the output weight of the τth iteration, e τ is the training error of the τth iteration, λ τ is the Lagrange multiplier of the τth iteration, It means that when L(e τ,β,λ τ ) reaches the minimum value of β, It means that when L(e,β τ+1 ,λ τ ) reaches the minimum value of e, β τ+1 is the output weight of the iteration τ+1, e τ+1 is the training error of the iteration τ+1, λ τ+1 is the Lagrange multiplier of the τ+1th iteration.
[0036] After determining the training samples (x1,y1),(x2,y2),...,(x N ,y N ), where x1,x2,...,x N represents input variables, y1,y2,...,y N Represents the output variable. After adjusting the parameter C, the steps of the ORELM-AdaBoost.MRT algorithm (i.e., AdaBoost.MRT reinforcement learning algorithm) proposed in the present invention are as follows:
[0037] Step 1: Randomly generate hidden layer parameters (w i ,b i ),i=1,2,...,L,w i is the hidden layer weight value of the i-th number of neurons, b is the hidden layer bias value of the i-th number of neurons, i represents the number of neurons in the hidden layer; L is the total number of neurons in the hidden layer; and initialize the maximum number of iterations M and the threshold parameter and the weight of each sample in the training set D m (j) is the jth sample weight in the mth iteration. N is the total number of training samples, m is the number of iterations, and m=1. The training samples (x1,y1), (x2,y2),..., (x N ,y N ) According to the weight D m Distribution, get training samples x m represents the input variable of the mth iteration, y m represents the output variable of the mth iteration;
[0038] This step is an improvement proposed by the present invention, which can enable each training sample to be integrated according to the weight of the best prediction result, and the best prediction result is better than the prediction result that is not integrated according to the weight distribution.
[0039] Step 2: ORELM is based on the training sample of the mth iteration Calculate the output weight β of the mth iteration m , training error e m , the Lagrange multiplier λm ; Get the prediction model f of the mth iteration m (g), and substitute the number of training samples into the prediction model f m (g), get the predicted output
[0040] Step 3: Output the prediction With sample output y m Subtract and get the prediction error error of the mth iteration m (j), error m (j) represents the prediction error value of the jth sample in the mth iteration and is equal to the threshold Compare all the values greater than the threshold error m (j), and their corresponding weights are added together and recorded as ε m , calculated as follows:
[0041]
[0042] Among them, ε m For the mth iteration, all values greater than the threshold error m (j), the result of adding the corresponding weights;
[0043] This step is an improvement proposed by the present invention, because the prediction error value of the obtained sample still contains predictable information, and the prediction result can be further corrected.
[0044] Step 4: According to the prediction error m (j), update the weight D m (j), calculated as follows:
[0045]
[0046] Among them, Z m is the normalization factor of the weight; D m (j) represents the mth iteration weight at time i, D m+1 (j) is the weight of the m+1th iteration;
[0047] Step 5: Let m = m + 1, repeat Step 2 to Step 5 until the maximum number of iterations M is reached, output M ORELM-AdaBoost.MRT prediction models, and complete the modeling.
[0048] Step 4: Finally, reconstruct the prediction results of the subsequence to obtain the initial prediction results of the model for the first time The multi-step initial prediction result is subtracted from the actual result Y(t), t=1261,1262,...,1800 to obtain the multi-step prediction error res(t), t=1261,1262,...,1800; a least squares support vector machine (LSSVM) error correction model is established for the multi-step prediction error res(t). The error correction model can further extract the variation law of a small part of the ship motion data contained in the multi-step error sequence res(t), and then correct the multi-step initial prediction result. Make corrections.
[0049] LSSVM is a variant of support vector machine (SVM). It introduces the least squares function and equality constraints at the cost of missing the sparsity feature in SVM, avoiding the problem of quadratic programming in SVM. LSSVM transforms the nonlinear quadratic optimization problem into a linear problem to solve. Without affecting the prediction accuracy, it reduces the complexity of the SVM algorithm and greatly improves the model calculation speed, which is helpful for estimating the error sequence of the real-time prediction model of ship motion.
[0050] The specific steps of using LSSVM to establish the error correction model are as follows:
[0051] Step 1: LSSVM maps the prediction error data to a high-dimensional space through nonlinear mapping, thereby transforming the nonlinear regression problem into a linear regression problem. The regression equation is expressed as follows:
[0052]
[0053] Among them, x res (i) represents the prediction error input at time i, res represents the prediction error, y(x res ) is the fitting output of the prediction error, is a high-dimensional nonlinear mapping of the prediction error, ω represents the weight vector, b represents the bias vector, and T represents the transpose.
[0054] Step 2: The solution of the LSSVM regression equation can be transformed into the following optimization problem:
[0055]
[0056] st
[0057] Among them, J(ω,e) is the weight vector ω and the regression error e i The objective function is i represents the regression error at time i, C' is the penalty parameter, l is the total number of sample pairs, is a constraint.
[0058] Step 3: Using the structural risk minimization method, the Lagrangian function of the objective function and constraints is given as:
[0059]
[0060] Among them, L(ω, b, e, α) is a Lagrangian function related to the weight vector ω, the bias vector b, the regression error e and the Lagrangian multiplier α, and α is the Lagrangian multiplier.
[0061] Step 4: By taking partial derivatives of the above variables, we can get the solution of the above Lagrangian function. The regression equation of LSSVM is as follows:
[0062]
[0063] in, is the predicted value of the error, K(x res (i),x res ) is the kernel function, x res (i) represents the error input of the i-th sample pair, x res Represents the total error input.
[0064] Step 5: Initial prediction results for multiple steps The correction is performed to obtain the final multi-step prediction result of the ship motion and output it. The multi-step final prediction result includes 10-step prediction, 20-step prediction and 30-step prediction. The final prediction result of the model is shown in Table 1. It can be seen from the corresponding errors that the model is feasible and stable in practical applications, and can provide real-time reliable guidance for marine operations.
[0065]
[0066] in, is the final prediction result. is the error prediction value at time t.
[0067] This step is an improvement proposed by the present invention, and the error correction process is completed after the corresponding weights are added.
[0068] In order to facilitate understanding of the technical effects of the present invention, the corresponding analysis results of the embodiments are provided as follows:
[0069] Table 1
[0070] Mean absolute error (degrees) Mean absolute percentage error (%) Root mean square error (degrees) 10-step forecast 0.1605 36.02 0.2021 20-step Forecast 0.1638 38.14 0.2043 30-step forecast 01658 42.64 0.2095
[0071] In specific implementation, the method proposed in the technical solution of the present invention can be implemented by technical personnel in this field using computer software technology to realize automatic operation process. System devices for implementing the method, such as computer-readable storage media storing the corresponding computer program of the technical solution of the present invention and computer equipment running the corresponding computer program, should also be within the scope of protection of the present invention.
[0072] In some possible embodiments, a hybrid system for multi-step real-time prediction of ship motion based on reinforcement learning is provided, including a processor and a memory, the memory being used to store program instructions, and the processor being used to call the stored instructions in the memory to execute a hybrid method for multi-step real-time prediction of ship motion based on reinforcement learning as described above.
[0073] In some possible embodiments, a hybrid system for multi-step real-time prediction of ship motion based on reinforcement learning is provided, including a readable storage medium on which a computer program is stored. When the computer program is executed, a hybrid method for multi-step real-time prediction of ship motion based on reinforcement learning as described above is implemented.
[0074] The specific embodiments described herein are merely examples of the spirit of the present invention. Those skilled in the art may make various modifications or additions to the specific embodiments described or replace them in similar ways, but they will not deviate from the spirit of the present invention or exceed the scope defined by the appended claims.
Claims
1. A hybrid method for multi-step real-time prediction of ship motion based on reinforcement learning, characterized in that: The steps include: 1) The original ship motion data is obtained by the attitude sensor installed on the ship, and the original ship motion data is divided into a training data set and a prediction data set; 2) Performing real-time wavelet packet decomposition (RTWPD) on the training data set and prediction data set divided in step 1) separately, decomposing the high-frequency component and low-frequency component of the ship motion data into subsequences with a fixed number of layers, wherein the volatility of each subsequence of the low-frequency component is smaller, and the mapping relationship between the change rules of the high-frequency component is easier to be learned by the model; 3) For each subsequence decomposed in step 2), an ORELM basic prediction model is established, and the AdaBoost.MRT reinforcement learning method is introduced. By continuously iteratively training ORELM, multiple ORELM models with trained initial weights and biases are combined together to avoid unstable prediction results of the ORELM model, and at the same time enhance the learning ability of a single ORELM for the ship attitude sequence in multi-step prediction; 4) Finally, the prediction results of the subsequence are reconstructed to obtain the model's multi-step initial prediction results. The multi-step initial prediction results are subtracted from the actual results to obtain the multi-step prediction error. An LSSVM error correction model is established for the multi-step prediction error. The error correction model is used to further extract the change law of a small part of the ship motion data contained in the multi-step error sequence, and then the multi-step initial prediction results are corrected to obtain the final multi-step prediction result of the ship motion and output it.
2. According to claim 1, a hybrid method for multi-step real-time prediction of ship motion based on reinforcement learning is characterized by: In step 1), the training data set is used to train the initial ORELM model and the AdaBoost.MRT algorithm to enhance the ORELM; the prediction data set is further divided into a validation set and a test set. The prediction error of the validation set is used to establish an error correction model, and the test set is used to verify the prediction performance of the proposed model.
3. The method for multi-step real-time prediction of ship motion based on reinforcement learning according to claim 1 is characterized in that: In step 2), the ship motion sequence training part uses a 3-layer WPD decomposition structure. The ship motion sequence training part S(t) is mapped to 2 by wavelet packet decomposition WPD. 3 In the wavelet packet subspaces, they are S(3,0), S(3,1), S(3,2), S(3,3), S(3,4), S(3,5), S(3,6), and S(3,7), respectively. The decomposition is achieved as follows: Where i is the scale parameter, i∈{3,2,1,0}; j is the frequency parameter, j∈{6,4,2,0}; W k (i,j) represents the wavelet coefficient of the jth decomposition frequency band at the i-th decomposition layer, with a translation parameter of k; and are the translation parameters; represents the wavelet coefficient of the jth decomposition frequency band at the i-th decomposition layer; h(k) and g(k) are respectively the high-pass filter group and the low-pass filter group with translation parameter k and respectively the high-pass filter group and the low-pass filter group of wavelet packet decomposition.
4. The method of multi-step real-time prediction of ship motion based on reinforcement learning according to claim 1, characterized in that: In step 2), the number of fixed layers of the high-frequency component subsequence is 4, and the number of fixed layers of the low-frequency component subsequence is 4.
5. The method of multi-step real-time prediction of ship motion based on reinforcement learning according to claim 1, characterized in that: In step 3), the ORELM model is enhanced using the AdaBoost.MRT method, which includes sequentially training a series of weak predictors ORELM. In each iteration, the distribution of the training data set is dynamically changed according to the prediction performance of the previous weak predictor, giving more weights to samples with large prediction errors, so that the next weak predictor can focus more on samples with poor training results.
6. A hybrid method for multi-step real-time prediction of ship motion based on reinforcement learning according to claim 1 or 2 or 3 or 4 or 5, characterized in that: In step 4), based on the LSSVM error correction model, the error prediction value is extracted to correct the initial prediction results of the multi-step.
7. A multi-step real-time prediction hybrid system for ship motion based on reinforcement learning, characterized by: Used to implement a hybrid method for multi-step real-time prediction of ship motion based on reinforcement learning as described in any one of claims 1-6.
8. The multi-step real-time prediction hybrid system for ship motion based on reinforcement learning according to claim 7 is characterized by: It includes a readable storage medium, on which a computer program is stored. When the computer program is executed, a hybrid method for multi-step real-time prediction of ship motion based on reinforcement learning as described in any one of claims 1 to 6 is implemented.
