Marine rocket launching platform wave compensation control method based on deep reinforcement learning

Abstract

The invention discloses a sea rocket launching platform wave compensation control method based on deep reinforcement learning. The sea rocket launching platform wave compensation control method comprises the following steps: S1, collecting real-time motion state data; s2, inputting the motion state data into an improved Shenchang differential equation model; s3, constructing a strategy network according to a SoftActor-Critic algorithm, and generating a corresponding continuous control action signal; s4, the control action signal is input into a propeller control system and a ballast adjusting system, and combined compensation control over the posture and the position of the platform is achieved; s5, in a simulation environment, carrying out end-to-end joint training on the ShentActor-Critic strategy network and the ShentActor-Critic strategy network; and S6, realizing wave compensation of the offshore rocket launching platform in the wave disturbance environment in the platform control system. According to the invention, accurate prediction and adaptive compensation of platform stability control under complex disturbance are realized, and the control precision and reliability are significantly improved.

Description

Technical Field

[0001] This invention relates to the field of rocket technology, and in particular to a wave compensation control method for marine rocket launch platforms based on deep reinforcement learning. Background Technology

[0002] As an important supplement to modern space launch methods, sea-based rocket launch platforms offer flexible launch windows and high-latitude launch capabilities, making them particularly suitable for specific missions such as sun-synchronous orbit and polar orbit. However, compared to land-based launch platforms, sea-based launch platforms face extremely complex environmental disturbances, especially when affected by multiple sources of disturbance such as waves, wind, and ocean currents. Their attitude stability and platform displacement control become critical factors affecting the safety and accuracy of rocket launches. Therefore, how to achieve real-time, precise, and robust compensation control of the platform's attitude and position under wave-disturbed environments has become a core technical challenge in current sea-based rocket launch control systems.

[0003] Existing control methods for offshore launch platforms mostly employ traditional rule-based control algorithms such as PID control, fuzzy control, or robust control, adjusting the platform's attitude deviation through preset feedback control laws. While these methods can achieve some control effectiveness under stable sea conditions, their performance often deteriorates significantly under complex sea conditions involving highly nonlinear disturbances, sudden random waves, and high-dimensional state coupling. This is because these rule-based control methods essentially rely on manually set control laws and parameter adjustment ranges, lacking the ability to learn from dynamic changes in disturbances and generalize to complex state spaces. Furthermore, as the complexity of control tasks increases, the difficulty of parameter tuning in multi-input multi-output systems increases significantly, making it difficult to achieve the goal of simultaneous real-time control and dynamic adaptation.

[0004] In recent years, with the rapid development of deep learning and reinforcement learning technologies, intelligent control methods have begun to be introduced into fields such as marine platform control, ship obstacle avoidance, and floating structure dynamics control. Deep reinforcement learning, as an intelligent algorithm that autonomously learns optimal policies through interaction with the environment, possesses the potential to complete control tasks in complex dynamic systems due to its policy generation capabilities in high-dimensional state spaces. For example, methods such as Deep Deterministic Policy Gradient (DDPG), Proximal Policy Optimization (PPO), and Soft Actor-Critic (SAC) have been gradually applied to scenarios such as path planning for surface robots and dynamic positioning and autonomous stabilization control of ships. However, directly applying deep reinforcement learning methods to high-precision control systems at the rocket launch level still faces several practical challenges.

[0005] First, deep reinforcement learning methods typically rely on discrete time steps and Markov decision processes for modeling. However, the attitude and dynamic response of a marine rocket launch platform exhibit continuous-time characteristics, influenced by six-degree-of-freedom motion coupling and nonlinear disturbances. Its evolution is difficult to model and predict directly using standard reinforcement learning frameworks, resulting in a lack of accurate state modeling foundations during policy network training and hindering high-precision control decision-making. Second, most existing deep learning control systems employ offline training methods. During training, the environmental model deviates significantly from real-world sea conditions, lacking the ability to finely represent real-world disturbances and thus failing to achieve online adaptive control and policy transfer. Furthermore, while some existing studies introduce neural networks for platform modeling, they generally employ feedforward structures, failing to fully incorporate the continuous modeling capabilities of system dynamics. This makes it difficult to fit the differential processes of state changes, resulting in significant shortcomings in state prediction and error feedback control.

[0006] In terms of model training, mainstream neural network optimization methods still employ gradient descent-based methods (such as SGD and Adam). These methods are prone to getting trapped in local optima or gradient vanishing in high-dimensional parameter spaces, especially in complex structures such as neural ordinary differential equations. Conventional optimizers struggle to guarantee convergence speed and accuracy. Although some intelligent optimization algorithms, such as genetic algorithms and particle swarm optimization, have been attempted for neural network parameter training, their structural designs are relatively crude, lacking structural guidance for non-convex optimization problems in dynamic ocean systems, making it difficult to balance parameter convergence efficiency and physical modeling accuracy.

[0007] Furthermore, at the joint deployment level, current maritime rocket launch platforms lack a system framework capable of end-to-end joint optimization of state modeling and strategy control modules. Most solutions only complete algorithm verification in simulation environments, lacking an architecture suitable for real-time deployment and closed-loop control. This makes it difficult for existing systems to effectively cope with complex environmental factors such as sudden changes in wave disturbance frequency, rapid changes in direction, and large uncertainties in wave height, ultimately affecting the real-time performance of platform attitude control and the safety of the launch process.

[0008] In summary, existing control technologies for marine rocket launch platforms have significant shortcomings in terms of nonlinear state modeling accuracy, control strategy generalization ability, training method optimization, and closed-loop linkage during platform deployment. These limitations make it difficult to meet the high-precision compensation control requirements for launch platform attitude stability and position holding performance under complex sea conditions. Therefore, there is an urgent need to propose a novel wave compensation control method for marine rocket launch platforms that integrates physical dynamic modeling capabilities with adaptive control strategies, can operate stably in complex disturbance environments, and possesses joint training and deployment capabilities. This method aims to improve the overall intelligence, stability, and practicality of the system.

[0009] Therefore, how to provide a wave compensation control method for marine rocket launch platforms based on deep reinforcement learning is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0010] One objective of this invention is to propose a wave compensation control method for marine rocket launch platforms based on deep reinforcement learning. This invention integrates neural network-based constant differential equation modeling, an improved bat algorithm parameter optimization method, and a Soft Actor-Critic policy control network. By constructing a joint control model with dynamic modeling and adaptive policy generation capabilities, it achieves high-precision closed-loop compensation control of the platform's attitude and position under complex wave disturbance environments. This invention introduces continuous-time modeling concepts into the model structure design and utilizes nonlinear disturbance kernels and attention mechanisms to improve the optimization process, enhancing the system's ability to represent high-dimensional state spaces and adapt to unsteady sea conditions. It possesses advantages such as strong generalization ability, high control accuracy, and strong real-time deployment, making it particularly suitable for marine rocket launch missions with high reliability requirements.

[0011] The wave compensation control method for a marine rocket launch platform based on deep reinforcement learning according to an embodiment of the present invention includes the following steps:

[0012] S1. Collect real-time motion status data of the sea-based rocket launch platform;

[0013] S2. Input the motion state data into the neural network constant differential equation model. The neural network constant differential equation model is constructed based on a neural network. Combine the improved bat algorithm to train the parameter set of the neural network constant differential equation model and output the future state trajectory.

[0014] S3. The future state trajectory is passed as input to the policy network constructed by the SoftActor-Critic algorithm. The policy network generates corresponding continuous control action signals based on the predicted state.

[0015] S4. Input the control action signal to the thruster control system and ballast adjustment system to achieve joint compensation control of the platform attitude and position;

[0016] S5. In the simulation environment, perform end-to-end joint training of the neural network and the constant differential equation model.

[0017] S6. Deploy the trained joint model into the actual platform control system, and use the real-time collected motion state data to complete state prediction and control action output, so as to realize wave compensation of the sea rocket launch platform in the wave disturbance environment.

[0018] Optionally, the real-time motion state data in step S1 specifically includes:

[0019] The platform's three position coordinates in a three-dimensional coordinate system;

[0020] The platform's attitude angles include roll, pitch, and yaw angles;

[0021] The linear velocity of the platform along the three coordinate axes;

[0022] The rate of change of the platform's attitude angles includes three angular velocities;

[0023] Arrange the above twelve state variables according to time After alignment, they are combined into a state vector. ;

[0024] The state vector Real-time data acquisition is performed by an inertial measurement unit, a differential global positioning system module, and a multi-axis attitude sensor.

[0025] Optionally, S2 specifically includes:

[0026] S21. Set the model input variables, including the platform state vector. Control input vector and perturbation input vector The platform state vector The control input vector includes twelve state components: position coordinates, attitude angles, linear velocity, and angular velocity. The disturbance input vector represents the current thrust of the thruster and the adjustment parameters of the ballast tank. This indicates the wave height, period, and direction information under the current sea state conditions;

[0027] S22. Construct a normal differential equation model for the platform, modeling the platform state derivatives. The model form is as follows:

[0028] ;

[0029] in, Let the state derivative prediction function be constructed from a neural network, and the parameter set be... This includes the weight matrices and bias vectors of each layer in the neural network;

[0030] S23. The neural network includes an input layer, three hidden layers, and an output layer. The input is a vector. The output is the state vector. Derivative vectors with consistent dimensions ;

[0031] S24. Construct a training sample set, including consecutive time points. and The state vector difference is used as an approximation of the objective derivative. Construct a loss function and combine it with an improved bat algorithm on the parameter set. The system is trained to fit the state change pattern of the platform under perturbation and output the future state trajectory.

[0032] Optionally, the neural network in the S23 neural ordinary differential equation model includes an input layer, three hidden layers, and an output layer, and its operation steps include:

[0033] In the first hidden layer, independent linear mapping operations are performed on the platform state vector, control input vector and disturbance input vector respectively to extract state sub-features, control sub-features and disturbance sub-features. The three sets of sub-features are concatenated in the channel dimension to generate a joint feature representation.

[0034] In the second hidden layer, the joint feature representation is received, the perturbation input vector is extracted to construct the perturbation attention weight vector, the element-wise weighting operation is performed on the joint feature representation to generate perturbation sensitivity enhancement features, and the original features and weighted results are superimposed using the residual connection structure to output the fused features;

[0035] In the third hidden layer, a learnable structure guidance matrix is ​​loaded, matrix transformation operation is performed on the fused features, the response structure between features of each dimension is adjusted, the transformed result is input into the normalization module, and the final feature vector after structural coupling is output.

[0036] In the output layer, the structural coupling feature vector is received, a linear mapping operation is performed, and a state derivative vector consistent with the platform state dimension is generated as the differential output of the neural constant differential equation modeling.

[0037] Optionally, S24 specifically includes:

[0038] S241, Set the parameter set Represented as a continuous vector, serving as the location code for each individual bat, initially constructing population and individual trajectory memory items. , The calculation method is the first The difference in trajectory changes among individuals;

[0039] S242. Constructing the loss function The model output is compared with the state difference between adjacent time periods. The difference is calculated using the constructed target derivative, and the squared Euclidean distance is used as the error measure. After traversing all samples, the average error is calculated and used as the loss function value to measure the state derivative prediction error of the neural ordinary differential equation model on the training samples.

[0040] S243. Construct a parameter update formula for each individual, consisting of the following structure:

[0041] ;

[0042] in, For the first The number of bat individuals in the first The parameter vector in the next iteration. For the first The number of bat individuals in the first The parameter vector in the next iteration. For perturbation mapping kernels based on Riemann geometry, It is a one-dimensional convolutional frequency coding network. For the convolutional attention mechanism module, For the projection operator under parametric manifold constraints, For mapping functions, For the first The individual in the first Search frequency in each iteration For all The individual in the first The frequency set in the next iteration;

[0043] S244. During the iteration process, the individual parameter vector is continuously updated with the loss function. The goal is to minimize the value and obtain the optimal set of parameters. ;

[0044] S245. Set the optimal parameters It is applied to the neural network of ordinary differential equations, taking the platform state vector, control input vector and disturbance input vector as inputs, and outputting the platform state derivative. The future state trajectory is obtained through numerical integration and used as the input to the control strategy network.

[0045] Optionally, S3 specifically includes:

[0046] S31. Utilizing the trained neural ordinary differential equation model It receives the current state vector, control input vector, and disturbance input vector, and outputs the corresponding predicted state derivative value, which is represented as the rate of change of the state vector with respect to time.

[0047] S32. Input the predicted state derivative value into the numerical integration module, perform integration calculation within the set integration time step range, and obtain the platform at the future target time. The predicted state vector value is denoted as ;

[0048] S33. The predicted future state vector It is provided as input to the policy control network built based on the SoftActor-Critic algorithm. In the process, the policy control network has been reinforced using a joint objective function of policy entropy and state-action value, and the policy parameter set is as follows: ;

[0049] S34. Map the future state vector through the policy control network to generate the continuous control action vector at the current moment. It drives the platform's propulsion system and ballast adjustment system to achieve compensated control of the platform's attitude and position.

[0050] Optionally, the S33 policy control network is trained based on an objective function that integrates a perturbation information reconstruction entropy mechanism and a Bayesian regulation mechanism. The objective function is constructed as follows:

[0051] Using state vectors Action vectors output by the policy Constructed joint dataset As training samples, the set of policy parameters is used during training. To optimize and maximize the policy objective function, the action value function is combined with the reconstruction entropy term modulated with perturbation information, resulting in the policy objective function:

[0052] ;

[0053] in, Represents the policy objective function and the perturbation mapping function. An information entropy adjustment term is constructed using state, action, and disturbance factor as inputs. For the state-action value function in a policy network, For experience expectations, The action probability density function output by the policy network. These are Bayesian regulation factors generated based on state features. This represents the uncertainty density distribution function under the disturbance intensity. To disturb space, It is a logarithmic function.

[0054] Optionally, S5 specifically includes:

[0055] S51, Receive from the policy control network Generated continuous control motion vector Divided into thruster control sub-vectors With ballast adjustment sub-vector ;

[0056] S52. Based on the current platform state vector Future state vector and continuous control motion vector Build a set of control commands This drives the platform's execution layer to complete the corresponding physical control behaviors;

[0057] S53, Set up control commands The data is sent to the thruster execution module and the ballast adjustment module, wherein the thruster execution module receives the control sub-vector. The ballast adjustment module controls the output of the thrusters in each direction and receives the control sub-vector. Adjust the liquid volume in each ballast tank of the control platform;

[0058] S54. After the control command is executed, the data acquisition platform will... Update state vector As a new input feedback to the constant differential equation model and policy network It processes the prediction of generated state derivatives and actions, forming a closed-loop control chain of platform state perception—action generation—control execution—state feedback.

[0059] Optionally, S6 specifically includes:

[0060] S61. The trained neural network constant differential equation model With policy control network Perform joint integration and deploy it to the platform control system;

[0061] S62. During platform operation, the platform's current state vector, control input vector, and disturbance input vector are collected in real time to form a joint input data stream;

[0062] S63. Input the real-time acquired state vector, control input vector, and disturbance input vector into the constant differential equation model. Output the derivative prediction result of the current state of the platform;

[0063] S64. Based on the state derivative prediction results, perform numerical integration to calculate the platform's future state vector, and input the future state vector into the policy control network. Generate the corresponding continuous control action vector ;

[0064] S65, Control motion vector It is applied to the platform execution system to drive the thrusters and ballast system to perform real-time compensation operations, thereby completing the attitude and position stability control of the platform in a wave-turbulent environment.

[0065] The beneficial effects of this invention are:

[0066] (1) This invention uses Neural ODE to model the state evolution of the platform in six degrees of freedom in continuous time. By combining wave disturbance input and control input, it realizes refined modeling of the dynamic response of the nonlinear platform. By constructing an input structure for disturbance characteristics and introducing a state derivative prediction mechanism, it effectively avoids the problems of lag and low accuracy of traditional discrete modeling for state changes, improves the quality of the prediction basis on which the control strategy depends, and provides a highly reliable state prior for the generation of subsequent control actions.

[0067] (2) This invention constructs a control policy network based on the Soft Actor-Critic reinforcement learning algorithm, and introduces a perturbation modulation entropy mechanism and a Bayesian uncertainty control module into the training objective, enabling the policy to dynamically adjust the output distribution according to the complexity of sea conditions. During the model training process, end-to-end joint training is completed through a simulation platform, which enhances the policy network's ability to predict the future state of the platform. Ultimately, it achieves a highly stable control response under typical complex sea conditions such as sudden changes in wave height and periodic non-stationarity, which is significantly better than the rigid structure of traditional rule-based control methods.

[0068] (3) This invention constructs a complete closed-loop architecture with real-time data acquisition, state prediction, control output, and platform execution by jointly deploying the trained neural network with the strategy control network to the platform control system. The system has rapid response and online operation capabilities, can handle state changes under ocean disturbances in real time, realize dynamic compensation for platform attitude and position, improve the safety, stability, and operational accuracy of the marine rocket launch platform, and has significant engineering application and promotion value. Attached Figure Description

[0069] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0070] Figure 1 This is a flowchart of the wave compensation control method for a marine rocket launch platform based on deep reinforcement learning proposed in this invention.

[0071] Figure 2 This is a flowchart of the training process for the Bat Algorithm Optimization Neural ODE in the wave compensation control method for marine rocket launch platforms based on deep reinforcement learning proposed in this invention.

[0072] Figure 3 This is a diagram of the SAC strategy control network structure of the wave compensation control method for marine rocket launch platforms based on deep reinforcement learning proposed in this invention. Detailed Implementation

[0073] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.

[0074] refer to Figure 1-3 A wave compensation control method for marine rocket launch platforms based on deep reinforcement learning includes the following steps:

[0075] S1. Collect real-time motion status data of the sea-based rocket launch platform;

[0076] S2. Input the motion state data into the neural network constant differential equation model. The neural network constant differential equation model is constructed based on a neural network. Combine the improved bat algorithm to train the parameter set of the neural network constant differential equation model and output the future state trajectory.

[0077] S3. The future state trajectory is passed as input to the policy network constructed by the SoftActor-Critic algorithm. The policy network generates corresponding continuous control action signals based on the predicted state.

[0078] S4. Input the control action signal to the thruster control system and ballast adjustment system to achieve joint compensation control of the platform attitude and position;

[0079] S5. In the simulation environment, perform end-to-end joint training of the neural network and the constant differential equation model.

[0080] S6. Deploy the trained joint model into the actual platform control system, and use the real-time collected motion state data to complete state prediction and control action output, so as to realize wave compensation of the sea rocket launch platform in the wave disturbance environment.

[0081] This invention addresses the proposed overall control process by constructing a compensatory control method for a marine rocket launch platform based on a fusion architecture of neural ordinary differential equations and deep reinforcement learning. This effectively solves the problems of insufficient rigidity of control strategies and inadequate modeling accuracy in existing methods under nonlinear, strong disturbance environments. By constructing an end-to-end closed-loop control chain of "state perception—state prediction—control decision—execution feedback," this invention possesses advantages such as autonomous learning capability, strong modeling continuity, and rapid response to sudden disturbances. It significantly improves the attitude and position stability of the marine platform during the launch preparation phase, providing support for precise rocket launches under dynamic sea conditions.

[0082] In this embodiment, the real-time motion state data in step S1 specifically includes:

[0083] The platform's three position coordinates in a three-dimensional coordinate system;

[0084] The platform's attitude angles include roll, pitch, and yaw angles;

[0085] The linear velocity of the platform along the three coordinate axes;

[0086] The rate of change of the platform's attitude angles includes three angular velocities;

[0087] Arrange the above twelve state variables according to time After alignment, they are combined into a state vector. ;

[0088] The state vector Real-time data acquisition is performed by an inertial measurement unit, a differential global positioning system module, and a multi-axis attitude sensor.

[0089] This invention addresses real-time motion state acquisition methods by comprehensively utilizing inertial measurement units, differential GPS, and multi-axis attitude sensors to achieve high-precision acquisition of the platform's six degrees of freedom (DOF) state. Compared to traditional single-mode acquisition that only acquires position or attitude angles, this invention integrates twelve-dimensional state information, including three-dimensional position, three-axis attitude, linear velocity, and angular velocity, into a state vector. This improves the completeness and timeliness of dynamic modeling input, providing a more stable and physically constrained input foundation for subsequent neural ODE modeling, and enhancing the model's sensitivity and accuracy in responding to wave disturbances.

[0090] In this embodiment, S2 specifically includes:

[0091] S21. Set the model input variables, including the platform state vector. Control input vector and perturbation input vector The platform state vector The control input vector includes twelve state components: position coordinates, attitude angles, linear velocity, and angular velocity. The disturbance input vector represents the current thrust of the thruster and the adjustment parameters of the ballast tank. This indicates the wave height, period, and direction information under the current sea state conditions;

[0092] S22. Construct a normal differential equation model for the platform, modeling the platform state derivatives. The model form is as follows:

[0093] ;

[0094] in, Let the state derivative prediction function be constructed from a neural network, and the parameter set be... This includes the weight matrices and bias vectors of each layer in the neural network;

[0095] S23. The neural network includes an input layer, three hidden layers, and an output layer. The input is a vector. The output is the state vector. Derivative vectors with consistent dimensions ;

[0096] S24. Construct a training sample set, including consecutive time points. and The state vector difference is used as an approximation of the objective derivative. Construct a loss function and combine it with an improved bat algorithm on the parameter set. The system is trained to fit the state change pattern of the platform under perturbation and output the future state trajectory.

[0097] This invention addresses the modeling process of neural network-based constant differential equations (RNNs) by fitting the platform's state derivative function using a neural network, effectively overcoming the accuracy bottleneck of traditional discrete-time modeling methods in continuous state prediction. By introducing state, control input, and perturbation vectors as modeling inputs and constructing a physically meaningful state derivative output, differential modeling of the platform's dynamic evolution process is achieved. Compared to traditional RNN or LSTM prediction methods, this invention offers significant advantages in temporal continuity representation and model generalization, making it suitable for state evolution prediction tasks under complex non-steady-state perturbation environments.

[0098] In this embodiment, the neural network in the S23 neural ordinary differential equation model includes an input layer, three hidden layers, and an output layer, and its operation steps include:

[0099] In the first hidden layer, independent linear mapping operations are performed on the platform state vector, control input vector and disturbance input vector respectively to extract state sub-features, control sub-features and disturbance sub-features. The three sets of sub-features are concatenated in the channel dimension to generate a joint feature representation.

[0100] In the second hidden layer, the joint feature representation is received, the perturbation input vector is extracted to construct the perturbation attention weight vector, the element-wise weighting operation is performed on the joint feature representation to generate perturbation sensitivity enhancement features, and the original features and weighted results are superimposed using the residual connection structure to output the fused features;

[0101] In the third hidden layer, a learnable structure guidance matrix is ​​loaded, matrix transformation operation is performed on the fused features, the response structure between features of each dimension is adjusted, the transformed result is input into the normalization module, and the final feature vector after structural coupling is output.

[0102] In the output layer, the structural coupling feature vector is received, a linear mapping operation is performed, and a state derivative vector consistent with the platform state dimension is generated as the differential output of the neural constant differential equation modeling.

[0103] Compared with traditional neural networks, this invention achieves three key structural innovations: improving the ability to identify state, control and disturbance features by decoupling the input channels; introducing a disturbance-driven attention mechanism and residual fusion to enhance the model's dynamic response to sea state changes; and constructing the coupling relationship between state derivatives by combining a learnable structured guiding matrix, which significantly improves the accuracy, robustness and physical consistency of platform dynamics modeling.

[0104] In this embodiment, S24 specifically includes:

[0105] S241, Set the parameter set Represented as a continuous vector, serving as the location code for each individual bat, initially constructing population and individual trajectory memory items. , The calculation method is the first The difference in trajectory changes among individuals;

[0106] S242. Constructing the loss function The model output is compared with the state difference between adjacent time periods. The difference is calculated using the constructed target derivative, and the squared Euclidean distance is used as the error measure. After traversing all samples, the average error is calculated and used as the loss function value to measure the state derivative prediction error of the neural ordinary differential equation model on the training samples.

[0107] S243. Construct a parameter update formula for each individual, consisting of the following structure:

[0108] ;

[0109] in, For the first The number of bat individuals in the first The parameter vector in the next iteration. For the first The number of bat individuals in the first The parameter vector in the next iteration. For perturbation mapping kernels based on Riemann geometry, It is a one-dimensional convolutional frequency coding network. For the convolutional attention mechanism module, For the projection operator under parametric manifold constraints, For mapping functions, For the first The individual in the first Search frequency in each iteration For all The individual in the first The frequency set in the next iteration;

[0110] S244. During the iteration process, the individual parameter vector is continuously updated with the loss function. The goal is to minimize the value and obtain the optimal set of parameters. ;

[0111] S245. Set the optimal parameters It is applied to the neural network of ordinary differential equations, taking the platform state vector, control input vector and disturbance input vector as inputs, and outputting the platform state derivative. The future state trajectory is obtained through numerical integration and used as the input to the control strategy network.

[0112] This invention innovatively introduces a global optimization mechanism based on an improved bat algorithm for the training process of neural ODEs. It combines a perturbation mapping kernel, a frequency convolutional attention module, and a parameter manifold constraint method to effectively improve the expressive power of parameter update paths and the optimization convergence efficiency during model training. Compared to traditional gradient descent optimization methods, which are prone to getting trapped in local optima, the proposed composite structure optimization mechanism possesses stronger search capabilities and perturbation adaptability in the high-dimensional parameter space, thereby improving the model's fitting accuracy and stability in real-world sea state modeling.

[0113] In this embodiment, S3 specifically includes:

[0114] S31. Utilizing the trained neural ordinary differential equation model It receives the current state vector, control input vector, and disturbance input vector, and outputs the corresponding predicted state derivative value, which is represented as the rate of change of the state vector with respect to time.

[0115] S32. Input the predicted state derivative value into the numerical integration module, perform integration calculation within the set integration time step range, and obtain the platform at the future target time. The predicted state vector value is denoted as ;

[0116] S33. The predicted future state vector It is provided as input to the policy control network built based on the SoftActor-Critic algorithm. In the process, the policy control network has been reinforced using a joint objective function of policy entropy and state-action value, and the policy parameter set is as follows: ;

[0117] S34. Map the future state vector through the policy control network to generate the continuous control action vector at the current moment. It drives the platform's propulsion system and ballast adjustment system to achieve compensated control of the platform's attitude and position.

[0118] This invention addresses the future trajectory prediction and control action generation process by collaboratively invoking a neural ODE and a Soft Actor-Critic policy network, achieving tight coupling between platform state prediction and control action generation. Compared to the traditional "single-step prediction + rule-based control" structure, this invention directly guides the policy network to generate optimal continuous control actions based on the predicted trajectory, effectively reducing the accumulation of intermediate state errors, improving the overall control closed-loop accuracy and policy adaptability, and making it suitable for real-world launch environments with rapidly changing sea states and large disturbance amplitudes.

[0119] In this embodiment, the S33 policy control network is trained based on an objective function that integrates a perturbation information reconstruction entropy mechanism and a Bayesian regulation mechanism. The objective function is constructed as follows:

[0120] Using state vectors Action vectors output by the policy Constructed joint dataset As training samples, the set of policy parameters is used during training. To optimize and maximize the policy objective function, the action value function is combined with the reconstruction entropy term modulated with perturbation information, resulting in the policy objective function:

[0121] ;

[0122] in, Represents the policy objective function and the perturbation mapping function. An information entropy adjustment term is constructed using state, action, and disturbance factor as inputs. For the state-action value function in a policy network, For experience expectations, The action probability density function output by the policy network. These are Bayesian regulation factors generated based on state features. This represents the uncertainty density distribution function under the disturbance intensity. To disturb space, It is a logarithmic function.

[0123] This invention addresses the proposed training objective of the control strategy network by designing a variant SAC structure that integrates a perturbation modulation entropy mechanism and a Bayesian control factor. While preserving the robustness of the maximum entropy strategy, it introduces a strategy entropy weighting mechanism for perturbation-sensitive regions, effectively enhancing the network's ability to focus on control strategies in areas with high uncertainty in sea states. This mechanism strengthens the stability and perturbation adaptability of the control strategy during training, enabling a dynamic balance between control strength and response speed in the output actions, thereby improving the safety and controllability of the platform operation.

[0124] In this embodiment, S5 specifically includes:

[0125] S51, Receive from the policy control network Generated continuous control motion vector Divided into thruster control sub-vectors With ballast adjustment sub-vector ;

[0126] S52. Based on the current platform state vector Future state vector and continuous control motion vector Build a set of control commands This drives the platform's execution layer to complete the corresponding physical control behaviors;

[0127] S53, Set up control commands The data is sent to the thruster execution module and the ballast adjustment module, wherein the thruster execution module receives the control sub-vector. The ballast adjustment module controls the output of the thrusters in each direction and receives the control sub-vector. Adjust the liquid volume in each ballast tank of the control platform;

[0128] S54. After the control command is executed, the data acquisition platform will... Update state vector As a new input feedback to the constant differential equation model and policy network It processes the prediction of generated state derivatives and actions, forming a closed-loop control chain of platform state perception—action generation—control execution—state feedback.

[0129] This invention addresses the execution and feedback mechanism of control actions by constructing a joint control command generation structure for the thruster control and ballast adjustment systems, thereby enhancing the analytical capability of the control strategy in scenarios with multiple control variables. By dividing the control action vector into specific executable module instructions and providing real-time feedback on the platform status, a complete closed-loop control chain of "action generation—physical execution—state update" is constructed, effectively improving the system's real-time response capability and execution accuracy, and enhancing the platform's self-stability and attitude maintenance capability under dynamic disturbances.

[0130] In this embodiment, S6 specifically includes:

[0131] S61. The trained neural network constant differential equation model With policy control network Perform joint integration and deploy it to the platform control system;

[0132] S62. During platform operation, the platform's current state vector, control input vector, and disturbance input vector are collected in real time to form a joint input data stream;

[0133] S63. Input the real-time acquired state vector, control input vector, and disturbance input vector into the constant differential equation model. Output the derivative prediction result of the current state of the platform;

[0134] S64. Based on the state derivative prediction results, perform numerical integration to calculate the platform's future state vector, and input the future state vector into the policy control network. Generate the corresponding continuous control action vector ;

[0135] S65, Control motion vector It is applied to the platform execution system to drive the thrusters and ballast system to perform real-time compensation operations, thereby completing the attitude and position stability control of the platform in a wave-turbulent environment.

[0136] This invention proposes a joint online model deployment structure for the deployment of training models and the operation of platform closed-loop control. It integrates neural ODE modeling and the SAC policy network into a deployable modular subsystem, which is embedded in the actual platform control system. By acquiring state and disturbance inputs in real time, it achieves rapid inference of state derivative prediction and policy action output, thereby completing closed-loop compensation control in the operational environment. This invention improves the transferability of the algorithm from offline verification to practical deployment, providing engineering-feasible technical support for actual maritime launch missions.

[0137] Example 1:

[0138] To verify the feasibility of this invention in practice, it was applied to a launch preparation control test mission conducted in August 2024 in a wave-turbulent environment in a certain sea area outside my country using a certain type of marine rocket launch platform. This test mission was based on a floating platform equipped with a multi-point positioning and mooring system, and carrying six sets of directional electric thrusters and four sets of adjustable ballast tanks. The platform's original control system used a traditional PID controller, which had a slow response speed to attitude control and was prone to attitude drift in high-wave conditions, making it difficult to meet the attitude accuracy requirements before vertical rocket launch.

[0139] To address the aforementioned issues, without altering the platform's hardware structure, a joint control system based on the neural network constant differential equations and the Soft Actor-Critic algorithm, as proposed in this invention, was deployed. This system was integrated with the platform's existing inertial navigation unit, differential GPS positioning device, and multi-axis attitude sensors to achieve real-time acquisition of the twelve-dimensional state vector. The acquired data includes the platform's three-dimensional position coordinates, attitude angles, three-axis velocities, and three-axis angular velocities, with a sampling frequency set to 10Hz.

[0140] During deployment, 3000 training samples were first generated based on the platform's historical motion data and simulated wave conditions to train the neural network's ordinary differential equation model. The model inputs included the platform's current state vector, control inputs (thrust and ballast adjustment values), and wave disturbance characteristics (wave height, period, and direction), with the output being the platform's state derivative. An improved bat algorithm was used to iteratively train the network parameters for 30 rounds, minimizing the platform state prediction error, and the optimal parameter set was obtained by combining simulation results. Subsequently, a Soft Actor-Critic policy control network was constructed, and policy optimization was performed by maximizing the desired state-action value function and the weighted information entropy objective, using the predicted state as input. The joint model underwent end-to-end training in both MATLAB-Simulink and ROS-Gazebo environments.

[0141] After deployment, a 6-hour continuous test was conducted in the actual operating environment of the platform. The test sea state wave height was between 0.5 and 1.8 meters, with a period variation range of 4 to 8 seconds, and the wave direction was mainly due east south of 30°. During the test, the platform's attitude stability, position holding accuracy, control response time, and attitude drift amplitude were sampled and recorded every 2 minutes, and the results were compared with the operating results of the platform's original control system.

[0142] Table 1: Comparison of Platform Control Performance (Traditional Methods vs. This Invention)

[0143] Test metrics The system mean of this invention Traditional control system mean Maximum value of the present invention Traditional maximum value Root mean square error of platform roll angle (°) 0.52 1.44 0.81 2.26 Root mean square error of platform pitch angle (°) 0.47 1.31 0.76 1.92 Platform position offset (m) 0.26 0.89 0.41 1.34 Platform attitude drift angle (°) 0.41 1.87 0.62 2.35 Average delay time of control response (s) 0.32 0.78 0.49 1.14 Average error rate of state prediction (%) 3.7 / 6.2 / Number of times the control system experiences oscillations or overshoots. 0 5 / / Online prediction frequency of the model (Hz) 10 not applicable 10 not applicable

[0144] From the perspective of control accuracy, the system of this invention significantly outperforms traditional control systems in maintaining the accuracy of platform roll and pitch angles. Specifically, the root mean square error of the platform roll angle decreases from 1.44° to 0.52°, and the maximum error decreases from 2.26° to 0.81°; the pitch angle error decreases from 1.31° to 0.47°, and the maximum value also decreases from 1.92° to 0.76°. This indicates that the control actions of this invention, based on the collaborative output of neural network-based modeling using ordinary differential equations, can more accurately counteract attitude changes caused by wave disturbances and significantly improve attitude stability under high-frequency disturbances.

[0145] Regarding platform position control, the control system of this invention reduces the average platform position offset from 0.89 meters in traditional control systems to 0.26 meters, and the maximum offset from 1.34 meters to 0.41 meters. Especially in sea conditions with rapidly changing wave cycles, the control system can adjust the thrusters and ballast adjustment system in a timely manner according to the predicted trajectory, reducing the risk of platform drift and providing more stable launch attitude support for rocket launch.

[0146] In terms of response speed, the average response delay of continuous control actions output by the policy network of this invention through the Soft Actor-Critic algorithm is only 0.32 seconds, while that of the traditional method is 0.78 seconds, and the maximum delay is further reduced from 1.14 seconds to 0.49 seconds. This difference demonstrates that the system of this invention has higher action generation efficiency and control immediacy, stronger ability to adapt to complex dynamic disturbances, and effectively ensures the rapid closed-loop stability of the control chain.

[0147] Regarding control robustness, the traditional control system exhibited five control oscillations or overshoots during the test, while the control system of this invention did not experience similar phenomena during the 6-hour test cycle. Meanwhile, the average error rate of the state prediction of the model of this invention was only 3.7%, and the maximum error rate did not exceed 6.2%, and it maintained stable inference at a frequency of 10Hz. This indicates that the neural ODE model can continuously and stably produce state derivatives that conform to physical laws, providing a reliable basis for policy decisions.

[0148] In summary, by integrating frequent differential modeling, improving the bat optimization mechanism, and the adaptive strategy control network, the control method of this invention significantly improves the platform attitude stability, response speed, and environmental adaptability compared to existing technologies, demonstrating strong engineering practicality and promotional value. It is especially suitable for deployment on sea-based rocket launch platforms with significant wave interference and extremely high control accuracy requirements.

[0149] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A wave compensation control method for a sea-based rocket launch platform based on deep reinforcement learning, characterized in that, The method comprises the following steps: S1, collecting real-time motion state data of a sea rocket launch platform; S2, inputting the motion state data into a neural ordinary differential equation model, the neural ordinary differential equation model being constructed based on a neural network, and a parameter set of the neural ordinary differential equation model being trained by combining an improved bat algorithm, and outputting a future state trajectory; S3, inputting the future state trajectory into a policy network constructed by a SoftActor-Critic algorithm as input, and generating a corresponding continuous control action signal according to the predicted state by the policy network; S4, inputting the control action signal into a propeller control system and a ballast adjustment system to realize joint compensation control of the platform attitude and position; S5, performing end-to-end joint training of the neural ordinary differential equation model and the policy network in a simulation environment; S6, deploying the trained joint model to an actual platform control system, and completing state prediction and control action output by using real-time collected motion state data to realize wave compensation of the sea rocket launch platform in a wave disturbance environment.

2. The deep reinforcement learning-based offshore rocket launch platform wave compensation control method according to claim 1, characterized in that, The real-time motion state data of the step S1 specifically comprises: three position coordinates of the platform in a three-dimensional coordinate system; attitude angles of the platform, including a roll angle, a pitch angle and a yaw angle; linear velocities of the platform in three coordinate axis directions; rates of change of the attitude angles of the platform, including three angular velocities; The twelve state quantities above are combined into a state vector after alignment ; The state vector Real-time acquisition by inertial measurement unit, differential global positioning system module and multi-axis attitude sensor.

3. The deep reinforcement learning-based offshore rocket launch platform wave compensation control method according to claim 1, characterized in that, The S2 specifically comprises: S21, set model input variables, including platform state vector , control input vector and disturbance input vector , the platform state vector includes twelve state components of position coordinates, attitude angles, linear velocities and angular velocities, the control input vector represents the current thruster thrust and ballast tank adjustment parameters, the disturbance input vector represents the wave height, period and direction information under the current sea condition; S22, constructing a neural ordinary differential equation model to model derivatives of the platform state, and the model form is: ； wherein, is a state derivative prediction function constructed from a neural network, a parameter set comprising weight matrices and bias vectors of the layers of the neural network; S23, the neural network comprises an input layer, three hidden layers and an output layer, the input is a vector , the output is a derivative vector consistent with the state vector dimension ; S24, construct a training sample set, take the state vector difference of the continuous time points With as the target derivative approximation value , construct a loss function, train the parameter set combined with the improved bat algorithm , fit the state change rule of the platform under the disturbance, and output the future state trajectory.

4. The deep reinforcement learning-based offshore rocket launch platform wave compensation control method according to claim 3, characterized in that, The neural network in the S23 neural ordinary differential equation model comprises an input layer, three hidden layers and an output layer, and the operation steps comprise: in the first hidden layer, independent linear mapping operations are respectively performed on a platform state vector, a control input vector and a disturbance input vector to extract state sub-features, control sub-features and disturbance sub-features, the three groups of sub-features are spliced in a channel dimension to generate a joint feature representation; in the second hidden layer, the joint feature representation is received, a disturbance attention weight vector is constructed by using the disturbance input vector, an element-wise weighting operation is performed on the joint feature representation to generate a disturbance sensitivity enhanced feature, and a residual connection structure is used to superimpose the original feature and the weighting result to output a fusion feature; in the third hidden layer, a learnable structure guide matrix is loaded to perform a matrix transformation operation on the fusion feature to adjust the response structure between the features in each dimension, and the transformed result is input into a normalization module to output a final feature vector after structure coupling; in the output layer, the structure-coupled feature vector is received to perform a linear mapping operation to generate a state derivative vector consistent with the platform state dimension as the differential output of the neural ordinary differential equation modeling.

5. The deep reinforcement learning-based offshore rocket launch platform wave compensation control method according to claim 3, characterized in that, The S24 specifically comprises: S241, set of parameters are represented in continuous vector form as position encodings of each bat individual, initial population and individual trajectory memory terms , is computed as the difference in trajectory change of the first individuals S242、constructing a loss function The model output result is compared with the difference value of the adjacent time state difference The difference is calculated by the target derivative constructed, the square of the Euclidean distance is used as the error measurement, the average error is calculated after traversing all samples, and the loss function value is used to measure the state derivative prediction error of the neural ordinary differential equation model on the training sample; S243, constructing a parameter update formula composed of the following structures for each individual: ； in, For the first The number of bat individuals in the first The parameter vector in the next iteration. For the first The number of bat individuals in the first The parameter vector in the next iteration. For perturbation mapping kernels based on Riemann geometry, It is a one-dimensional convolutional frequency coding network. For the convolutional attention mechanism module, For the projection operator under parametric manifold constraints, For mapping functions, For the first The individual in the first Search frequency in each iteration For all The individual in the first The frequency set in the next iteration; S244, updating the individual parameter vector constantly in the iteration process, taking the loss function as the target, obtaining the optimal parameter set ; S245. Set the optimal parameters It is applied to the neural network of ordinary differential equations, taking the platform state vector, control input vector and disturbance input vector as inputs, and outputting the platform state derivative. The future state trajectory is obtained through numerical integration and used as the input to the control strategy network.

6. The deep reinforcement learning-based offshore rocket launch platform wave compensation control method according to claim 1, wherein, The S3 specifically comprises: S31, utilizing the trained neural ordinary differential equation model , receiving a state vector, a control input vector and a disturbance input vector at a current time, and outputting a corresponding state derivative prediction value, denoted as a rate of change of the state vector with respect to time; S32, input the state derivative prediction value into a numerical integration module, perform integration calculation in a set integration time step range, obtain a state vector prediction value of the platform at a future target time, denoted as . ​ S33, the predicted future state vector as input to a policy control network constructed based on a SoftActor-Critic algorithm wherein the policy control network has been reinforced trained by a policy entropy and state-action value joint objective function, the set of policy parameters being ; S34, mapping the future state vector through the policy control network to generate a continuous control action vector at the current time The driving platform propulsion system and the ballast adjustment system are driven to realize compensation control on the platform posture and position.

7. The deep reinforcement learning-based offshore rocket launch platform wave compensation control method according to claim 6, characterized in that, The S33 policy control network is trained based on a target function that combines a disturbance information fusion entropy mechanism and a Bayesian regulation mechanism, and the target function is constructed as follows: Utilizing state vectors action vectors output by the policy constructed joint dataset As training samples, the policy parameter set is optimized during the training process maximizing the policy objective function, which combines the action value function with a reconstructed entropy term modulated with perturbation information: ； where, denotes the policy objective function, the perturbation mapping function builds the information entropy adjustment term with state, action and perturbation factor as inputs, is the state-action value function under the policy network, is the experience expectation, is the action probability density function output by the policy network, is the Bayesian regulatory factor generated based on the state feature, denotes the uncertainty density distribution function under the perturbation intensity, is the perturbation space, is the logarithmic function.

8. The deep reinforcement learning-based offshore rocket launch platform wave compensation control method according to claim 1, wherein, The S5 specifically comprises: S51, receiving by the policy control network generated continuous control action vector , divided into thruster control sub-vectors and ballast adjustment sub-vectors ; S52. Based on the current platform state vector Future state vector and continuous control motion vector Build a set of control commands This drives the platform's execution layer to complete the corresponding physical control behaviors; S53、the control instruction set is sent to a thruster execution module and a ballast adjustment module, wherein the thruster execution module receives control sub-vectors , controlling the output of thrusters in each direction, and the ballast adjustment module receives control sub-vectors , controlling the amount of liquid adjustment in each ballast tank of the platform; S54, after the control instruction is executed, the collection platform feeds back the state vector of the platform at the time point to the neural differential equation model as a new input and the policy network , to process to generate the state derivative prediction and the action, to form the closed-loop control chain of the platform state perception-action generation-control execution-state feedback.

9. The deep reinforcement learning-based offshore rocket launch platform wave compensation control method according to claim 1, wherein, The S6 specifically comprises: S61、the trained neural ordinary differential equation model with policy control network jointly integrated, deployed into a platform control system; S62, in the platform running process, the state vector of the platform current time, the control input vector and the disturbance input vector are collected in real time to form a joint input data stream; S63, input the real-time collected state vector, control input vector and disturbance input vector into the neural differential equation model , output the prediction result of the current state derivative of the platform S64, numerical integration is performed based on the state derivative prediction result to calculate a future state vector of the platform, and the future state vector is input to the policy control network to generate a corresponding continuous control action vector ; S65, the control action vector The application is applied to a platform execution system, drives the thruster and the ballast system to perform real-time compensation operation, and completes the attitude and position stability control of the platform in a wave disturbance environment.

Images