Ship motion trail optimization control method based on deep learning

Abstract

The invention provides a deep learning-based ship motion trajectory optimization control method, which relates to the technical field of ship control, and comprises the following steps: constructing a ship motion simulation environment, and establishing a dynamic simulation system containing marine environment parameters; acquiring ship operation data; a dual-network structure model is constructed, the dual-network structure model comprises a macroscopic strategy network and a microscopic control network, the macroscopic strategy network generates a course planning strategy at a long time interval, and the microscopic control network generates a specific control instruction at a short time interval; a hierarchical training strategy is adopted to train the dual-network structure model in a ship motion simulation environment, firstly, a macroscopic strategy network is trained to generate an overall course strategy, and then a microscopic control network is trained to realize accurate control; and deploying the trained dual-network structure model to a ship control system, generating an optimal control instruction, and realizing real-time optimization control of the ship motion trail. According to the invention, on the premise of ensuring the control precision, the real-time optimization control of the ship motion trail is realized.

Description

Technical Field

[0001] The present invention relates to the field of ship control technology, and in particular to a ship motion trajectory optimization control method based on deep learning. Background Art

[0002] The optimal control of the ship's motion trajectory is a key technology for maritime navigation safety and efficiency, and is of great significance for improving shipping efficiency, reducing energy consumption and ensuring navigation safety. During navigation, ships face a complex and changeable marine environment, including the influence of various environmental factors such as wind, wave force, and water current. At the same time, it is also necessary to consider the ship's own motion characteristics, maneuverability and other aspects. Therefore, how to achieve precise control of the ship's motion trajectory under complex sea conditions is an important research topic in the shipping field.

[0003] At present, ship motion trajectory optimization control mainly adopts traditional PID control, model predictive control and other methods, which usually design control strategies based on simplified ship motion models. With the development of artificial intelligence technology, some studies have begun to try to apply deep learning methods to ship trajectory control, by building neural network models to learn control strategies, or using reinforcement learning methods to optimize control decisions.

[0004] However, the existing control methods still have many shortcomings in practical applications: first, traditional control methods are difficult to cope with complex and changeable marine environments, and control performance is easily affected by external interference; second, methods based on a single control strategy are difficult to take into account both the integrity of trajectory planning and the accuracy of control instructions; third, existing deep learning methods have limitations in dealing with the coordinated optimization of long-term prediction and short-term control, and lack the ability to adapt to extreme sea conditions. These problems seriously restrict the practical application of ship motion trajectory optimization control technology. Summary of the invention

[0005] In view of this, the present invention proposes a ship motion trajectory optimization control method based on deep learning. By constructing a dual network structure model and a hierarchical training strategy, the coordinated optimization of long-term planning and short-term control of the ship motion trajectory is realized, and the control system's ability to adapt to complex and changeable sea conditions is improved. Therefore, real-time optimization control of the ship motion trajectory is achieved while ensuring control accuracy.

[0006] The technical solution of the present invention is achieved in this way:

[0007] The present invention provides a ship motion trajectory optimization control method based on deep learning, comprising:

[0008] S1. Construct a ship motion simulation environment and establish a dynamic simulation system including marine environment parameters; collect ship operation data, wherein the operation data includes ship state parameters and marine environment parameters, wherein the ship state parameters are obtained through a ship-borne sensor system, and the marine environment parameters are obtained through an environmental monitoring system;

[0009] S2. Construct a dual network structure model, including a macro strategy network and a micro control network, wherein the macro strategy network generates a heading planning strategy at a longer time interval, and the micro control network generates a specific control instruction at a shorter time interval; the input of the dual network structure model includes ship state parameters and marine environment parameters, and the two networks realize information interaction through a temporal attention mechanism;

[0010] S3. A hierarchical training strategy is adopted to train the dual network structure model in a ship motion simulation environment. First, the macro strategy network is trained to generate the overall heading strategy, and then the micro control network is trained to achieve precise control. The macro strategy network is optimized based on long-term returns, and the micro control network is optimized based on short-term control accuracy. The two networks maintain consistency through a gradient alignment mechanism.

[0011] S4. Deploy the trained dual network structure model to the ship control system, implement predictive control through the sliding time window method, receive ship state parameters and ocean environment parameters in real time, dynamically adjust the control strategy based on the predicted environmental changes, generate optimal control instructions, and realize real-time optimization control of the ship's motion trajectory.

[0012] Preferably, the ship status parameters include: the ship's position coordinates, heading angle, speed, pitch angle, roll angle, yaw angle, bow angular velocity, lateral speed and longitudinal speed; the marine environment parameters include: wind speed, wind direction, wave height, wave direction, water flow speed, water flow direction, water depth and sea temperature.

[0013] Preferably, in step S1, constructing a ship motion simulation environment includes:

[0014] Establish a three-dimensional ship model, including the hull geometry model and mass distribution model;

[0015] Construct a hydrodynamics module to calculate various hydrodynamic forces acting on ships;

[0016] Set up the propulsion system model, including the main thruster and steering gear system;

[0017] Establish environmental force models to simulate the effects of wind, wave and current forces;

[0018] The dynamic simulation system is integrated into the ship motion simulation environment, including:

[0019] Construct an ocean environment parameter generation module to generate time-varying environment parameters;

[0020] Establish a solver for the ship motion equations to calculate the motion response of the ship in complex sea conditions;

[0021] Set up a sensor data simulation module to simulate the data collection process of the shipborne sensor system and environmental monitoring system;

[0022] Establish a data storage and analysis module to record and process various types of data during the simulation process.

[0023] Preferably, the dual network structure model includes:

[0024] Macro Strategy Network:

[0025] An input layer, a first input sublayer and a second input sublayer for receiving ship state parameters and ocean environment parameters respectively;

[0026] The temporal coding layer is a temporal coding module set after each input sublayer to perform feature marking on input signals with long time intervals;

[0027] The feature extraction layer combines a bidirectional long short-term memory network with a one-dimensional convolutional network to process the input ship state parameters and ocean environment parameters in parallel and extract time series features respectively;

[0028] The feature fusion layer fuses the features extracted by the bidirectional long short-term memory network and the one-dimensional convolutional network through a gating mechanism to form a comprehensive feature representation;

[0029] The strategy generation layer connects two layers of fully connected networks. Each layer of the network uses a learnable residual connection and the activation function is Swish, which is used to generate a heading planning strategy vector.

[0030] Micro-control network:

[0031] The input layer sets a multi-channel convolution input structure for data with shorter time intervals, mapping the high-frequency sampled ship state parameters and ocean environment parameters into a multi-dimensional feature space;

[0032] The multi-scale convolution layer combines dilated convolution with variable kernel convolution to capture the temporal features of the micro-control stage at multiple scales.

[0033] The attention enhancement layer uses a multi-head causal attention mechanism to assign feature weights of different historical moments to the control output at the current moment;

[0034] Control the output layer, connect a double-branch fully connected network, one branch is used to output the rudder angle distribution, the other branch is used to output the propulsion force distribution, and the activation function is Mish;

[0035] Cross-network information interaction mechanism:

[0036] The temporal interactive attention module is set between the strategy generation layer of the macro strategy network and the input layer of the micro control network to achieve dynamic alignment of information between the output of the macro strategy network and the input of the micro control network.

[0037] Preferably, step S3 includes:

[0038] S31. Construct comprehensive evaluation function R t , based on the comprehensive evaluation function R t Define the long-term return R long-term and the short-term immediate reward R short-term ;

[0039] S32, define the state space S and action space A of the ship's motion, the state vector s t ∈S is defined as:

[0040]

[0041] In the formula, (x t ,y t ) is the position coordinate of the ship in the global coordinate system; θ t is the heading angle; v t is the ship speed; pit is the pitch angle; φ roll is the roll angle; φ yaw The head swing angle; ω yaw is the yaw angular velocity; v lat is the lateral speed; v long is the longitudinal velocity; v wind is the wind speed; θ wind is the wind direction; h wave is the wave height; θ wave is the wave direction; v cur is the water flow velocity; θ cur is the water flow direction; d depth is the water depth; T water is the sea temperature; (x tar ,y tar ) is the target position coordinate; φ task The current task stage;

[0042] Action vector a t ∈A is defined as:

[0043] a t =[n t ,δ t ] T

[0044] Where n tis the propeller speed ratio, which represents the ratio of the desired propeller speed to the rated speed; δ t is the rudder angle;

[0045] S33, first train the macro strategy network, based on the long-term return R long-term Iterate and optimize the course planning strategy; use a multi-batch experience playback mechanism to screen key training samples covering a variety of sea conditions;

[0046] S34, after the macro strategy network initially converges, switch to the training phase of the micro control network; based on the short-term immediate reward R short-term , conduct reinforcement learning iterations of the micro-control network to generate specific control strategies;

[0047] S35. Through dynamic task allocation, different environmental complexity scenarios are introduced to make the macro strategy network and micro control network adapt to different sea conditions respectively;

[0048] S36. A gradient alignment module is set up during training to monitor the policy differences between the macro policy network and the micro control network; the parameters of the macro policy network and the micro control network are adjusted synchronously through the gradient alignment mechanism;

[0049] S37. After the macro-strategy network and the micro-control network are trained to the preset convergence standard, joint fine-tuning is performed. During the joint fine-tuning process, the macro-strategy network and the micro-control network are simultaneously optimized by minimizing the comprehensive loss function. When the fine-tuning is completed, a trained dual-network structure model is obtained.

[0050] Preferably, the calculation formula of the comprehensive evaluation function is as follows:

[0051] R t =w1·f traj (s t )+w2·f energy (a t )+w3·f smooth (s t ,a t )

[0052] In the formula, R t is the reward value at time step t; f traj (s t ) indicates state s t The trajectory tracking error at time ; f energy (a t ) means taking action a t Energy consumption when smooth (s t ,a t ) indicates state s t Take action a tThe smoothness of the control command when ; w1, w2, w3 are dynamic weight coefficients;

[0053] Long-term return R long-term The calculation formula is as follows:

[0054]

[0055] In the formula, ρ t is the timing discount factor; g(H t ) is the sea condition adaptation function, according to the current sea condition H t Adjust the intensity of rewards;

[0056] Short-term immediate reward R short-term The calculation formula is: R short-term =R t , that is, the short-term immediate reward directly uses the reward value calculated by the comprehensive evaluation function at the current time step t.

[0057] Preferably, in step S36, the gradient alignment mechanism is as follows:

[0058] For the macro strategy network and the micro control network, the loss function L is defined respectively macro and L micro ;

[0059] Calculate the respective gradient vector and Among them, θ macro and θ micro denote the parameter sets of the macro strategy network and the micro control network respectively;

[0060] Using the gradient alignment formula, the gradient vectors of the two networks are aligned and the network parameters are adjusted synchronously. The gradient alignment formula is as follows:

[0061]

[0062] Where, L align is the gradient alignment loss, λ is the trade-off coefficient, η(t) is the time-varying adjustment factor, and ‖·‖ represents the Euclidean norm.

[0063] Preferably, the comprehensive loss function is as follows:

[0064] L total =L macro +L micro +α·L align

[0065] L macro =L policy +β1·L value +β2·L entropy

[0066]

[0067]

[0068] L micro =L TD +μ1·L smooth +μ2·L constraint

[0069]

[0070] L smooth =‖a t -a t-1 ‖ 2

[0071] L constraint =max(0,‖a t ‖-a max ) 2

[0072] Where, L total is the comprehensive loss function, L macro is the macro strategy network loss function, L micro is the micro-control network loss function, α is the trade-off coefficient, L align is the gradient alignment loss; L policy is the strategy loss, L value is the value function loss, L entropy is the entropy regularization loss, β1 and β2 are weight coefficients; π θ is the parameterized policy function, π θ (a t |s t ) means that in a given state s t Take action a t The probability of θ (a|s t ) means that in a given state s t The probability distribution of taking any action a under t =R long-term -V(s t ) is the advantage function, V(s t ) is the state value function, In the strategy π θ , ‖·‖ represents the Euclidean norm; L TD is the timing difference loss, L smooth is the smoothing loss, L constraint is the constraint loss, μ1 and μ2 are weight coefficients; Q is the action value function, Q(s t ,a t) means that in a given state s t Take action a t The expected return that can be obtained, Q(s t+1 ,a t+1 ) means that in a given state s t+1 Take action a t+1 The expected return that can be obtained, γ is the discount factor; a t 、a t-1 are the action instructions for the current time step and the previous time step respectively; ‖a t ‖ indicates action instruction a t The modulus length, i.e. a t The Euclidean norm of max is the maximum allowed modulus of the action space.

[0073] Preferably, the following attention calculation formula is used in the temporal interaction attention module:

[0074]

[0075] Where Q, K, and V are query, key, and value vectors respectively; d k is the dimension of the key vector; M(t) is the temporal mask function; G(ΔT) is the time interval adaptation function, which dynamically adjusts the attention intensity according to the predicted time span.

[0076] Preferably, step S4 comprises:

[0077] S41, loading the trained dual network structure model into the ship control system, initializing the model parameters and setting the operating environment;

[0078] S42, receiving ship status parameters and ocean environment parameters in real time through a sliding time window method, wherein the sliding time window method includes dynamic updating of historical data and current data;

[0079] S43, based on the received ship state parameters and ocean environment parameters, using the macro strategy network to generate a heading planning strategy;

[0080] S44, generating specific control instructions including rudder angle distribution and propulsion force distribution according to the heading planning strategy generated by the macro strategy network and in combination with the micro control network;

[0081] S45. Monitor the ship's motion state and environmental changes in real time, dynamically adjust the control strategy based on the predicted environmental changes, generate optimal control instructions, and achieve real-time optimization control of the ship's motion trajectory.

[0082] Compared with the prior art, the present invention has the following beneficial effects:

[0083] (1) The present invention realizes real-time optimization control of the ship's motion trajectory by constructing a dual network structure model and a hierarchical training strategy. This method can dynamically adjust the control strategy in a complex and changeable marine environment, improve the safety and economy of ship navigation, and enhance the overall performance and response speed of the ship control system; (2) The dual network structure combining a macro strategy network and a micro control network is adopted to effectively distinguish between the generation of long-term heading planning and short-term control instructions. The macro strategy network is responsible for generating the overall heading planning strategy to ensure that the ship's navigation direction is reasonable within a large range; the micro control network generates specific rudder angles and propulsion force instructions based on real-time status and environmental parameters, thereby achieving precise control of the ship's motion. This structural design enhances the coordination ability of the control system at different time scales, improves the accuracy of trajectory planning and the execution accuracy of control instructions;

[0084] (3) Through the hierarchical training strategy, the macro policy network and the micro control network are trained successively to ensure that each network can independently improve its performance under its own optimization goal. At the same time, the gradient alignment mechanism is introduced to maintain the parameter consistency between the two networks and avoid policy deviations caused by different training orders or optimization goals. This strategy effectively improves the stability and consistency of the collaborative work of the two networks, ensuring that the entire control system can coordinate and improve performance during the optimization process;

[0085] (4) The designed comprehensive evaluation function comprehensively considers the trajectory tracking error, energy consumption and control command smoothness. Through the adaptive adjustment of the dynamic weight coefficient, the dynamic balance of multi-objective optimization is achieved, so that the control system can take into account energy efficiency and control stability while ensuring trajectory accuracy.

[0086] (5) The temporal interactive attention module is introduced to achieve dynamic information alignment between the output of the macro strategy network and the input of the micro control network through the temporal mask function and the time interval adaptation function, thereby improving the real-time and accuracy of the dual network collaborative control. BRIEF DESCRIPTION OF THE DRAWINGS

[0087] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0088] Figure 1 is a flow chart of the method of the present invention;

[0089] Figure 2 It is a framework diagram of the dual network structure model of the present invention;

[0090] Figure 3 This is a diagram of the model training process of the present invention DETAILED DESCRIPTION

[0091] The following will be combined with the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0092] like Figure 1 As shown, the present invention provides a ship motion trajectory optimization control method based on deep learning, comprising:

[0093] S1. Construct a ship motion simulation environment and establish a dynamic simulation system including marine environment parameters; collect ship operation data, wherein the operation data includes ship state parameters and marine environment parameters, wherein the ship state parameters are obtained through a ship-borne sensor system, and the marine environment parameters are obtained through an environmental monitoring system;

[0094] S2. Construct a dual network structure model, including a macro strategy network and a micro control network, wherein the macro strategy network generates a heading planning strategy at a longer time interval, and the micro control network generates a specific control instruction at a shorter time interval; the input of the dual network structure model includes ship state parameters and marine environment parameters, and the two networks realize information interaction through a temporal attention mechanism;

[0095] S3. A hierarchical training strategy is adopted to train the dual network structure model in a ship motion simulation environment. First, the macro strategy network is trained to generate the overall heading strategy, and then the micro control network is trained to achieve precise control. The macro strategy network is optimized based on long-term returns, and the micro control network is optimized based on short-term control accuracy. The two networks maintain consistency through a gradient alignment mechanism.

[0096] S4. Deploy the trained dual network structure model to the ship control system, implement predictive control through the sliding time window method, receive ship state parameters and ocean environment parameters in real time, dynamically adjust the control strategy based on the predicted environmental changes, generate optimal control instructions, and realize real-time optimization control of the ship's motion trajectory.

[0097] In one embodiment of the present invention, the ship state parameters include: the ship's position coordinates, heading angle, speed, pitch angle, roll angle, yaw angle, bow angular velocity, lateral speed and longitudinal speed; the ocean environment parameters include: wind speed, wind direction, wave height, wave direction, water flow speed, water flow direction, water depth and sea temperature. Step S1 includes:

[0098] Building a ship motion simulation environment:

[0099] Establish a three-dimensional ship model: Use the NURBS surface modeling method to construct the hull geometry model to describe the hull shape characteristics; establish a mass distribution model based on the finite element meshing method to calculate the mass, center of gravity position and moment of inertia of each mesh unit.

[0100] Construct a hydrodynamics module: use three-dimensional potential flow theory to calculate the radiation and diffraction forces of the hull; use Morison's formula to calculate the viscous hydrodynamic force; establish a Green's function method solver that takes into account the free surface effect to calculate the wave excitation force.

[0101] Set up the propulsion system model: establish a fixed-pitch propeller model and use the B-series propeller characteristic curve; build a rudder system model; use four-quadrant thrust coefficients and torque coefficients to describe the hydrodynamic characteristics of the thruster-rudder system.

[0102] Establish an environmental force model: the Isherwood wind coefficient method is used for wind force calculation; the wave force calculation is based on the second-order wave theory, taking into account the first-order and second-order wave forces; the water flow force calculation adopts the relative velocity method, taking into account the distribution of water flow velocity on the hull surface.

[0103] Integrated dynamic simulation system:

[0104] (1) Ocean environment parameter generation module: The JONSWAP spectrum model is used to generate irregular wave fields; the logarithmic wind speed profile model is used to generate wind field data; and the current field data is generated based on the tidal current forecast model.

[0105] (2) Ship motion equation solver: Establish a numerical solver based on the six-degree-of-freedom motion equation; use the fourth-order Runge-Kutta method for time domain integration; set motion constraints.

[0106] (3) Shipboard sensing system: simulates the data collection process of GPS receivers, fiber optic gyroscopes, motion sensors, Doppler speed meters and other equipment; Environmental monitoring system: simulates the data collection process of ultrasonic anemometers, wave radars, acoustic Doppler current profilers, temperature sensors and other equipment.

[0107] (4) Data storage and analysis module: Use a time series database to store collected sensor data; set up a data cache mechanism; implement data preprocessing functions, including outlier detection, data smoothing, and resampling; establish a data analysis interface to support real-time data statistics and historical data query.

[0108] In one embodiment of the present invention, Figure 2 As shown, in step S2, the dual network structure model includes:

[0109] Macro Strategy Network:

[0110] The input layer is provided with a first input sublayer and a second input sublayer for receiving ship status parameters and ocean environment parameters respectively; the input sublayers are provided in parallel.

[0111] The timing coding layer sets a timing coding module after each input sublayer to perform feature marking on input signals with long time intervals to maintain timing information.

[0112] The feature extraction layer combines a bidirectional long short-term memory network with a one-dimensional convolutional network to process the input ship state parameters and ocean environment parameters in parallel and extract time series features respectively. The bidirectional long short-term memory network contains two hidden layers to capture long-term dependencies. The one-dimensional convolutional network uses multiple convolutional layers with different convolution kernel sizes to extract local time series features.

[0113] The feature fusion layer fuses the features extracted by the bidirectional long short-term memory network and the one-dimensional convolutional network through a gating mechanism to form a comprehensive feature representation; the gating mechanism includes an update gate and a reset gate to control the selective fusion of features.

[0114] The strategy generation layer connects two layers of fully connected networks. A learnable residual connection is used between each layer of the network. The activation function is Swish, which is used to generate a heading planning strategy vector.

[0115] Micro-control network:

[0116] In the input layer, for data with shorter time intervals, a multi-channel convolution input structure is set to map the high-frequency sampled ship state parameters and ocean environment parameters into a multi-dimensional feature space.

[0117] The multi-scale convolution layer combines dilated convolution with variable kernel convolution to capture the temporal features of the micro-control stage at multiple scales. The dilated convolution expands the receptive field by setting different dilation rates, and the variable kernel convolution adapts to features of different scales by learning different convolution kernel shapes.

[0118] The attention enhancement layer adopts a multi-head causal attention mechanism. Multiple attention heads are set. Each attention head independently calculates the feature correlation between the current moment and the historical moment, and performs weighted aggregation to assign feature weights of different historical moments to the control output of the current moment.

[0119] The control output layer is connected to a two-branch fully connected network. One branch is used to output the rudder angle distribution, and the other branch is used to output the propulsion force distribution. Both branches use the Mish activation function.

[0120] Cross-network information interaction mechanism:

[0121] The temporal interactive attention module is set between the strategy generation layer of the macro policy network and the input layer of the micro control network to realize the dynamic alignment of information between the output of the macro policy network and the input of the micro control network. This module uses the self-attention mechanism to realize the dynamic alignment of information. Specifically, the heading planning strategy vector output by the macro policy network is used as the query vector, and the historical state characteristics of the micro control network are used as the key vector and value vector. By calculating the similarity between the query vector and the key vector, and combining the temporal mask function and the time interval adaptation function, the selective use of historical information is realized to ensure that the micro control network can generate appropriate control instructions according to the planning results of the macro policy network. This module uses the following attention calculation formula:

[0122]

[0123] Where Q, K, and V are query, key, and value vectors respectively; d k is the dimension of the key vector; M(t) is the temporal mask function; G(ΔT) is the time interval adaptation function, which dynamically adjusts the attention intensity according to the predicted time span.

[0124] Through the design and connection of the above network structure, the macro-strategy network optimizes long-term heading planning, and the micro-control network accurately generates short-term control instructions. At the same time, the information coordination between the two networks is guaranteed through the temporal interactive attention module.

[0125] In one embodiment of the present invention, Figure 3 As shown, step S3 includes:

[0126] S31. Construct comprehensive evaluation function R t , based on the comprehensive evaluation function R t Define the long-term return R long-term and the short-term immediate reward R short-term .

[0127] Before the model training begins, a comprehensive evaluation function is first constructed as the optimization target for training. This evaluation function comprehensively considers three aspects: trajectory tracking error, energy consumption, and control command smoothness, and achieves a balance of multiple targets through dynamic weight coefficients. The calculation formula of the comprehensive evaluation function is as follows:

[0128] R t =w1·f traj (s t )+w2·f energy (a t )+w3·f smooth (s t ,a t )

[0129] In the formula, R tis the reward value at time step t; f traj (s t ) indicates state s t The trajectory tracking error at time ; f energy (a t ) means taking action a t Energy consumption when smooth (s t ,a t ) indicates state s t Take action a t w1, w2, and w3 are dynamic weight coefficients. w1, w2, and w3 will be adaptively adjusted according to the current sea conditions, mission phase, and control objectives. Specifically, in severe sea conditions, the system will increase the weight w1 of the trajectory tracking error to ensure control accuracy; in stable sea conditions, the system will increase the weight w2 of energy consumption to improve energy efficiency; in mission phases that require rapid maneuvers, the system will reduce the weight w3 of control smoothness to increase response speed; and in stable navigation phases, w3 will be increased to ensure control stability.

[0130] Specifically, f traj (s t ) represents the deviation between the current ship position and the target trajectory, and the calculation formula is:

[0131]

[0132] In the formula, (x t ,y t ) is the current ship position; (x tar ,y tar ) is the target position; θ t is the current heading angle; θ tar is the target heading angle; ω1 and ω2 are weight coefficients; the negative sign indicates that the larger the error, the smaller the reward value.

[0133] f energy (a t ) represents the energy consumption required to execute the control action, and the calculation formula is:

[0134]

[0135] Where n t is the propeller speed ratio, which represents the ratio of the desired propeller speed to the rated speed; δ t is the rudder angle; c1 and c2 are energy consumption coefficients; the negative sign means that the greater the energy consumption, the smaller the reward value.

[0136] f smooth (s t ,a t) represents the degree of change between adjacent control instructions, and the calculation formula is:

[0137] f smooth (s t ,a t )=-[(n t -n t-1 ) 2 +k·(δ t -δ t-1 ) 2 ]

[0138] Where n t 、n t-1 are the propeller speed ratios at the current and previous moments, respectively; δ t ,δ t-1 are the rudder angles at the current and previous moments respectively; k is the weight coefficient of the rudder angle change; the negative sign indicates that the greater the change in the control command, the smaller the reward value.

[0139] Based on the comprehensive evaluation function, long-term rewards and short-term immediate rewards are defined. The calculation of long-term rewards takes into account the timing discount and sea state adaptability, and its calculation formula is:

[0140]

[0141] In the formula, ρ t is the timing discount factor; g(H t ) is the sea condition adaptation function, according to the current sea condition H t Adjust the reward strength.

[0142] Short-term immediate reward R short-term The calculation formula is: R short-term =R t , that is, the short-term immediate reward directly uses the reward value calculated by the comprehensive evaluation function at the current time step t.

[0143] S32, define the state space S and action space A of the ship's motion, the state vector s t ∈S is defined as a 21-dimensional vector:

[0144]

[0145] In the formula, (x t ,y t ) is the position coordinate of the ship in the global coordinate system; θ t is the heading angle; v t is the ship speed; φ pit is the pitch angle; φ roll is the roll angle; φ yaw The head swing angle; ω yaw is the yaw angular velocity; v latis the lateral speed; v long is the longitudinal velocity; v wind is the wind speed; θ wind is the wind direction; h wave is the wave height; θ wave is the wave direction; v cur is the water flow velocity; θ cur is the water flow direction; d depth is the water depth; T water is the sea temperature; (x tar ,y tar ) is the target position coordinate; φ task The current task stage;

[0146] Action vector a t ∈A is defined as a two-dimensional vector of propeller speed ratio and rudder angle:

[0147] a t =[n t ,δ t ] T

[0148] Where n t is the propeller speed ratio, which represents the ratio of the desired propeller speed to the rated speed; δ t The rudder angle.

[0149] S33, first train the macro strategy network, based on the long-term return R long-term Iterate and optimize the heading planning strategy; adopt a multi-batch experience replay mechanism to screen key training samples covering a variety of sea conditions.

[0150] Specifically, the model training adopts a hierarchical strategy, and the macro policy network is trained first. At this stage, a multi-batch experience replay mechanism is used to select key samples containing various sea conditions from historical data for training. The loss function of the macro policy network includes policy loss, value function loss, and entropy regularization loss:

[0151] L macro =L policy +β1·L value +β2·L entropy

[0152] in:

[0153]

[0154]

[0155] Where, L macro is the macro strategy network loss function, L policy is the strategy loss, L value is the value function loss, Lentropy is the entropy regularization loss, β1 and β2 are weight coefficients; π θ is the parameterized policy function, π θ (a t |s t ) means that in a given state s t Take action a t The probability of θ (a|s t ) means that in a given state s t The probability distribution of taking any action a under t =R long-term -V(s t ) is the advantage function, V(s t ) is the state value function, In the strategy π θ is the expected value under , ‖·‖ represents the Euclidean norm.

[0156] Strategy loss L policy Used to optimize the policy function by maximizing the log-likelihood weighted by the advantage function. The value function loss L value The state value function is optimized by minimizing the mean square error between the predicted value and the actual return. Entropy regularization loss L entropy Used to maintain the exploratory nature of the strategy and prevent premature convergence to a suboptimal strategy.

[0157] In this embodiment, the multi-batch experience playback mechanism is implemented as follows:

[0158] First, define each experience sample e in the experience replay pool D as a five-tuple:

[0159] e=(s t ,a t ,R t ,s t+1 ,H t )

[0160] Among them, s t is the current state, a t For the action to be performed, R t For the reward, s t+1 is the next state, H t The current sea conditions.

[0161] The core calculation formula of the experience replay mechanism is as follows:

[0162] Sample importance weight calculation: Among them, g(H t ) is the sea state adaptation function, which is used to increase the diversity of samples of different sea conditions. max The maximum reward value in the experience pool.

[0163] Batch division and sampling probability: P(e) = w(e) / ∑w(e i ), where P(e) represents the probability that sample e is selected.

[0164] Multi-batch sampling strategy: Batch i ={e j |e j ~P(e),j=1,2,...,M}, where Batch i represents the i-th batch, and M is the batch size. N batches are sampled in each training cycle to form a batch set: Batch = {Batch1, Batch2, ..., Batch N}.

[0165] Batch difference calculation: D i ,Batch i )=‖μ(Batch i )-μ(Batch j )‖ 2 ,μ(Batch i ) indicates batch i The feature mean of all samples in .

[0166] Batch screening criteria: Batch * = argmax Batch [∑ i ∑ j D(Batch i ,Batch i )]; subject to |Batch| = K, where K is the number of batches finally retained, and the diversity of samples is ensured by maximizing the difference between batches.

[0167] S34, after the macro strategy network initially converges, switch to the training phase of the micro control network; based on the short-term immediate reward R short-term , conduct reinforcement learning iterations of the micro-control network and generate specific control strategies.

[0168] Specifically, the loss function of the micro-control network includes temporal difference loss, smoothing loss and constraint loss:

[0169] L micro =L TD +μ1·L smooth +μ2·L constraint

[0170] in:

[0171]

[0172] L smooth =‖a t -a t-1 ‖ 2

[0173] L constraint =max(0,‖a t ‖-a max ) 2

[0174] Where, L micro is the micro-control network loss function, L TD is the timing difference loss, L smooth is the smoothing loss, L constraint is the constraint loss, μ1 and μ2 are weight coefficients; Q is the action value function, Q(s t ,a t ) means that in a given state s t Take action a t The expected return that can be obtained, Q(s t+1 ,a t+1 ) means that in a given state s t+1 Take action a t+1 The expected return that can be obtained, γ is the discount factor; a t 、a t-1 are the action instructions for the current time step and the previous time step respectively; ‖a t ‖ indicates action instruction a t The modulus length, i.e. a t The Euclidean norm of max is the maximum allowed modulus of the action space.

[0175] Timing difference loss L TD Used to optimize the action value function by minimizing the difference between the current Q value and the target Q value. Smooth loss L smooth Used to ensure the continuity of control instructions and constrain the loss L constraint Ensure that control instructions are within reasonable range.

[0176] S35. Through dynamic task allocation, different environmental complexity scenarios are introduced to enable the macro strategy network and micro control network to adapt to different sea conditions respectively.

[0177] Specifically, we first establish an environmental complexity assessment system and define the environmental complexity index C:

[0178] C=w1·f(H wave )+w2·f(H wind )+w3·f(H current )

[0179] In the formula, f(H wave ) represents the wave complexity, which is calculated from the wave height and wave direction; f(H wind ) represents the complexity of wind conditions, which is calculated from wind speed and wind direction; f(H current ) represents the complexity of water flow, which is calculated by flow velocity and flow direction; w1, w2, and w3 are weight coefficients.

[0180] Task allocation formula: T macro =f macro (C); T micro =f micro (C); where T macro and T micro denote the training tasks assigned to the macro policy network and the micro control network, respectively, and f macro 、f micro Function for dynamically adjusting task allocation.

[0181] S36. A gradient alignment module is set up during training to monitor the strategy differences between the macro strategy network and the micro control network. Through the gradient alignment mechanism, the parameters of the macro strategy network and the micro control network are adjusted synchronously.

[0182] The gradient alignment mechanism is as follows:

[0183] For the macro strategy network and the micro control network, the loss function L is defined respectively macro and L micro ;

[0184] Calculate the respective gradient vector and Among them, θ macro and θ micro denote the parameter sets of the macro strategy network and the micro control network respectively;

[0185] Using the gradient alignment formula, the gradient vectors of the two networks are aligned and the network parameters are adjusted synchronously. The gradient alignment formula is as follows:

[0186]

[0187] Where, L align is the gradient alignment loss, λ is the trade-off coefficient, η(t) is the time-varying adjustment factor, and ‖·‖ represents the Euclidean norm.

[0188] S37. After the macro-strategy network and the micro-control network are trained to the preset convergence standard, joint fine-tuning is performed. During the joint fine-tuning process, the macro-strategy network and the micro-control network are simultaneously optimized by minimizing the comprehensive loss function. When the fine-tuning is completed, a trained dual-network structure model is obtained.

[0189] The joint fine-tuning process uses a comprehensive loss function for overall optimization:

[0190] L total =L macro +L micro +α·L align

[0191] α is the trade-off coefficient.

[0192] By minimizing the comprehensive loss function, the macro policy network and the micro control network are optimized together. During the joint optimization process, the weight coefficients of each loss term are dynamically adjusted to balance different training objectives. When the model performance is no longer significantly improved, the training process ends.

[0193] In one embodiment of the present invention, step S4 includes:

[0194] S41, loading the trained dual network structure model into the ship control system, initializing the model parameters and setting the operating environment.

[0195] First, the trained dual network structure model is loaded into the ship control system, including the network structure and weight parameters of the macro strategy network and the micro control network. The model running environment is initialized, and the parameter range of the state space and action space is set, where the state space contains a 21-dimensional state vector, and the action space contains two control quantities: the propeller speed ratio and the rudder angle. At the same time, the parameters of the temporal interactive attention module are configured, including the dimensions of the query vector, key vector, and value vector, as well as the initial parameters of the temporal mask function and the time interval adaptation function.

[0196] S42. Receive ship status parameters and ocean environment parameters in real time through a sliding time window method, wherein the sliding time window method includes dynamic updating of historical data and current data.

[0197] The sliding time window method is used to receive and update data in real time, and the window length is dynamically adjusted according to the control cycle. The ship's state parameters are collected in real time through the shipboard sensor system, including position coordinates, heading angle, speed, pitch angle, roll angle, bow angle, bow angular velocity, lateral speed and longitudinal speed; the marine environmental parameters are collected through the environmental monitoring system, including wind speed, wind direction, wave height, wave direction, water flow speed, water flow direction, water depth and sea temperature. The system maintains a certain length of historical data and realizes dynamic update of data through the sliding window mechanism to ensure that the model can obtain continuous time series information.

[0198] S43. Based on the received ship state parameters and ocean environment parameters, a heading planning strategy is generated using a macro strategy network.

[0199] Based on the data collected in the sliding time window, the macro-strategy network is activated to generate the heading planning strategy. First, the input ship state parameters and ocean environment parameters are processed in parallel by the bidirectional long short-term memory network and the one-dimensional convolutional network to extract the time series features. Then, these features are fused through the gating mechanism to form a comprehensive feature representation. Finally, through the two-layer fully connected network, combined with the Swish activation function, the heading planning strategy vector is generated to provide guidance for micro-control.

[0200] S44. Based on the heading planning strategy generated by the macro strategy network, specific control instructions are generated in combination with the micro control network, including rudder angle distribution and propulsion force distribution.

[0201] The micro-control network receives the heading planning strategy generated by the macro-strategy network and realizes dynamic information alignment through the temporal interactive attention module. The multi-channel convolution input structure is used to process high-frequency sampling data, and the multi-scale temporal features are captured through the combination of dilated convolution and variable kernel convolution. The multi-head causal attention mechanism is used to assign feature weights at different historical moments to the current control output. Finally, the dual-branch fully connected network is combined with the Mish activation function to output specific control instructions for the rudder angle distribution and the propulsion force distribution.

[0202] S45. Monitor the ship's motion state and environmental changes in real time, dynamically adjust the control strategy based on the predicted environmental changes, generate optimal control instructions, and achieve real-time optimization control of the ship's motion trajectory.

[0203] The system continuously monitors the ship's motion state and environmental changes, and predicts the trend of environmental changes through the time interval adaptive function G(ΔT) in the temporal interactive attention module. When significant environmental changes are detected, the system automatically adjusts the control strategy and recalculates the optimal control instructions. During the adjustment process, the macro strategy network updates the heading plan based on the predicted environmental changes, and the micro control network adjusts the specific control parameters accordingly to achieve real-time optimization control of the ship's motion trajectory, ensuring that the ship can complete the scheduled navigation mission safely and efficiently.

[0204] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A ship motion trajectory optimization control method based on deep learning, characterized in that: include: S1. Construct a ship motion simulation environment and establish a dynamic simulation system including marine environment parameters; Collecting ship operation data, the operation data includes ship status parameters and marine environment parameters, wherein the ship status parameters are obtained through a ship-borne sensor system, and the marine environment parameters are obtained through an environmental monitoring system; S2. Construct a dual network structure model, including a macro strategy network and a micro control network, wherein the macro strategy network generates a heading planning strategy at a longer time interval, and the micro control network generates a specific control instruction at a shorter time interval; the input of the dual network structure model includes ship state parameters and marine environment parameters, and the two networks realize information interaction through a temporal attention mechanism; S3. A hierarchical training strategy is adopted to train the dual network structure model in a ship motion simulation environment. First, the macro strategy network is trained to generate the overall heading strategy, and then the micro control network is trained to achieve precise control. The macro strategy network is optimized based on long-term returns, and the micro control network is optimized based on short-term control accuracy. The two networks maintain consistency through a gradient alignment mechanism. S4. Deploy the trained dual network structure model to the ship control system, implement predictive control through the sliding time window method, receive ship state parameters and ocean environment parameters in real time, dynamically adjust the control strategy based on the predicted environmental changes, generate optimal control instructions, and realize real-time optimization control of the ship's motion trajectory.

2. The ship motion trajectory optimization control method based on deep learning according to claim 1, characterized in that: The ship status parameters include: the ship's position coordinates, heading angle, speed, pitch angle, roll angle, yaw angle, bow angular velocity, lateral speed and longitudinal speed; the ocean environment parameters include: wind speed, wind direction, wave height, wave direction, water flow speed, water flow direction, water depth and sea temperature.

3. The ship motion trajectory optimization control method based on deep learning according to claim 2, characterized in that: In step S1, constructing a ship motion simulation environment includes: Establish a three-dimensional ship model, including the hull geometry model and mass distribution model; Construct a hydrodynamics module to calculate various hydrodynamic forces acting on ships; Set up the propulsion system model, including the main thruster and steering gear system; Establish environmental force models to simulate the effects of wind, wave and current forces; The dynamic simulation system is integrated into the ship motion simulation environment, including: Construct an ocean environment parameter generation module to generate time-varying environment parameters; Establish a solver for the ship motion equations to calculate the motion response of the ship in complex sea conditions; Set up a sensor data simulation module to simulate the data collection process of the shipborne sensor system and environmental monitoring system; Establish a data storage and analysis module to record and process various types of data during the simulation process.

4. The ship motion trajectory optimization control method based on deep learning according to claim 2, characterized in that: The dual network structure model includes: Macro Strategy Network: An input layer, a first input sublayer and a second input sublayer for receiving ship state parameters and ocean environment parameters respectively; The temporal coding layer is a temporal coding module set after each input sublayer to perform feature marking on input signals with long time intervals; The feature extraction layer combines a bidirectional long short-term memory network with a one-dimensional convolutional network to process the input ship state parameters and ocean environment parameters in parallel and extract time series features respectively; The feature fusion layer fuses the features extracted by the bidirectional long short-term memory network and the one-dimensional convolutional network through a gating mechanism to form a comprehensive feature representation; The strategy generation layer connects two layers of fully connected networks. Each layer of the network uses a learnable residual connection and the activation function is Swish, which is used to generate a heading planning strategy vector. Micro-control network: The input layer sets a multi-channel convolution input structure for data with shorter time intervals, mapping the high-frequency sampled ship state parameters and ocean environment parameters into a multi-dimensional feature space; The multi-scale convolution layer combines dilated convolution with variable kernel convolution to capture the temporal features of the micro-control stage at multiple scales. The attention enhancement layer uses a multi-head causal attention mechanism to assign feature weights of different historical moments to the control output at the current moment; Control the output layer, connect a double-branch fully connected network, one branch is used to output the rudder angle distribution, the other branch is used to output the propulsion force distribution, and the activation function is Mish; Cross-network information interaction mechanism: The temporal interactive attention module is set between the strategy generation layer of the macro strategy network and the input layer of the micro control network to achieve dynamic alignment of information between the output of the macro strategy network and the input of the micro control network.

5. The ship motion trajectory optimization control method based on deep learning according to claim 4 is characterized in that: Step S3 includes: S31. Construct comprehensive evaluation function R t , based on the comprehensive evaluation function R t Define the long-term return R long-term and the short-term immediate reward R short-term ; S32, define the state space S and action space A of the ship's motion, the state vector s t ∈S is defined as: In the formula, (x t ,y t ) is the position coordinate of the ship in the global coordinate system; θ t is the heading angle; v t is the ship speed; φ pit is the pitch angle; φ roll is the roll angle; φ yaw The head swing angle; ω yaw is the yaw angular velocity; v lat is the lateral speed; v long is the longitudinal velocity; v wind is the wind speed; θ wind is the wind direction; h wave is the wave height; θ wave is the wave direction; v cur is the water flow velocity; θ cur is the direction of water flow; d depth is the water depth; T water is the sea temperature; (x tar ,y tar ) is the target position coordinate; φ task The current task stage; Action vector a t ∈A is defined as: a t =[n t ,d t ] T Where n t is the propeller speed ratio, which represents the ratio of the desired propeller speed to the rated speed; δ t is the rudder angle; S33, first train the macro strategy network, based on the long-term return R long-term Iterate and optimize the course planning strategy; use a multi-batch experience playback mechanism to screen key training samples covering a variety of sea conditions; S34, after the macro strategy network initially converges, switch to the training phase of the micro control network; based on the short-term immediate reward R short-term , conduct reinforcement learning iterations of the micro-control network to generate specific control strategies; S35. Through dynamic task allocation, different environmental complexity scenarios are introduced to make the macro strategy network and micro control network adapt to different sea conditions respectively; S36. A gradient alignment module is set up during training to monitor the policy differences between the macro policy network and the micro control network; the parameters of the macro policy network and the micro control network are adjusted synchronously through the gradient alignment mechanism; S37. After the macro-strategy network and the micro-control network are trained to the preset convergence standard, joint fine-tuning is performed. During the joint fine-tuning process, the macro-strategy network and the micro-control network are simultaneously optimized by minimizing the comprehensive loss function. When the fine-tuning is completed, a trained dual-network structure model is obtained.

6. The ship motion trajectory optimization control method based on deep learning according to claim 5, characterized in that: The calculation formula of the comprehensive evaluation function is as follows: R t =w1·f traj (s t )+w2·f energy (a t )+w3·f smooth (s t ,a t ) In the formula, R t is the reward value at time step t; f traj (s t ) indicates state s t The trajectory tracking error at time ; f energy (a t ) means taking action a t Energy consumption when smooth (s t ,a t ) indicates state s t Take action a t The smoothness of the control command when ; w1, w2, w3 are dynamic weight coefficients; Long-term return R long-term The calculation formula is as follows: In the formula, ρ t is the timing discount factor; g(H t ) is the sea condition adaptation function, according to the current sea condition H t Adjust the intensity of rewards; Short-term immediate reward R short-term The calculation formula is: R short-term =R t , that is, the short-term immediate reward directly uses the reward value calculated by the comprehensive evaluation function at the current time step t.

7. The ship motion trajectory optimization control method based on deep learning according to claim 5, characterized in that: In step S36, the gradient alignment mechanism is as follows: For the macro strategy network and the micro control network, the loss function L is defined respectively macro and L micro ; Calculate the respective gradient vector and in, and denote the parameter sets of the macro strategy network and the micro control network respectively; Using the gradient alignment formula, the gradient vectors of the two networks are aligned and the network parameters are adjusted synchronously. The gradient alignment formula is as follows: Where, L align is the gradient alignment loss, λ is the trade-off coefficient, η(t) is the time-varying adjustment factor, and ‖·‖ represents the Euclidean norm.

8. The ship motion trajectory optimization control method based on deep learning according to claim 5, characterized in that: The comprehensive loss function is as follows: L total =L macro +L micro +α·L align L macro =L policy +β1·L value +β2·L entropy L micro =L TD +μ1·L smooth +μ2·L constraint L smooth =‖a t -a t-1 ‖ 2 L constraint =max(0,‖a t ‖-a max ) 2 Where, L total is the comprehensive loss function, L macro is the macro strategy network loss function, L micro is the micro-control network loss function, α is the trade-off coefficient, L align is the gradient alignment loss; L policy is the strategy loss, L value is the value function loss, L entropy is the entropy regularization loss, β1 and β2 are weight coefficients; is a parameterized policy function, In a given state s t Take action a t The probability of In a given state s t The probability distribution of taking any action a under t =R long-term -V(s t ) is the advantage function, V(s t ) is the state value function, Indicated in strategy , ‖·‖ represents the Euclidean norm; L TD is the timing difference loss, L smooth is the smoothing loss, L constraint is the constraint loss, μ1 and μ2 are weight coefficients; Q is the action value function, Q(s t ,a t ) means that in a given state s t Take action a t The expected return that can be obtained, Q(s t+1 ,a t+1 ) means that in a given state s t+1 Take action a t+1 The expected return that can be obtained, γ is the discount factor; a t 、a t-1 are the action instructions for the current time step and the previous time step respectively; ‖a t ‖ indicates action instruction a t The modulus length, i.e. a t The Euclidean norm of max is the maximum allowed modulus of the action space.

9. The ship motion trajectory optimization control method based on deep learning according to claim 4, characterized in that: The following attention calculation formula is used in the temporal interaction attention module: Where Q, K, and V are query, key, and value vectors respectively; d k is the dimension of the key vector; M(t) is the temporal mask function; G(ΔT) is the time interval adaptation function, which dynamically adjusts the attention intensity according to the predicted time span.

10. The ship motion trajectory optimization control method based on deep learning according to claim 5, characterized in that: Step S4 includes: S41, loading the trained dual network structure model into the ship control system, initializing the model parameters and setting the operating environment; S42, receiving ship status parameters and ocean environment parameters in real time through a sliding time window method, wherein the sliding time window method includes dynamic updating of historical data and current data; S43, based on the received ship state parameters and ocean environment parameters, using the macro strategy network to generate a heading planning strategy; S44, generating specific control instructions including rudder angle distribution and propulsion force distribution according to the heading planning strategy generated by the macro strategy network and in combination with the micro control network; S45. Monitor the ship's motion state and environmental changes in real time, dynamically adjust the control strategy based on the predicted environmental changes, generate optimal control instructions, and achieve real-time optimization control of the ship's motion trajectory.

Images