An Energy-Saving Path Planning and Learning Optimization Method for Intelligent Ship Navigation

Abstract

This invention discloses an energy-saving path planning and learning optimization method for intelligent ship navigation, belonging to the field of ship path planning. This method is based on the kinematic parameter mapping relationship between a fixed coordinate system and an appendage coordinate system, combined with the Fossen rigid body dynamics model, to construct a ship maneuvering motion model. By constructing mathematical models of sea wind, waves, and currents and substituting them into the ship maneuvering motion model, a ship navigation energy consumption model is finally constructed. Three types of constraints—speed, heading angle, and safety distance—are set, and a safety reinforcement learning algorithm is designed based on the MDP framework. The safety reinforcement learning algorithm is improved, and experiments are conducted through a simulation environment and multiple sets of experimental conditions, with results analyzed. This invention constructs an energy-saving path planning model for ships, proposes an improved safety reinforcement learning method, enhances ship navigation energy efficiency and safety, and provides an effective solution for green and intelligent shipping.

Description

Technical Field

[0001] This invention belongs to the field of ship path planning, and specifically relates to an energy-saving path planning and learning optimization method for intelligent ship navigation. Background Technology

[0002] With the rapid development of the global shipping industry, ships play a vital role as the primary means of transportation, undertaking the majority of global cargo transport tasks. The applications of modern ships extend beyond traditional cargo transport to emerging fields such as marine engineering, ocean exploration, and underwater robotics. Due to technological advancements, ship automation and intelligence have become significant development trends, with autonomous navigation and automatic control technologies receiving widespread attention. In ship operation automation, path planning, as one of the key technologies for achieving efficient and safe navigation, plays a crucial role. A well-designed path planning strategy can not only improve navigation efficiency and save fuel but also effectively avoid collision risks, ensuring the safety of the ship and crew. Therefore, designing reliable and efficient path planning algorithms has become a research focus in both academic research and industrial applications. Despite significant progress, ship path planning still faces many challenges. In complex marine environments, path planning must consider factors such as marine meteorological conditions and ship dynamic performance, while simultaneously meeting the requirements of collision avoidance and safe navigation. Furthermore, path planning algorithms need to be adaptable to different navigation environments and capable of handling unexpected situations and uncertainties. Therefore, developing intelligent path planning methods that comprehensively consider multiple factors is crucial for improving the endurance and safety of ship navigation.

[0003] Currently, both domestic and international efforts primarily focus on improving the endurance of unmanned surface vessels (USVs) through several methods, including optimizing hull structure, adopting hybrid enhanced energy propulsion, and reducing hull weight. While these methods can effectively extend a vessel's range, they require determining the hull structure or propulsion energy source during the initial design phase, or involving complex and large-scale modifications to existing vessels. In practical applications, these methods have significant limitations in terms of both time and economic costs. Therefore, improving the path planning algorithm to enhance endurance has become the mainstream direction for ship energy conservation. Domestic and international scholars have mainly focused on improving path planning algorithms to enhance a vessel's endurance. However, these efforts often fail to adequately consider the varying environmental loads affecting a vessel's actual navigation, focusing solely on speed, wind, and currents. This results in paths that deviate from reality, failing to comprehensively account for the interference of wind, waves, and currents. Furthermore, they neglect the varying energy consumption required to resist wind, waves, and currents depending on the vessel's course.

[0004] Therefore, there is an urgent need to propose an energy-saving path planning model and optimization method for intelligent ship navigation, in order to solve the technical problem of achieving energy-saving path planning for ships under the influence of complex environmental interference such as wind, waves and currents in the existing technology. This invention is proposed to solve this problem. Summary of the Invention

[0005] To address the shortcomings of the existing technologies, the purpose of this invention is to propose an energy-saving path planning and learning optimization method for intelligent ship navigation. Focusing on the three degrees of freedom of motion—sway, roll, and pitch—a ship maneuvering motion model is constructed. By incorporating environmental disturbances such as wind, waves, and currents into the ship maneuvering motion model, a ship navigation energy consumption model is built. Constraints on speed, heading angle, and safe distance are set. Combined with an improved safety reinforcement learning method (including ε-greedy strategy and KMDP framework), a simulation environment incorporating real ocean features and obstacles is built. Multiple groups and conditions of experiments are designed and the results are analyzed to achieve energy-saving and safe optimal path planning for ships in complex ocean environments. Simultaneously, the algorithm's performance is verified, and directions for improvement are clarified, providing technical support for the greening and safety of autonomous navigation of intelligent ships.

[0006] The technical solution adopted in this invention:

[0007] An energy-saving path planning and learning optimization method for intelligent ship navigation includes the following steps:

[0008] S1) Based on the motion of the three degrees of freedom of sway, roll and bow, and based on the kinematic parameter mapping relationship between the fixed coordinate system and the attached coordinate system, combined with the Fossen rigid body dynamics model, a ship maneuvering motion model is constructed.

[0009] S2) Construct mathematical models of sea wind, sea waves, and ocean currents and substitute them into the ship maneuvering motion model to finally construct a ship navigation energy consumption model in sway, roll, and bow motion.

[0010] S3) Set constraints and design a secure reinforcement learning algorithm based on the MDP framework;

[0011] S4) An improved secure reinforcement learning algorithm based on ε-greedy policy and KMDP;

[0012] S5) Verify and analyze the results by building a simulation environment and setting multiple sets of conditions.

[0013] Preferably, in step S1), the specific process of constructing the ship maneuvering motion model is as follows:

[0014] Based on the state of a ship undergoing arbitrary degrees of freedom of motion, two coordinate systems are established: a fixed coordinate system and an attached coordinate system. The ship's motion generates six degrees of freedom: pitch, sway, bow roll, heave, roll, and pitch.

[0015] Based on the ship's motion in the horizontal plane, the motion components of heel, roll, and pitch are simplified, and the heel angle is set. pitch angle heave linear velocity Pitch angular velocity Roll angular velocity Only the three degrees of freedom of sway, roll, and pitch are analyzed. The kinematic parameter relationship between the attached coordinate system and the inertial coordinate system is calculated, that is, the kinematic parameter transformation relationship between the fixed coordinate system and the attached coordinate system. The calculation is shown in formula (1):

[0016] (1)

[0017] in, For ships in a fixed coordinate system The derivative of the position vector of the axis. Let be the longitudinal velocity of the ship in the attached coordinate system. Let be the ship's sway linear velocity in the attached coordinate system. For ships in a fixed coordinate system The derivative of the position vector of the axis. It is the azimuth angle. The derivative of the azimuth angle. The bow roll angular velocity in the attached coordinate system;

[0018] The forces and moments acting on a ship in its three degrees of freedom include propeller thrust. Hydrodynamics and torques, as well as disturbances from the marine environment. The ship's center of gravity is located at the mid-longitudinal section. The Fossen rigid body dynamics model is used to describe the ship's forces and motions in three degrees of freedom, i.e., the ship's maneuvering motion model. Its calculation is shown in formula (2):

[0019] (2)

[0020] In the formula, This is the inertia matrix, which is related to the ship's linear velocity and angular acceleration. Motion state vector The time derivative; The Coriolis centripetal force matrix depends on the ship's added mass; This is the motion state vector; It is the damping coefficient matrix;

[0021] in, The calculation formula is shown in formula (3):

[0022] (3)

[0023] The calculation formula is shown in formula (4):

[0024] (4)

[0025] The calculation formula is shown in formula (5):

[0026] (5)

[0027] In the formula, Each of these represents a coefficient of hydrodynamic force.

[0028] Preferably, in step S2), the specific steps for constructing the ship's navigation energy consumption model are as follows:

[0029] (1) Constructing a mathematical model of sea breeze

[0030] When a ship is in motion, the forces and moments of wind loads are analyzed using relative wind speed, and a mathematical model of sea wind is constructed. The calculation is shown in formula (6):

[0031] (6)

[0032] In the formula, The vector of wind disturbance force and torque. air density, Relative wind speed, The relative wind angle of the bow. Let be the cross-sectional area of ​​the ship. This represents the projected area of ​​the ship's side. As captain, The drag coefficient is related to the smoothness of its surface, its overall shape, and its characteristic area (frontal area).

[0033] The relative wind angle of the bow is calculated as shown in formula (7):

[0034] (7)

[0035] In the formula, For wind direction, It is the azimuth angle;

[0036] (2) Constructing a mathematical model of ocean waves

[0037] A mathematical model of ocean waves was established using Daidola's conclusions to quantify the forces exerted by waves on the heave, pitch, roll, and other motions of the ship. The mathematical model is shown in formula (8):

[0038] (8)

[0039] In the formula, The wave disturbance force and torque vector. The density of seawater, The gravitational coefficient, As captain, For amplitude, The wave angle, also known as the wave encounter angle, is the angle of the wave measured relative to the stern line. For wavelength, Wave load factor;

[0040] The wave load factor is calculated using an empirical formula and is the ratio of wave wavelength to ship length. This reflects the force characteristics of waves of different wavelengths on the longitudinal, lateral, and yaw directions of the ship, and its calculation is shown in formula (9):

[0041] (9)

[0042] (3) Constructing a mathematical model of ocean currents

[0043] Based on the disturbance force exerted by ocean currents on the ship's motion, the velocity generated by the ocean current on the ship's motion is superimposed on the ship's speed to obtain the ship's relative velocity to the ocean current; the projection of the ship's relative velocity to the ocean current is then calculated. and A mathematical model of ocean currents is constructed, and its calculation is shown in formula (10):

[0044] (10)

[0045] In the formula This represents the longitudinal linear velocity component of the ocean current in the attached coordinate system. This represents the lateral linear velocity component of the ocean current in the attached coordinate system. This represents the longitudinal absolute velocity component of the ship. This represents the lateral absolute velocity component of the ship. This represents the longitudinal relative velocity component of the ship with respect to the ocean current. This represents the lateral relative velocity component of the ship with respect to the ocean current. Indicates the bow roll rate of the ship;

[0046] Finally, the velocity components of the ship relative to the ocean current are expressed as follows: ;

[0047] (4) Construct a model of ship navigation energy consumption.

[0048] Substituting the sea breeze mathematical model, the sea wave mathematical model, and the ocean current mathematical model into the ship maneuvering motion model, the ship motion model is obtained, and the calculation of the ship motion model is shown in formula (11):

[0049] (11)

[0050] In the formula, The inertia matrix, The time derivative of the relative velocity vector of the ships. The Coriolis centripetal force matrix, Here is the damping coefficient matrix. This is the vector of the resultant force and moment acting on the ship;

[0051] Based on the thrust and torque provided by the propeller, the ship motion model is extended and deformed as shown in formula (12):

[0052] (12)

[0053] In the formula, These are the thrust or torque components in the directions of sway, roll, and pitch, respectively. For the quality of the ship; These are the additional mass derivatives in the directions of sway, roll, and pitch, respectively. These are the longitudinal and lateral relative acceleration components of the ship relative to the ocean current, respectively. These are the velocity damping derivatives in the sway, roll, and pitch directions, respectively. These are the longitudinal and lateral relative velocity components of the ship relative to the ocean current, respectively. The bow roll angular velocity; air density; Relative wind speed; These are the aerodynamic coefficients for longitudinal sway, transverse sway, and vertical sway, respectively. The relative wind angle of the bow; This represents the cross-sectional area of ​​the ship. This represents the projected area of ​​the ship's side. The density of seawater; This is the gravity coefficient; Captain; Amplitude; Wave direction angle; Wave load factor;

[0054] Finally, a model of ship navigation energy consumption under the environmental loads of sea wind, sea waves and ocean currents is constructed, as shown in formula (13):

[0055] (13)

[0056] In the formula, The total energy consumption of the unmanned vessel; These represent the energy consumed by the unmanned vessel during swaying, rolling, and bow-to-swing motions, respectively. These represent the displacement and rotation angle of the unmanned vessel during unit movement, respectively.

[0057] Preferably, in step S3), the specific steps for designing the security reinforcement learning algorithm are as follows:

[0058] (1) Constraints

[0059] Three types of constraints are set: speed constraint, heading angle constraint, and safety distance constraint. These constraints define the "boundaries that must be satisfied" for path planning, which are expressed by formula (14):

[0060] (14)

[0061] in, Indicates energy consumption. Let the energy consumption function be... For state, For a moment, and These are the lower and upper limits of ship speed, respectively. Indicates the speed during navigation. For the ship in position The heading angle at that point, and These are the lower and upper limits of the heading angle, respectively. for The ship's heading angle at any given time. express The ship's heading angle at any given time. For adjacent time periods ( arrive The maximum value of the change in heading angle at any given time. and These are the positions of the ship and the obstacles, respectively. To maintain a safe distance;

[0062] (2) Design of a security reinforcement learning algorithm based on the MDP framework

[0063] The secure reinforcement learning algorithm is built on the Markov Decision Process (MDP) framework, in which the MDP framework consists of... The composition, meaning of each component, and mathematical expression are as follows:

[0064] : A set of states that contains all possible states of the ship. ;

[0065] Action set: Contains all actions that a ship can perform. ;

[0066] : State transition probability model, representing the state transition probability model. Next action Then transition to the next state The probability of;

[0067] Calculate the next state The probability of is calculated as shown in formula (15):

[0068] (15)

[0069] : Reward function, representing the state Next action And transition to the next state The instant reward is 1 for violating security constraints, and a negative reward is obtained for violating security constraints.

[0070] Time discount factor, with a value range of: , The closer it is to 1, the more attention should be paid to long-term returns; the closer it is to 0, the more attention should be paid to short-term returns.

[0071] : Optimal policy, representing the optimal strategy in any state The action selection strategy that maximizes long-term cumulative reward is expressed mathematically as shown in formula (16):

[0072] (16)

[0073] in, To be optimal A function represents a state. Next action The maximum long-term cumulative reward that can be obtained later;

[0074] optimal function The state is characterized by the Bellman equation Next action To maximize long-term cumulative rewards, the optimal The mathematical expression of the Bellman equation is shown in formula (17):

[0075] (17)

[0076] in, Indicates the next state The action to be performed.

[0077] Preferably, in step S4), the specific steps for improving the security reinforcement learning algorithm are as follows:

[0078] First, an ε-greedy strategy is employed to balance the relationship between exploration and exploitation. Second, the knowledge dimension is improved through Markov decision processes, specifically by introducing a knowledge dimension. By constructing KMDP, the assumption that "the model is completely known" in MDP is broken;

[0079] In KMDP, the knowledge dimension is relied upon. The model estimates in the data, combined with the knowledge dimension For the optimal The function is modified and represented as follows: The mathematical expression is shown in formula (18):

[0080] (18)

[0081] In the formula, To combine knowledge dimensions Corrected optimal function;

[0082] It incorporates a knowledge-based reward function, which adjusts based on model confidence, unlike traditional MDPs that rely solely on state transition reward functions. .

[0083] Preferably, in step S5), the results are verified and analyzed by setting up a simulation environment and setting multiple sets of conditions. The specific steps are as follows:

[0084] ① Environment setup

[0085] a. Environmental characteristics:

[0086] Create a simulation environment that incorporates marine features and dynamic elements to simulate complex scenarios of actual ship navigation;

[0087] b. Obstacle setting: The coastline is set as an obstacle to test the algorithm's obstacle avoidance and safe navigation capabilities;

[0088] ② Condition settings

[0089] Set different weather and ocean conditions, and set different mission types and difficulties;

[0090] ③ Simulation Results and Analysis

[0091] a. Results Presentation: Use graphs and tables to present the results in detail, compare and verify the results under different experimental settings, and highlight the performance advantages and limitations of the algorithm;

[0092] b. Performance analysis: Analyze the algorithm's performance under different conditions and evaluate its security performance.

[0093] Compared with existing technologies, this invention proposes an energy-saving path planning and learning optimization method for intelligent ship navigation. The advantages of this method are:

[0094] (1) The ship energy-saving path planning model considering environmental disturbances constructed in this paper significantly improves the realism and practicality of path planning by integrating the ship maneuvering motion model and the environmental disturbance mathematical model. Based on the principles of rigid body mechanics, the model describes in detail the ship's motion response in three degrees of freedom: pitch, sway, and yaw. It also introduces disturbance force modeling of key environmental factors such as wind, waves, and currents, making the ship's dynamic behavior closer to the actual marine environment. The energy consumption model derived on this basis can accurately reflect the changes in thrust and torque of the propeller under different environmental loads, thus providing a reliable basis for energy-saving path optimization. The model not only fully considers the coupling mechanism between ship motion and environmental effects, but also effectively ensures the feasibility of the path and navigation safety by setting multiple constraints such as speed, heading angle, and safety distance, laying a solid theoretical framework and environmental foundation for the subsequent application of reinforcement learning algorithms.

[0095] (2) In our research on energy-saving path search methods using safety reinforcement learning, we systematically designed a learning mechanism centered on energy saving and safety. By incorporating energy consumption, ship position, speed, and obstacle information into the state space, and using heading and speed adjustments as the action space, the agent can continuously learn the optimal strategy during interaction with the environment. A safety penalty term is introduced into the reward function to negatively incentivize behaviors that violate constraints, guiding the agent to actively avoid collision risks and movement limits while exploring the energy-saving potential of the path. This method also integrates a safety constraint layer to evaluate and correct the actions output by reinforcement learning in real time, ensuring that each decision meets safety requirements. This achieves an effective balance between exploration and utilization in the complex and ever-changing marine environment, significantly improving the safety and learning efficiency of the path planning process.

[0096] (3) Based on improved safety reinforcement learning, the ship energy-saving path planning method further enhances the adaptability and convergence performance of the algorithm by introducing an ε-greedy strategy and a knowledge-enhanced Markov decision process (KMDP). The ε-greedy strategy effectively balances the relationship between exploring unknown states and utilizing known experience, avoiding getting trapped in local optima; while the KMDP structure models the model confidence by introducing a knowledge dimension, enabling the system to dynamically adjust the reward mechanism according to the level of environmental cognition, enhancing the incentive to explore unknown areas. Multiple sets of experiments in simulation environments containing static and dynamic obstacles and variable sea conditions show that the proposed method can generate high-quality paths that balance energy saving and safety under different starting points, task types, and environmental conditions, and exhibits good robustness in extreme scenarios. This method not only provides a feasible technical path for ships to achieve autonomous, green, and safe navigation in real marine environments, but also lays the foundation for subsequent processing of more complex multi-agent collaborative and uncertain environmental decision-making problems. Attached Figure Description

[0097] Figure 1 This is a flowchart of the energy-saving path planning model and learning optimization method for intelligent ship navigation of the present invention.

[0098] Figure 2 This refers to the final trajectory band searched in the navigation environment during the specific implementation experiment proposed in this invention;

[0099] Figure 3 This refers to the final trajectory band searched in the navigation environment using the improved method in a specific embodiment of the present invention;

[0100] Figure 4 This refers to the final trajectory band searched in the navigation environment using the unmodified method in a specific embodiment of the present invention. Detailed Implementation

[0101] The technical solutions of the embodiments of this application will be further described clearly and completely below with reference to the accompanying drawings. It should be noted that the described embodiments are only some embodiments of this application, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.

[0102] To make the inventive objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be further described in detail below with reference to the accompanying drawings: In order to better understand the above-mentioned objectives, features, and advantages of this invention, the advantages of this invention will be further illustrated below by comparing the embodiments with the accompanying drawings and specific implementation methods.

[0103] This invention proposes an energy-saving path planning and learning optimization method for intelligent ship navigation, such as... Figure 1 The flowchart illustrates the steps of this method in detail:

[0104] S1) Based on the motion of the three degrees of freedom of sway, roll and bow, and based on the kinematic parameter mapping relationship between the fixed coordinate system and the attached coordinate system, combined with the Fossen rigid body dynamics model, a ship maneuvering motion model is constructed.

[0105] Specifically, in step S1), the specific process of constructing the ship maneuvering motion model is as follows:

[0106] Based on the state of a ship undergoing arbitrary degrees of freedom of motion, two coordinate systems are established: a fixed coordinate system (inertial coordinate system, with the Earth as a fixed reference frame, used to describe the ship's position in the ocean) and an attached coordinate system (ship-attached coordinate system, fixed to the ship and moving with it, used to describe the ship's motion relative to itself). The arbitrary motion of the ship is analyzed from the perspective of rigid body mechanics, consisting of the superposition of translation with respect to a reference point (center of gravity) and rotation about that reference point, resulting in six degrees of freedom: pitch, sway, bow, heave, roll, and pitch. The arbitrary motion of the ship in these six degrees of freedom can be expressed in the attached coordinate system using velocity vectors. ( Let be the linear velocity of the oscillation. The sway linear velocity, (For the linear velocity and angular velocity vectors of the heave) ( The pitch angular velocity, The angular velocity of the roll is... This can be described as the bow roll angular velocity, or it can be expressed as the derivative of the position vector in a fixed coordinate system. ( for The derivative of the position vector along the axial direction. for The derivative of the position vector along the axial direction. for The derivative of the position vector along the axis and the derivative of the position vector of the Euler angles. express;

[0107] Based on the ship's motion in the horizontal plane, the motion components of heel, roll, and pitch are simplified, and the heel angle is... pitch angle heave linear velocity Pitch angular velocity Roll angular velocity Only the three degrees of freedom of sway, roll, and pitch are analyzed, and the kinematic parameter relationship between the attached coordinate system and the inertial coordinate system is calculated, that is, the kinematic parameter transformation relationship between the fixed coordinate system and the attached coordinate system. This can greatly reduce the complexity of the analysis, while retaining the core characteristics of the maneuvering motion in the horizontal plane. Its calculation is shown in formula (1):

[0108] (1)

[0109] in, For ships in a fixed coordinate system The derivative of the position vector of the axis. Let be the longitudinal velocity of the ship in the attached coordinate system. Let be the ship's sway linear velocity in the attached coordinate system. For ships in a fixed coordinate system The derivative of the position vector of the axis. It is the azimuth angle. The derivative of the azimuth angle. The bow roll angular velocity in the attached coordinate system;

[0110] The forces and moments acting on a ship in its three degrees of freedom include propeller thrust. (The ship's own power), hydrodynamics and torques (water resistance, lift, etc. on the ship), and the disturbance forces of the marine environment. (e.g., the forces of wind, waves, and current), with the ship's center of gravity located at the mid-longitudinal section, the Fossen rigid body dynamics model is used to describe the force and motion relationship of the ship in three degrees of freedom (swell, roll, and bow), i.e., the ship maneuvering motion model, and its calculation is shown in formula (2):

[0111] (2)

[0112] In the formula, This is the inertia matrix, which is related to the ship's linear velocity and angular acceleration. Motion state vector The time derivative; The Coriolis centripetal force matrix depends on the ship's added mass; This is the motion state vector; It is the damping coefficient matrix;

[0113] in, The calculation formula is shown in formula (3):

[0114] (3)

[0115] The calculation formula is shown in formula (4):

[0116] (4)

[0117] The calculation formula is shown in formula (5):

[0118] (5)

[0119] In the formula, Each of these represents a coefficient of hydrodynamic force.

[0120] S2) Construct mathematical models of sea wind, sea waves, and ocean currents and substitute them into the ship maneuvering motion model to finally construct a ship navigation energy consumption model in sway, roll, and bow motion.

[0121] Specifically, in step S2), the specific steps for constructing the ship's navigation energy consumption model are as follows:

[0122] When a ship is in motion, it is subject to the combined effects of various disturbances, including hydrodynamics, wind, waves, and ocean currents. These disturbances are generated by mathematical models of sea wind, waves, and currents, directly impacting the ship's navigation efficiency, maneuverability, and safety. During navigation, ships are primarily affected by three types of disturbances: hydrodynamic disturbances affect propulsion resistance through pressure distribution on the hull surface and water current eddies; wind-induced disturbances act on the part of the ship above the waterline, causing deviation from course and lateral drift; and wave disturbances cause complex movements such as heave, pitch, and roll. The combined effect of these disturbances can lead to deviations from the intended course, increased fuel consumption, impaired maneuverability, and potential navigational safety. Therefore, modern ships mitigate the impact of these disturbances through optimized hull design, the installation of anti-roll devices, and the use of advanced navigation systems, thereby improving navigation efficiency and safety.

[0123] (1) Constructing a mathematical model of sea breeze

[0124] When a ship is in motion, the forces and moments of wind loads are analyzed using relative wind speed, and a mathematical model of sea wind is constructed. The calculation is shown in formula (6):

[0125] (6)

[0126] In the formula, The vector of wind disturbance force and torque. air density, Relative wind speed, The relative wind angle of the bow. Let be the cross-sectional area of ​​the ship. This represents the projected area of ​​the ship's side. As captain, The drag coefficient is related to the smoothness of its surface, its overall shape, and its characteristic area (windward area). It is usually an experimental value, generally between 0.3 and 0.6, and is tentatively taken as 0.5.

[0127] The relative wind angle of the bow is calculated as shown in formula (7):

[0128] (7)

[0129] In the formula, For wind direction, It is the azimuth angle;

[0130] (2) Constructing a mathematical model of ocean waves

[0131] A mathematical model of ocean waves was established using Daidola's conclusions to quantify the forces exerted by waves on the heave, pitch, roll, and other motions of the ship. The mathematical model is shown in formula (8):

[0132] (8)

[0133] In the formula, The wave disturbance force and torque vector. The density of seawater, The gravitational coefficient, As captain, For amplitude, The wave angle, also known as the wave encounter angle, is the angle of the wave measured relative to the stern line. For wavelength, Wave load factor;

[0134] The wave load factor is calculated using an empirical formula and is the ratio of wave wavelength to ship length. This reflects the force characteristics of waves of different wavelengths on the longitudinal, lateral, and yaw directions of the ship, and its calculation is shown in formula (9):

[0135] (9)

[0136] (3) Constructing a mathematical model of ocean currents

[0137] When a ship travels at a constant speed in a uniform current, the current does not exert any disturbance force on the ship. However, since ocean currents are essentially the movement of water in the ocean, the disturbance force exerted by the current on the ship's motion is calculated. This is done by adding the velocity of the current relative to the ship's speed to the ship's overall speed, thus obtaining the ship's relative velocity. The projection of this relative velocity is then calculated. and A mathematical model of ocean currents is constructed, and its calculation is shown in formula (10):

[0138] (10)

[0139] In the formula This represents the longitudinal linear velocity component of the ocean current in the attached coordinate system. This represents the lateral linear velocity component of the ocean current in the attached coordinate system. This represents the longitudinal absolute velocity component of the ship. This represents the lateral absolute velocity component of the ship. This represents the longitudinal relative velocity component of the ship with respect to the ocean current. This represents the lateral relative velocity component of the ship with respect to the ocean current.

[0140] The bow roll rate of a ship is consistent regardless of whether it is relative to land or water, under uniform flow conditions, and is expressed as... The final velocity component of the ship relative to the ocean current is expressed as follows: ;

[0141] (4) Construct a model of ship navigation energy consumption.

[0142] Substituting the mathematical models of sea wind, waves, and currents into the Fossen rigid body dynamics model, the ship motion model is obtained, and the calculation of the ship motion model is shown in formula (11):

[0143] (11)

[0144] In the formula, The inertia matrix, The time derivative of the relative velocity vector of the ships. The Coriolis centripetal force matrix, Here is the damping coefficient matrix. This is the vector of the resultant force and moment acting on the ship;

[0145] Based on the thrust and torque provided by the propeller, the ship motion model is extended and deformed as shown in formula (12):

[0146] (12)

[0147] In the formula, These are the thrust or torque components in the directions of sway, roll, and pitch, respectively. For the quality of the ship; These are the additional mass derivatives in the directions of sway, roll, and pitch, respectively. These are the longitudinal and lateral relative acceleration components of the ship relative to the ocean current, respectively. These are the velocity damping derivatives in the sway, roll, and pitch directions, respectively. These are the longitudinal and lateral relative velocity components of the ship relative to the ocean current, respectively. The bow roll angular velocity; air density; Relative wind speed; These are the aerodynamic coefficients for longitudinal sway, transverse sway, and vertical sway, respectively. The relative wind angle of the bow; This represents the cross-sectional area of ​​the ship. This represents the projected area of ​​the ship's side. The density of seawater; This is the gravity coefficient; Captain; Amplitude; Wave direction angle; Wave load factor;

[0148] Finally, a model of ship navigation energy consumption under the environmental loads of sea wind, sea waves and ocean currents is constructed, as shown in formula (13):

[0149] (13)

[0150] In the formula, The total energy consumption of the unmanned vessel; These represent the energy consumed by the unmanned vessel during swaying, rolling, and bow-to-swing motions, respectively. These represent the displacement and rotation angle of the unmanned vessel during unit movement, respectively.

[0151] S3) Set constraints and design a secure reinforcement learning algorithm based on the MDP framework to achieve energy-saving optimal path planning while satisfying the constraints;

[0152] Specifically, in step S3), the specific steps for designing the security reinforcement learning algorithm are as follows:

[0153] (1) Constraints

[0154] Setting constraints is crucial for ensuring the model's practicality, addressing both energy-saving requirements and guaranteeing navigation reliability through safety mechanisms. First, ship speed constraints consider engine power limitations and navigation safety requirements. Second, heading angle constraints ensure the smoothness and feasibility of turning maneuvers. Finally, safety distance constraints prevent collisions with other vessels or fixed obstacles. The appropriate setting of these constraints directly affects the practical operability of the path planning. A schematic diagram of the ship energy-saving path planning model is shown below. Figure 2 As shown;

[0155] Three types of constraints are set: speed constraint, heading angle constraint, and safety distance constraint. This describes the ship energy-saving path planning problem and defines the "boundaries that must be met" for the path planning. This transforms the ship energy-saving path planning from a qualitative requirement into a quantitatively solvable mathematical problem, ensuring that the planning result is both energy-saving and meets the requirements of navigation safety and maneuverability. This is expressed by formula (14):

[0156] (14)

[0157] in, Indicates energy consumption. Let the energy consumption function be... For state, For a moment, and These are the lower and upper limits of ship speed, respectively. Indicates the speed during navigation. For the ship in position The heading angle at that point, and These are the lower and upper limits of the heading angle, respectively. for The ship's heading angle at any given time. express The ship's heading angle at any given time. For adjacent time periods ( arrive The maximum value of the change in heading angle at any given time. and These are the positions of the ship and the obstacles, respectively. To maintain a safe distance;

[0158] (2) Research on energy-saving path search method of security reinforcement learning

[0159] In the ship energy-saving path planning stage, a safety reinforcement learning algorithm is used to solve a specific path planning problem under constraints. This algorithm not only emphasizes learning efficiency and finding the optimal strategy but also focuses on safety during the learning process, which is particularly important for ships operating in complex and uncertain marine environments. Specifically, the safety reinforcement learning algorithm is built upon the Markov Decision Process (MDP) framework, which consists of… The composition, meaning of each component, and mathematical expression are as follows:

[0160] : A set of states that contains all possible states of the ship. ;

[0161] Action set: Contains all actions that a ship can perform. ;

[0162] : State transition probability model, representing the state transition probability model. Next action Then transition to the next state The probability of;

[0163] The mathematical expression is as shown in formula (15):

[0164] (15)

[0165] : Reward function, representing the state Next action And transition to the next state The instant reward is 1 when the safety constraint is violated (such as approaching an obstacle), and the negative reward (penalty) is obtained when the safety constraint is violated.

[0166] Time discount factor: Used to weigh the importance of immediate rewards against future rewards, with a value range of [value missing]. , The closer it is to 1, the more attention should be paid to long-term returns; the closer it is to 0, the more attention should be paid to short-term returns.

[0167] : Optimal policy, representing the optimal strategy in any state The action selection strategy that maximizes long-term cumulative reward is expressed mathematically as shown in formula (16):

[0168] (16)

[0169] in, To be optimal A function represents a state. Next action The maximum long-term cumulative reward that can be obtained later;

[0170] optimal function The state is characterized by the Bellman equation Next action To maximize long-term cumulative rewards, the optimal The mathematical expression of the function recursion formula (Bellman equation) is shown in formula (17):

[0171] (17)

[0172] in, Indicates the next state The action to be performed;

[0173] The specific implementation process of the secure reinforcement learning algorithm is as follows:

[0174] ① Reward function design:

[0175] The constraints are transformed into reward functions and penalty terms. The constraints are determined by the reward function and penalty terms. A reward function is defined and a penalty term is introduced. Negative rewards are applied to behaviors that violate safety rules (such as touching the safety distance or exceeding the heading / speed constraints) to guide the learner to learn the optimal strategy that meets the constraints.

[0176] ②State space and action space:

[0177] Introduce state space and action space;

[0178] State space: The state information of the unmanned surface vessel should be described in detail, including energy consumption, position, speed, and surrounding environment (such as the position of obstacles), to ensure that the algorithm can fully perceive the navigation scene;

[0179] Action space: Defines the actions that the unmanned surface vessel can take, such as changing course and adjusting speed, providing the algorithm with a "set of optional actions" for decision-making.

[0180] ③ Integration of safety strategies: Add a "safety verification layer", that is, design a safety constraint mechanism to ensure that the unmanned surface vessel complies with safety rules during the learning process; implement safety strategies to evaluate and modify the actions recommended by reinforcement learning in order to avoid potential dangers and ensure the compatibility of the learning process with navigation safety;

[0181] ④ Balance between exploration and exploitation: This is used to properly handle the balance between exploration (exploring new strategies) and exploitation (using known strategies), ensuring that the algorithm can both escape local optima and stably converge to an efficient path.

[0182] S4) An improved secure reinforcement learning algorithm based on ε-greedy policy and KMDP;

[0183] In step S4), the specific steps for designing and experimenting with the improved safety reinforcement learning method for ship energy-saving path planning are as follows:

[0184] Specifically, in step S4), the specific steps for improving the security reinforcement learning algorithm are as follows:

[0185] In practical applications, to improve the performance of the algorithm, traditional secure reinforcement learning methods were improved. First, an ε-greedy strategy was adopted to balance exploration and exploitation, ensuring that the algorithm can fully explore new possibilities while effectively utilizing known excellent strategies. Second, substantial improvements were made to knowledge through Markov Decision Processes (MDPs): by introducing a knowledge dimension. The construction of KMDP (Knowledge-Enhanced Markov Decision Process) breaks the assumption that MDP "model is completely known", and expands the environmental adaptability and exploration efficiency of MDP.

[0186] In KMDP, the knowledge dimension is relied upon. The model estimates in the data, combined with the knowledge dimension For the optimal The function is modified and represented as follows: The mathematical expression is shown in formula (18):

[0187] (18)

[0188] In the formula, To combine knowledge dimensions Corrected optimal function; It is a reward function that incorporates knowledge and adjusts based on model confidence (e.g., giving higher exploration rewards for unknown state-action pairs), unlike traditional MDPs which only rely on state transition reward functions. ;

[0189] S5) By building a simulation environment, setting multiple sets of experimental conditions, conducting experiments and analyzing the results, the energy-saving and safety performance is verified, and the shortcomings and directions for improvement are identified.

[0190] Specifically, in step S5), the results are verified and analyzed by setting up a simulation environment and setting multiple sets of conditions. The specific steps are as follows:

[0191] ① Environment setup

[0192] a. Environmental characteristics:

[0193] Create a simulation environment that incorporates marine features and dynamic elements to simulate complex scenarios of actual ship navigation. Real-world marine features include different types of water bodies, geographical features of the coastline, and changes in water depth. Dynamic elements such as moving ships, changing weather conditions, and water currents are also included.

[0194] b. Obstacle setting: The coastline is set as an obstacle to test the algorithm's obstacle avoidance and safe navigation capabilities;

[0195] ② Experimental setup

[0196] a. Clear experimental objectives: Clearly define the objectives of the experiment, such as evaluating the accuracy, safety, and efficiency of path planning; testing the model's performance under extreme or special conditions to assess its robustness;

[0197] b. Experimental Design:

[0198] Different experimental groups are defined to test the model's performance under various conditions, covering multiple scenarios of ship navigation, such as different starting points, destinations, and environmental conditions;

[0199] c. Experimental group design: Design control and experimental groups to compare the performance differences between the algorithm and traditional methods or different algorithm variants, and highlight the advantages of the improved algorithm through comparison;

[0200] d. Experimental conditions setup:

[0201] Set different weather and ocean conditions, such as calm, windy, and foggy; set different mission types and difficulties, such as simple straight-line navigation and complex obstacle avoidance missions.

[0202] ③ Simulation Results and Analysis

[0203] a. Results Presentation: Use graphs and tables to present experimental results in detail, such as path diagrams and statistical data on performance metrics; compare results under different experimental settings to highlight the performance advantages and limitations of the algorithm;

[0204] b. Performance Analysis: Analyze the algorithm's performance under different conditions, including success rate, path optimization degree, and time consumption; evaluate safety performance, such as obstacle avoidance ability and rule compliance.

[0205] c. Problems and Improvements: Discuss the main problems and challenges encountered in the experiment; propose possible directions for improvement and future research ideas.

[0206] The specific embodiments of the present invention are described in detail below with reference to the technical solution and accompanying drawings.

[0207] (1) Experimental design

[0208] Using ships as the simulation research object, a simulation path planning was designed to generate simulation route and energy consumption data to verify the accuracy of the method of the present invention.

[0209] In the simulation experiment, the navigation task of the agent on a 100*100 two-dimensional plane was first verified. Static circular obstacles were randomly distributed in the scene to simulate the path planning task of a ship in a simple water area. The purpose was to verify the performance of the safety reinforcement learning method. The relevant parameters are shown in Table 1.

[0210] Table 1. Parameters used in the experiment

[0211]

[0212] The generated result is as follows Figure 2As shown in the figure; the results show that the safe reinforcement learning method effectively generates smooth and efficient paths, minimizes path length and energy consumption, and avoids obstacles; the smoothness of the generated trajectory highlights the algorithm's navigation ability in complex environments, demonstrating its advanced path planning and obstacle avoidance strategies.

[0213] After verifying the advanced path planning and obstacle avoidance strategies of the safety reinforcement learning method, the experiment modified the simulation environment to simulate actual sea areas. To verify the accuracy of the invented energy-saving path planning model and learning optimization method for intelligent ship navigation, the navigation case used a historical navigation plan with the departure port of Tarifa, Spain, and the destination port of Dos Bocas, Mexico, on December 2, 2019. The simulation map range was set as follows: lon_range: [-95, 0], lat_range: [15, 50], Tarifa, Spain (Lat: 36.13175°N, Lon: 5.38043°W) and Dos Bocas, Mexico (Lat: 19.82950°N, Lon: 5.38043°W). Using a 1440*531 two-dimensional plane (91.92854°W), the model simulates a ship's path planning task in actual sea areas. Regarding the setting of obstacles and moving obstacles, scale compatibility needs to be considered. The scale of ocean route optimization typically spans hundreds to thousands of nautical miles. Since a single grid point on a map has a horizontal resolution of six to seven kilometers, and a single ship's spatial scale is 200–400 meters long (VLCC / ULCV) and tens of meters wide, which is 2–3 orders of magnitude smaller than the map grid size, it is almost a "point" on the map. Therefore, for scale compatibility considerations, dynamic obstacles (ships and floating obstacles in the actual navigation environment) were not set; only the coastline was considered as an obstacle. Simultaneously, marine meteorological data such as wind and waves from historical voyage plans (departure port: Tarifa, Spain; destination port: Dos Bocas, Mexico; departure time: December 2, 2019) were considered, and combined with the constructed ship energy-saving path planning model to simulate real voyage plans and verify the performance of the invention in the simulated actual sea environment. Relevant parameters are shown in Table 2.

[0214] Table 2 Parameters used in the experiment

[0215]

[0216] Experimental results are as follows Figure 3As shown in the test, the optimal search effect was achieved in rounds 3801-3900, with the path converging to the target point, effectively generating a smooth and efficient path. Furthermore, compared with the path parameters generated by the historical voyage plan (departure port: Tarifa, Spain; destination port: Dos Bocas, Mexico; departure time: December 2, 2019), as shown in Table 3, the optimized route (selecting the energy-efficient route within the track zone) resulted in a longer voyage than the actual route, increasing by 195.63 nautical miles; the optimized route also increased the voyage time by 15.09 hours. However, in terms of energy consumption, the optimized route resulted in a lower energy consumption than the actual route, decreasing by approximately 145.2 tons, thus meeting the energy-optimization objective of the method in this invention. The main reason why the optimized route increases the distance but reduces fuel consumption compared to the actual route is that the ship's optimized route, which is loaded with meteorological data in real time, takes navigation safety and the smooth completion of the voyage mission as the premise, and searches for marine meteorological conditions that are conducive to saving fuel in navigable waters as much as possible, such as downwind, waves or stern-sloping winds and waves.

[0217] Table 3 Comparison of Optimized and Actual Route Data

[0218]

[0219] The unmodified secure reinforcement learning method was tested using the same parameters, and the experimental results are as follows: Figure 4 As shown, after completing 900,000 steps under the same parameters, the path did not completely converge to the target point, and some paths collided with obstacles. Compared with the unimproved safety reinforcement learning method, the test results of the improved safety reinforcement learning method show that its path planning and obstacle avoidance strategies are better than those of the unimproved safety reinforcement learning method.

[0220] Although preferred embodiments of this application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of this application.

[0221] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. An energy-saving path planning and learning optimization method for intelligent ship navigation, characterized in that, include: S1) Based on the motion of the three degrees of freedom of sway, roll and bow, a ship maneuvering motion model is constructed by mapping the kinematic parameters between the fixed coordinate system and the attached coordinate system, combined with the Fossen rigid body dynamics model. S2) Construct mathematical models of sea wind, sea waves, and ocean currents and substitute them into the ship maneuvering motion model to finally construct a ship navigation energy consumption model in sway, roll, and bow motion. S3) Set constraints and design a secure reinforcement learning algorithm based on the MDP framework; The specific steps for designing a secure reinforcement learning algorithm are as follows: (1) Constraints Set speed constraints The three types of constraints—including safety distance constraints—define the "boundaries that must be satisfied" for path planning, expressed by formula (14): （14） in, Indicates energy consumption. Let the energy consumption function be... For state, For a moment, and These are the lower and upper limits of ship speed, respectively. Indicates the speed during navigation. For the ship in position The heading angle at that point, and These are the lower and upper limits of the heading angle, respectively. for The ship's heading angle at any given time. express The ship's heading angle at any given time. This represents the maximum change in heading angle between adjacent time points. and These are the positions of the ship and the obstacles, respectively. To maintain a safe distance; (2) Design of a security reinforcement learning algorithm based on the MDP framework The secure reinforcement learning algorithm is built on the Markov Decision Process (MDP) framework, in which the MDP framework consists of... The composition, meaning of each component, and mathematical expression are as follows: : A set of states that contains all possible states of the ship. ; Action set: Contains all actions that a ship can perform. ; : State transition probability model, representing the state transition probability model. Next action Then transition to the next state The probability of; Calculate the next state The probability of is calculated as shown in formula (15): （15） : Reward function, representing the state Next action And transition to the next state The instant reward is 1 for violating security constraints, and a negative reward is obtained for violating security constraints. Time discount factor, with a value range of: , The closer it is to 1, the more attention should be paid to long-term returns; the closer it is to 0, the more attention should be paid to short-term returns. : Optimal policy, representing the optimal strategy in any state The action selection strategy that maximizes long-term cumulative reward is expressed mathematically as shown in formula (16): （16） in, To be optimal A function represents a state. Next action The maximum long-term cumulative reward that can be obtained later; optimal function The state is characterized by the Bellman equation Next action To maximize long-term cumulative rewards, the optimal The mathematical expression of the Bellman equation is shown in formula (17): （17） in, Indicates the next state The action to be performed; S4) An improved secure reinforcement learning algorithm based on ε-greedy policy and KMDP; S5) Verify and analyze the results by building a simulation environment and setting multiple sets of conditions.

2. The energy-saving path planning and learning optimization method for intelligent ship navigation according to claim 1, characterized in that, In step S1), the specific process of constructing the ship maneuvering motion model is as follows: Based on the state of a ship undergoing arbitrary degrees of freedom of motion, two coordinate systems are established: a fixed coordinate system and an attached coordinate system. The ship's motion generates six degrees of freedom: pitch, sway, bow roll, heave, roll, and pitch. Based on the ship's motion in the horizontal plane, the motion components of heel, roll, and pitch are simplified, and the heel angle is set. pitch angle heave linear velocity Pitch angular velocity Roll angular velocity Only the three degrees of freedom of sway, roll, and pitch are analyzed. The kinematic parameter relationship between the attached coordinate system and the inertial coordinate system is calculated, that is, the kinematic parameter transformation relationship between the fixed coordinate system and the attached coordinate system. The calculation is shown in formula (1): （1） in, For ships in a fixed coordinate system The derivative of the position vector of the axis. Let be the longitudinal velocity of the ship in the attached coordinate system. Let be the ship's sway linear velocity in the attached coordinate system. For ships in a fixed coordinate system The derivative of the position vector of the axis. It is the azimuth angle. The derivative of the azimuth angle. The bow roll angular velocity in the attached coordinate system; The forces and moments acting on a ship in its three degrees of freedom include propeller thrust. Hydrodynamics and torques, as well as disturbance forces from the marine environment. The ship's center of gravity is located at the mid-longitudinal section. The Fossen rigid body dynamics model is used to describe the ship's forces and motions in three degrees of freedom, i.e., the ship's maneuvering motion model. Its calculation is shown in formula (2): （2） In the formula, This is the inertia matrix, which is related to the ship's linear velocity and angular acceleration. Motion state vector The time derivative; The Coriolis centripetal force matrix depends on the ship's added mass; This is the motion state vector; It is the damping coefficient matrix; in, The calculation formula is shown in formula (3): （3） The calculation formula is shown in formula (4): （4） The calculation formula is shown in formula (5): （5） In the formula, Each of these represents a coefficient of hydrodynamic force.

3. The energy-saving path planning and learning optimization method for intelligent ship navigation according to claim 1, characterized in that, In step S2), the specific steps for constructing the ship navigation energy consumption model are as follows: (1) Constructing a mathematical model of sea breeze When a ship is in motion, the forces and moments of wind loads are analyzed using relative wind speed, and a mathematical model of sea wind is constructed. The calculation is shown in formula (6): （6） In the formula, The vector of wind disturbance force and torque. air density, Relative wind speed, The relative wind angle of the bow. Let be the cross-sectional area of ​​the ship. This represents the projected area of ​​the ship's side. As captain, The air drag coefficient is related to the smoothness of its surface, its overall shape, and its characteristic area. The relative wind angle of the bow is calculated as shown in formula (7): （7） In the formula, For wind direction, It is the azimuth angle; (2) Constructing a mathematical model of ocean waves A mathematical model of ocean waves was established using Daidola's conclusions to quantify the force exerted by waves on the heave, pitch, and roll motions of the ship. The mathematical model is shown in formula (8): （8） In the formula, The wave disturbance force and torque vector. The density of seawater, The gravitational coefficient, As captain, For amplitude, The wave angle, also known as the wave encounter angle, is the angle of the wave calculated with the stern line as the reference. For wavelength, Wave load factor; The wave load factor is calculated using an empirical formula, specifically the ratio of wave wavelength to ship length. This reflects the force characteristics of waves of different wavelengths on the longitudinal, lateral, and yaw directions of the ship, and its calculation is shown in formula (9): （9） (3) Constructing a mathematical model of ocean currents Based on the disturbance force exerted by ocean currents on the ship's motion, the velocity generated by the ocean current on the ship's motion is superimposed on the ship's speed to obtain the ship's relative velocity to the ocean current; the projection of the ship's relative velocity to the ocean current is then calculated. and A mathematical model of ocean currents is constructed, and its calculation is shown in formula (10): （10） In the formula This represents the longitudinal linear velocity component of the ocean current in the attached coordinate system. This represents the lateral linear velocity component of the ocean current in the attached coordinate system. This represents the longitudinal absolute velocity component of the ship. This represents the lateral absolute velocity component of the ship. This represents the longitudinal relative velocity component of the ship with respect to the ocean current. This represents the lateral relative velocity component of the ship with respect to the ocean current. Indicates the bow roll rate of the ship; Finally, the velocity components of the ship relative to the ocean current are expressed as follows: ; (4) Construct a model of ship navigation energy consumption. Substituting the sea breeze mathematical model, the sea wave mathematical model, and the ocean current mathematical model into the ship maneuvering motion model, the ship motion model is obtained, and the calculation of the ship motion model is shown in formula (11): （11） In the formula, The inertia matrix, The time derivative of the relative velocity vector of the ships. The Coriolis centripetal force matrix, Here is the damping coefficient matrix. This is the vector of the resultant force and moment acting on the ship; Based on the thrust and torque provided by the propeller, the ship motion model is extended and deformed as shown in formula (12): （12） In the formula, These are the thrust or torque components in the directions of sway, roll, and pitch, respectively. For the quality of the ship; These are the additional mass derivatives in the directions of sway, roll, and pitch, respectively. These are the longitudinal and lateral relative acceleration components of the ship relative to the ocean current, respectively. These are the velocity damping derivatives in the sway, roll, and pitch directions, respectively. These are the longitudinal and lateral relative velocity components of the ship relative to the ocean current, respectively. The bow roll angular velocity; air density; Relative wind speed; These are the aerodynamic coefficients for longitudinal sway, transverse sway, and vertical sway, respectively. The relative wind angle of the bow; This represents the cross-sectional area of ​​the ship. This represents the projected area of ​​the ship's side. The density of seawater; It is the gravity coefficient; Captain; Amplitude; Wave direction angle; Wave load factor; Finally, a model of ship navigation energy consumption under the environmental loads of sea wind, sea waves and ocean currents is constructed, as shown in formula (13): （13） In the formula, The total energy consumption of the unmanned vessel; These represent the energy consumed by the unmanned vessel during swaying, rolling, and bow-to-swing motions, respectively. These represent the displacement and rotation angle of the unmanned vessel during unit movement, respectively.

4. The energy-saving path planning and learning optimization method for intelligent ship navigation according to claim 1, characterized in that, In step S4), the specific steps for improving the security reinforcement learning algorithm are as follows: First, an ε-greedy strategy is employed to balance the relationship between exploration and exploitation. Second, the knowledge dimension is improved through Markov decision processes, specifically by introducing a knowledge dimension. By constructing KMDP, the assumption that "the model of MDP is completely known" is broken; In KMDP, the knowledge dimension is relied upon. The model estimates in the data, combined with knowledge dimensions For the optimal The function is modified and represented as follows: The mathematical expression is shown in formula (18): （18） In the formula, To combine knowledge dimensions Corrected optimal function; It incorporates a knowledge-based reward function, which adjusts based on model confidence, unlike traditional MDPs that rely solely on state transition reward functions. .

5. The energy-saving path planning and learning optimization method for intelligent ship navigation according to claim 1, characterized in that, In step S5), the simulation environment is set up, multiple sets of conditions are set up, and the results are verified and analyzed. The specific steps are as follows: ① Environment setup a. Environmental characteristics: Create a simulation environment that incorporates marine features and dynamic elements to simulate complex scenarios of actual ship navigation; b. Obstacle setting: The coastline is set as an obstacle to test the algorithm's obstacle avoidance and safe navigation capabilities; ② Condition settings Set different weather and ocean conditions, and set different mission types and difficulties; ③ Simulation Results and Analysis a. Results Presentation: Use graphs and tables to present the results in detail, compare and verify the results under different experimental settings, and highlight the performance advantages and limitations of the algorithm; b. Performance analysis: Analyze the algorithm's performance under different conditions and evaluate its security performance.

Images