Unmanned ship path tracking and obstacle avoidance method based on reinforcement learning guided MPC

Abstract

The invention discloses an unmanned ship path tracking and obstacle avoidance method based on reinforcement learning guided MPC. The method belongs to the technical field of ship automatic control, and comprises the following steps: estimating a system state and unknown environment ocean current interference in real time by using EnKF, and feeding forward to a prediction model; the planning layer generates a global optimal reference path by using an RRT * algorithm in combination with a B-spline smoothing technology; the decision-making layer adopts a PPO algorithm, outputs a reference speed and course correction bias according to a real-time observation state, and realizes dynamic adjustment of a global path; and the control layer constructs an MPC optimization problem containing a COLREGs regular potential field and dynamic constraints, and receives the corrected reference signal to solve the optimal thrust and torque. According to the method, the environmental adaptability of reinforcement learning and the constraint processing capacity of MPC are combined, the problem that a traditional method is prone to falling into a local minimum value or violating a navigation rule under strong nonlinearity, dynamic obstacles and environmental interference is effectively solved, and the obstacle avoidance success rate, tracking precision and control smoothness of USV are remarkably improved.

Description

Technical Field

[0001] This invention belongs to the field of autonomous control technology for unmanned surface vessels (USVs), and relates to a path tracking and obstacle avoidance method for USVs based on reinforcement learning-guided MPC (a path tracking and obstacle avoidance method for USVs based on reinforcement learning-guided model predictive control in complex environments); specifically, it relates to a hierarchical path tracking and obstacle avoidance control method for USVs that combines global path planning, deep reinforcement learning decision-making, and model predictive control in environments with strong wind and flow interference and complex dynamic obstacles. Background Technology

[0002] With the rapid development of the marine economy and the expansion of the blue ocean, unmanned surface vessels (USVs) have been widely used in marine resource exploration, environmental monitoring, and surface security due to their significant advantages such as low cost, high operational efficiency, and no risk of personnel injury. However, the marine environment is highly unstructured. When performing missions, USVs not only face strong nonlinear hydrodynamic interference such as wind, waves, and currents, but also need to achieve autonomous obstacle avoidance in complex environments containing dynamic vessels and static obstacles in accordance with the International Regulations for Preventing Collisions at Sea (COLREGs). How to achieve efficient, safe, and smooth autonomous navigation under multiple constraints has become a key technical bottleneck restricting the improvement of the intelligence level of USVs.

[0003] Existing obstacle avoidance and control methods for USVs mainly include traditional geometric or potential field methods, model predictive control (MPC), and reinforcement learning (RL) methods that have emerged in recent years. However, they all have limitations that are difficult to reconcile in practical applications. Early artificial potential field (APF) or velocity obstacle (VO) methods, although computationally simple, often ignore the large inertia and underactuated dynamic characteristics of ships, making it difficult to accurately track the planned path. Moreover, they are prone to getting stuck in local minima and causing navigation to stall under complex obstacle distributions. Although model predictive control (MPC) can explicitly handle dynamic hard constraints, as a local optimization algorithm, it lacks a global navigation vision and is prone to deadlock when facing U-shaped traps or dense obstacle groups. At the same time, traditional MPC usually uses fixed prediction models and weight parameters, and cannot adaptively adjust the control strategy according to the degree of congestion or risk level of the environment, often resulting in the system exhibiting overly conservative or slow-responding phenomena in dynamic scenarios.

[0004] On the other hand, while end-to-end deep reinforcement learning methods possess strong environmental adaptability, their "black box" nature often results in a lack of smoothness in control actions, easily leading to high-frequency oscillations in actuators, and making it difficult to theoretically guarantee navigation safety and compliance with regulations. Furthermore, most existing control schemes, when dealing with environmental disturbances, often fail to explicitly estimate and feedforward time-varying ocean currents, causing USVs to be highly susceptible to track drift in heavily disturbed waters, further exacerbating collision risks. In summary, existing technologies struggle to simultaneously address global path optimization, adaptive decision-making in dynamic environments, underlying dynamic constraints, and robust control under strong disturbances. Therefore, a control architecture that integrates the advantages of multiple layers is urgently needed to solve these problems. Summary of the Invention

[0005] Purpose of the invention: The purpose of this invention is to provide a path tracking and obstacle avoidance method for unmanned surface vessels (USVs) based on reinforcement learning-guided model predictive control in complex environments that can solve the problems of existing single control methods easily getting trapped in local minima, struggling to balance global optimization and dynamic obstacle avoidance, and failing to strictly comply with international maritime collision avoidance rules in environments with strong nonlinear disturbances and dense obstacles, and significantly improve the success rate and control smoothness of obstacle avoidance.

[0006] The technical solution of this invention is: a path tracking and obstacle avoidance method for unmanned surface vessels based on reinforcement learning-guided MPC, comprising the following steps: Step (1) Environmental perception and disturbance estimation; Step (2) Global path planning; Step (3) PPO adaptive decision-making; Step (4) Reference state fusion; Step (5) MPC optimization control; Step (6) is executed in a closed loop.

[0007] Furthermore, the environmental perception and disturbance estimation mentioned in step (1) specifically involves: acquiring sensor data of the unmanned surface vessel, establishing a state observer based on ensemble Kalman filtering (EnKF), reconstructing the motion state of the unmanned surface vessel in real time, and estimating unknown ocean current disturbances in the environment; The specific implementation of the state observer using ensemble Kalman filtering (EnKF) includes: defining the augmented state vector: In the formula, This represents the position and heading in the geodetic coordinate system. For the longitudinal velocity, lateral velocity, and bow roll angular velocity in the attached coordinate system, The ocean current velocity vector in the inertial frame is obtained by... The statistical characteristics of each set member approximate the state distribution, avoiding the calculation of the Jacobian matrix. In the prediction step, the state estimate from the previous time step is substituted into a 3-DOF kinematic model that includes nonlinear damping and ocean current disturbances for time updating. In the update step, the Kalman gain is calculated using the observations and the set members are updated. The final output is the state estimate. and ocean current estimates For the set mean, and Feedforward to the MPC prediction model in step (5).

[0008] Furthermore, the global path planning described in step (2) specifically involves: using the asymptotically optimal fast expanding random tree algorithm to search and generate a global collision-free path in a known static environment map, and using B-spline technology to smooth the path to obtain a global nominal reference path.

[0009] Furthermore, the PPO adaptive decision-making in step (3) specifically involves: constructing a PPO reinforcement learning agent, inputting the current observation state of the unmanned surface vessel into the agent, and obtaining the action output of the PPO reinforcement learning agent. Specifically, this includes the reference speed correction offset and the reference heading correction offset for the reference path; The current observation status of the unmanned surface vessel. Includes the following seven components: target distance Heading error Current speed status of the unmanned surface vessel Distance to dynamic obstacles Lateral tracking error Heading alignment error And reference path point index; The action output of the PPO reinforcement learning agent The definition of is: In the formula, The correction bias for the reference speed. The correction offset for the reference heading; The PPO reinforcement learning agent does not directly output rudder angle or thrust control values.

[0010] Furthermore, the constructed PPO reinforcement learning agent is trained using a composite reward function; Among them, the composite reward function include: (1) Task Rewards The distance to the target is proportional to the reduction in target distance and the heading alignment. (2) Safety Rewards : Exponential function based on distance to obstacles The closer the distance, the more exponentially the penalty increases; (3) Rules and Rewards A negative reward is given when a violation of COLREGs rules is detected. (4) Energy consumption reward Punish excessively large movements and shocks.

[0011] Furthermore, the reference state fusion described in step (4) specifically involves: establishing a reference state fusion mechanism, superimposing the global nominal reference path, the corrected bias of the PPO output, and the rule bias based on COLREGs rules to generate the final reference state trajectory input to the underlying controller. The specific calculation formula for the reference state fusion mechanism is as follows: In the formula, For final reference state, For the global nominal reference path generated in step (2), The action vector output by PPO in step (3) This is the PPO bias mapping matrix. For rule biases based on COLREGs, This is the regular bias mapping matrix.

[0012] Furthermore, the rule bias based on COLREGs The generation logic is: determine the current encounter scenario between the unmanned surface vessel and dynamic obstacles; If the scenario is determined to be an encounter or a starboard crossing, then set... , The reference heading is forced to deflect to the right by a preset right turn guidance coefficient; If the scenario is otherwise, then set This allows for the construction of an asymmetric rule-guided mechanism, breaking the symmetry of the collision avoidance potential field.

[0013] Furthermore, the MPC optimization control mentioned in step (5) specifically involves: taking the final reference state trajectory obtained in step (4) as the tracking target, combining the ocean current interference estimated in step (1), constructing an (MPC) optimization problem containing dynamic constraints and collision avoidance rule potential field, and solving it to obtain the optimal longitudinal thrust and steering torque command of the unmanned surface vessel. Wherein, the cost function of the MPC optimization problem Includes a weighted sum of four terms: In the formula, To track costs, the predicted state is used to minimize the difference between the predicted state and the final reference state generated in step (4). Deviation between; To control input costs, penalties are imposed on the rate of change of thrust and rudder angle to prevent high-frequency oscillations in the actuators; This is a static safety cost, used to maintain a safe distance from obstacles; The cost is the regular potential field cost, which is constructed based on the relative azimuth angle of the unmanned surface vessel relative to dynamic obstacles. A high penalty is imposed when the relative azimuth angle is in the danger sector defined by COLREGs.

[0014] Furthermore, the closed-loop execution in step (6) specifically involves sending control commands to the actuator to drive the unmanned surface vessel to move, and then returning to step (1) to perform the next cycle.

[0015] Furthermore, an unmanned surface vessel control system for implementing the method specifically includes: (1) Sensing module: including lidar, GPS positioning unit, inertial measurement unit (IMU), used to collect environmental and motion data; (2) Computation module: includes three parallel processing units, which are used to run the EnKF state observer, PPO deep neural network inference and MPC nonlinear optimization solver respectively; (3) Storage module: Stores pre-trained PPO network weight parameters and global nautical chart data; (4) Low-level execution module: Receives the thrust and torque commands output by the calculation module and converts them into the rotational speed commands of the left and right thrusters through the thrust allocation matrix.

[0016] Furthermore, the dynamic model of the unmanned surface vessel used in this invention includes linear damping terms, quadratic nonlinear damping terms, and cubic nonlinear damping terms. Wherein, the third nonlinear damping term This is used to characterize the higher-order hydrodynamic drag of unmanned surface vessels under high-speed maneuvers, and its specific forms include the longitudinal third-order damping coefficient. and bow roll third damping coefficient ; Its mathematical expression is as follows: ; Specific damping matrix Expanded to: In the formula, The inertia matrix, The Coriolis centripetal force matrix, This is the velocity vector in the attached coordinate system. To control the input, For environmental interference; It is a cubic nonlinear damping coefficient used to characterize high-order hydrodynamic drag under high-speed maneuvering.

[0017] Beneficial Effects: Compared with existing technologies, this invention has the following significant advantages: 1. This invention proposes a hierarchical control architecture that integrates global planning using the progressive optimal fast expanding random tree algorithm and PPO reinforcement learning guidance. The upper layer utilizes the progressive optimal fast expanding random tree algorithm to provide global collision-free geometric guidance, solving the problem that traditional local methods are prone to getting trapped in local minima when facing U-shaped traps or dense obstacles. At the same time, the PPO agent outputs a reference bias based on the real-time dynamic environment, making up for the lag in global planning response to dynamic obstacles, and realizing the complementary advantages of macro-planning and micro-decision-making. 2. This invention introduces asymmetric rule bias and penalty potential field based on COLREGs in the bottom-level model predictive control. By designing a dual mechanism of penalty potential field plus guidance bias, the symmetry of the traditional potential field method is broken, forcing the unmanned surface vessel to perform right-turn avoidance behavior in encounter and intersection scenarios, effectively solving the problem of decision swinging in the rule ambiguity zone of existing algorithms, and ensuring the compliance and safety of navigation. 3. This invention introduces an environmental perception and interference compensation mechanism based on EnKF and establishes a high-order dynamic model containing a cubic nonlinear damping term. EnKF can estimate and feedforward compensate for unknown ocean current disturbances in real time. Combined with a high-order damping model, it significantly improves the path tracking accuracy and anti-drift capability of unmanned surface vessels in high-speed navigation and strong current environments. 4. This invention adopts a PPO adjustment of reference values ​​instead of directly controlling the rudder angle, combined with the dynamic constraint processing capability of the underlying MPC. This design not only gives the system a flexible decision-making ability similar to a human driver's "risk perception and early deceleration", but also strictly limits the rate of change of thrust and rudder angle through MPC, eliminating the control oscillation phenomenon common in pure end-to-end reinforcement learning, and achieving a smooth, energy-saving and safe control effect. Attached Figure Description

[0018] Figure 1 This is a flowchart illustrating the overall logic of the method provided in this embodiment of the invention. Figure 2 This is a schematic diagram illustrating the definitions of the geodetic coordinate system and the attached coordinate system used in this invention; Figure 3 This is a flowchart of the overall architecture of the hierarchical control system proposed in this invention, which integrates global planning using the progressive optimal fast expanding random tree algorithm, PPO reinforcement learning decision-making, and MPC model predictive control. Figure 4 This is a comparison of the average reward convergence curves of the PPO algorithm and the SAC algorithm during the training process in this invention; Figure 5 This is a comparison chart of the trajectory tracking and obstacle avoidance path performance of the method of this invention and existing methods (pure MPC, pure RL, etc.) in a dense obstacle environment; Figure 6This is a time-domain response curve of the key state and control variables (including yaw rate, lateral control force, longitudinal speed and distance to the nearest obstacle) of the method of the present invention during navigation. Figure 7 This is a convergence curve of the estimation error of the EnKF observer in this invention for ocean current disturbances in an unknown environment; Figure 8 This is a bar chart comparing the statistical performance indicators (including collision rate, average flight time, energy consumption, and number of replanning attempts) of the method of this invention with existing methods in random test scenarios. Detailed Implementation

[0019] The specific technical solution of the present invention will be further described in detail below with reference to specific examples.

[0020] I. Dynamics Modeling of Unmanned Surface Vessels This embodiment uses an underactuated unmanned surface vessel (USV) as the controlled object; such as Figure 2 As shown, the geodetic coordinate system (NED) and the attached coordinate system are defined; Step S1: Construct a kinematic and dynamic model including high-order nonlinear damping Considering the wind and current interference in the marine environment and the characteristics of high-speed navigation, the following mathematical model is adopted in this embodiment: 1. Kinematic model: In the formula, This indicates the USV's position (ordinate and abscissa) and bow angle in the geodetic coordinate system. Represents the rotation transformation matrix; , representing the longitudinal velocity, lateral velocity, and yaw rate in the attached coordinate system; , representing the ocean current velocity vector in the geodetic coordinate system; 2. Dynamic equations: In the formula, The inertia matrix includes the added mass; The Coriolis centripetal force matrix; The control input vector is obtained by mapping from the dual-thruster thrust model. This is the environmental disturbance vector; In particular, to accurately describe the hydrodynamic characteristics at high speeds, this embodiment constructs a nonlinear damping matrix including cubic terms. : In the formula, These are the linear damping coefficients in the longitudinal, lateral, and yaw directions, respectively; These are the second-order nonlinear damping coefficients in the corresponding directions; It is a cubic nonlinear damping coefficient used to characterize high-order hydrodynamic drag under high-speed maneuvering; Indicates the bow roll rate The coupling damping coefficient for lateral motion. Indicates lateral velocity Coupling damping coefficient for lateral motion; II. Implementation Process of Hierarchical Control System like Figure 1 As shown, the method flow of this invention includes a complete loop from environmental perception to closed-loop execution; the system first performs environmental perception and disturbance estimation, then sequentially goes through global planning, adaptive decision-making, state fusion and model predictive control, and finally executes motion; after executing single-step control, the system performs safety detection and target determination: if the target is not reached and there is no collision, the process will jump back to the first step through the feedback loop and enter the control loop of the next moment; the specific implementation steps are as follows: Step S2: Environmental perception and disturbance estimation based on EnKF In view of the characteristics of high sensor noise and unknown ocean currents in the marine environment, this embodiment uses ensemble Kalman filtering (EnKF) for state reconstruction; (1) Initialization: Define augmented state vector ,initialization The set members have an initial state that follows a Gaussian distribution; (2) Prediction step: For each set member Substituting the nonlinear discrete model from step (1) into the derivation, we obtain the prior estimate set. This process does not require calculating the complex Jacobian matrix; (3) Calibration step: Acquire GPS and IMU observations Calculate Kalman gain The state of each set member is corrected using the observations; (4) Output and Feedforward: Calculate the set mean as the best estimate for the current time. Among them, the estimated ocean current velocity This will be directly used as a known disturbance term, fed forward into the underlying MPC prediction model to achieve real-time compensation for environmental drift (the effect is as follows). Figure 7 As shown, the error quickly converges to near 0). Step S3: Global path planning based on the asymptotically optimal fast expanding random tree algorithm Given the static nautical chart information, the planning layer is responsible for generating a collision-free geometric path; (1) Path search: Using the RRT* (Rapidly-exploring Random Tree Star) algorithm, a random tree is grown from the starting point as the root node to the target point; during the expansion process, a rewire mechanism is introduced to optimize the path cost and generate an asymptotically optimal node sequence. (2) Path smoothing: Since the path of the original asymptotically optimal fast expanding random tree algorithm is composed of broken line segments, it does not meet the USV kinematic constraints; in this embodiment, B-spline curves are used to fit the path nodes to generate a continuous and differentiable global nominal reference path. and the corresponding reference speed ; Step S4: Adaptive decision-making based on PPO (policy bias generation) The decision-making layer does not directly control the USV, but adjusts the global policy based on the local dynamic environment; the agent is trained using the Proximal Policy Optimization (PPO) algorithm; (1) State space: observation vector Includes 7-dimensional information: target distance Heading error Current speed Closest distance to dynamic obstacles lateral deviation , heading alignment and path point index; (2) Action Space: Output action vector These correspond to the correction amounts for reference speed (e.g., deceleration commands) and reference heading (e.g., avoidance angles), respectively; the range is limited to a reasonable interval (e.g., speed correction). ); 3. Reward function design: (1) Task Rewards Encourage USVs to approach the target point; (2) Safety Rewards Based on exponential function When the USV approaches an obstacle, it is given a sharp increase in negative reward, forcing the agent to learn to "avoid in advance"; (3) Rules and Rewards If a violation of COLREGs is detected (such as seizing a passageway), a penalty will be imposed. (4) Energy consumption reward Punishment involves drastic changes in the range of motion, encouraging smoother output; Step S5: Multi-source reference state fusion and rule embedding (a key link connecting upper-level decision-making and lower-level control) (1) Rule bias generation: Based on the relative position and speed of the current USV and the obstacle vessel, determine the encounter scenario (face-to-face, cross, overtaking). If determined to be a head-on or starboard-crossing: generate an asymmetric rule offset (e.g.), forcing the reference heading to shift to the right; For other scenarios: ; Final reference state calculation: ; This formula achieves the following: tracking the RRT* path when there are no obstacles; and overlaying the PPO's intelligent avoidance bias and COLREGs' rule bias when there are dynamic obstacles. Step S6: MPC Optimization Solution and Control Execution (Constructing an optimization problem in the finite time domain at the lower control layer) (1) Cost function : In the formula, It is a potential field penalty term based on the relative azimuth angle; (2) Constraints: Dynamic constraints: satisfy the discretized model in step S1; Input constraints: (Thrust limitation) (Change rate limit, protecting the implementing agency); Safety constraints: Distance from obstacles ; (3) Solution: The above non-convex optimization problem is solved using the SQP (Sequence Quadratic Programming) algorithm to obtain the optimal control sequence. The first control variable is applied to the USV; III. Simulation Results and Verification To verify the effectiveness of this embodiment, a high-fidelity simulation environment was built on the MATLAB platform; the scenario was set as follows. The sea area contains randomly distributed static obstacles and dynamically disruptive vessels, and is superimposed with... The time-varying ocean currents; (1) Training convergence: such as Figure 4 As shown, the PPO algorithm converges after about 5000 rounds, exhibiting a more stable average reward than the SAC algorithm. (2) Tracking effect: such as Figure 5 As shown, in areas with dense obstacles, traditional Pure MPC is prone to getting stuck in local minima, leading to deadlock; Pure RL path jitter is severe; while the path generated by this embodiment (PPO+MPC) is both smooth and safe, with a success rate of 100%. (3) Critical state response: such as Figure 6 As shown, this method automatically outputs a deceleration bias when approaching an obstacle. Figure 6 c), the underlying MPC ensures smooth rudder angle changes ( Figure 6 a) No high-frequency oscillations were observed; (4) Statistical indicators: as shown in Table 1 and Figure 8As shown, compared with traditional methods, the collision rate of this embodiment is reduced to 5%, and the average energy consumption is the lowest while ensuring safety, achieving the optimal balance between safety and economy.

[0021] Table 1. Comparison of statistical performance of different control strategies in 100 random scenarios

[0022] As shown in Table 1, the present invention (PG-MPC) reduces the collision risk from 11.0% of the traditional MPC to 5.0% while ensuring a 100% mission success rate. In terms of energy consumption, although it is higher than the pure reinforcement learning method, it reduces energy consumption by about 17.7% compared with the model-based MPC method, and has the shortest average flight time, proving that the present invention achieves the optimal balance between safety and efficiency.

Claims

1. A path tracking and obstacle avoidance method for unmanned surface vessels based on reinforcement learning-guided MPC, characterized in that, Includes the following steps: Step (1) Environmental perception and disturbance estimation; Step (2) Global path planning; Step (3) PPO adaptive decision-making; Step (4) Reference state fusion; Step (5) MPC optimization control; Step (6) is executed in a closed loop.

2. The unmanned surface vessel path tracking and obstacle avoidance method based on reinforcement learning-guided MPC according to claim 1, characterized in that, The environmental perception and disturbance estimation mentioned in step (1) specifically involves: acquiring sensor data from the unmanned surface vessel, establishing a state observer based on ensemble Kalman filtering, reconstructing the motion state of the unmanned surface vessel in real time, and estimating unknown ocean current disturbances in the environment; The specific implementation of the state observer using ensemble Kalman filtering includes: defining the augmented state vector: In the formula, This represents the position and heading in the geodetic coordinate system. For the longitudinal velocity, lateral velocity, and bow roll angular velocity in the attached coordinate system, Let be the ocean current velocity vector in the inertial frame.

3. The unmanned surface vessel path tracking and obstacle avoidance method based on reinforcement learning-guided MPC according to claim 1, characterized in that, The global path planning described in step (2) is as follows: In the known static environment map, the asymptotic optimal fast expanding random tree algorithm is used to search and generate a global collision-free path, and the B-spline technique is used to smooth the path to obtain a global nominal reference path.

4. The unmanned surface vessel path tracking and obstacle avoidance method based on reinforcement learning-guided MPC according to claim 1, characterized in that, The PPO adaptive decision-making process described in step (3) specifically involves: constructing a PPO reinforcement learning agent to process the current observation state of the unmanned surface vessel. The input is given to the agent, and the action output of the PPO reinforcement learning agent is obtained. Specifically, this includes the reference speed correction offset and the reference heading correction offset for the reference path; The current observation status of the unmanned surface vessel. It includes the following seven components: target distance Heading error Current speed status of the unmanned surface vessel Distance to dynamic obstacles Lateral tracking error Heading alignment error and reference path point index; The action output of the PPO reinforcement learning agent The definition of is: In the formula, The correction bias for the reference speed. The offset is the correction amount for the reference heading.

5. The unmanned surface vessel path tracking and obstacle avoidance method based on reinforcement learning-guided MPC according to claim 4, characterized in that, The constructed PPO reinforcement learning agent is trained using a composite reward function; Wherein, the composite reward function include: (1) Task Rewards The distance to the target is proportional to the reduction in target distance and the heading alignment. (2) Safety Rewards : An exponential function based on the distance to obstacles The closer the distance, the more exponentially the penalty increases; (3) Rules and Rewards A negative reward is given when a violation of COLREGs rules is detected. (4) Energy consumption reward Punish excessively large movements and shocks.

6. The unmanned surface vessel path tracking and obstacle avoidance method based on reinforcement learning-guided MPC according to claim 1, characterized in that, The reference state fusion described in step (4) specifically involves: establishing a reference state fusion mechanism, superimposing the global nominal reference path, the corrected bias of the PPO output, and the rule bias based on COLREGs rules to generate the final reference state trajectory input to the underlying controller. The calculation formula for the reference state fusion mechanism is as follows: ; In the formula, For final reference state, For the global nominal reference path generated in step (2), The action vector output by PPO in step (3) This is the PPO bias mapping matrix. For rule biases based on COLREGs, This is the regular bias mapping matrix.

7. The unmanned surface vessel path tracking and obstacle avoidance method based on reinforcement learning-guided MPC according to claim 6, characterized in that, The rule bias based on COLREGs The generation logic is: determine the current encounter scenario between the unmanned surface vessel and dynamic obstacles; If the scenario is determined to be an encounter or a starboard crossing, then set... , The reference heading is forced to deflect to the right by the preset right turn guidance coefficient; If the scenario is otherwise, then set This allows for the construction of an asymmetric rule-guided mechanism, breaking the symmetry of the collision avoidance potential field.

8. The unmanned surface vessel path tracking and obstacle avoidance method based on reinforcement learning-guided MPC according to claim 1, characterized in that, The MPC optimization control described in step (5) is as follows: taking the final reference state trajectory obtained in step (4) as the tracking target, and combining the ocean current interference estimated in step (1), constructing an MPC optimization problem that includes dynamic constraints and collision avoidance rules potential field, and solving it to obtain the optimal longitudinal thrust and steering torque command of the unmanned surface vessel. Wherein, the cost function of the MPC optimization problem Includes a weighted sum of four terms: In the formula, To track costs, the predicted state is used to minimize the difference between the predicted state and the final reference state generated in step (4). Deviation between; To control input costs, penalties are imposed on the rate of change of thrust and rudder angle to prevent high-frequency oscillations in the actuators; This is a static safety cost, used to maintain a safe distance from obstacles; The cost is the regular potential field cost, which is constructed based on the relative azimuth angle of the unmanned surface vessel relative to dynamic obstacles. A high penalty is imposed when the relative azimuth angle is in the danger sector defined by COLREGs.

9. The unmanned surface vessel path tracking and obstacle avoidance method based on reinforcement learning-guided MPC according to claim 1, characterized in that, The closed-loop execution in step (6) specifically involves sending control commands to the actuator to drive the unmanned surface vessel to move, and then returning to step (1) to perform the next cycle.

10. An unmanned surface vessel control system for implementing the method according to any one of claims 1 to 9, characterized in that, include: (1) Sensing module: including lidar, GPS positioning unit and inertial measurement unit, used to collect environmental and motion data; (2) Computation module: includes three parallel processing units, which are used to run the EnKF state observer, PPO deep neural network inference and MPC nonlinear optimization solver respectively; (3) Storage module: Stores pre-trained PPO network weight parameters and global nautical chart data; (4) Low-level execution module: Receives the thrust and torque commands output by the calculation module and converts them into the rotational speed commands of the left and right thrusters through the thrust allocation matrix.

Images