A method, system, computer equipment and medium for automatic berthing control of ships

Abstract

This invention provides a method, system, computer equipment, and medium for automatic ship berthing control, belonging to the field of intelligent ship control. The method includes acquiring the ship's pose state, berth target state, and distance-from-shore environmental state; extracting state features from the ship's pose state, berth target state, and distance-from-shore environmental state and generating action probability distributions; taking the mean of the action probability distributions as the nominal control action and constructing a nominal control command through inverse normalization; determining safety constraints based on the nominal control command, the maneuvering parameters of the ship to be berthed, and a risk quantification value; when the nominal control command is unsafe, revising the nominal control command to obtain a quadratic programming-based action control command. The risk quantification value is determined based on the real-time state of the ship to be berthed and the geometric information of the dock. This method significantly improves the robustness of berthing control under shore-based interference and overcomes the problem of low versatility of existing technologies in different berthing scenarios.

Description

Technical Field

[0001] This invention belongs to the field of intelligent ship control, specifically relating to a method, system, computer equipment, and medium for automatic ship berthing control. Background Technology

[0002] With the continued growth of global port throughput, the role of automated ship berthing technology in intelligent shipping systems is becoming increasingly prominent. According to the European Maritime Safety Authority's (EMSA) "Annual Overview of Maritime Accidents and Incidents 2024" report (2014–2023), approximately 51.8% of maritime accidents occurred in nearshore areas, with the berthing phase being the most frequent. The primary cause of these accidents is human error, often leading to collisions between the vessel and pier structures, other vessels, or the vessel itself, resulting in significant economic losses and casualties. This underscores the challenge and necessity of achieving high-precision, safe berthing control in complex nearshore environments.

[0003] To address the aforementioned issues, scholars both domestically and internationally have conducted extensive research on automatic berthing control algorithms, gradually forming three mainstream technical approaches: The first is the traditional step-by-step control method, which typically divides the berthing task into two parts: trajectory planning at the policy layer and trajectory tracking at the control layer. Berthing paths are generated through sampling, optimization, or heuristic methods, and attitude control is based on virtual targets or path tracking algorithms. However, this method decouples trajectory planning from the underlying control, making it difficult to balance system robustness and real-time performance. The second approach involves constructing an end-to-end neural network controller, learning directly from historical data to map control commands from the state space. However, it heavily relies on high-quality data, has limited generalization capabilities, and struggles to guarantee policy stability under complex or unseen operating conditions. The third approach is Deep Reinforcement Learning (DRL), which iteratively optimizes policies through interaction with the environment, possessing both intelligence and adaptability. However, this method also relies on small-sample supervised learning and still has insufficient ability to cope with complex shore-based disturbances.

[0004] In summary, the three existing mainstream technical approaches all suffer from insufficient adaptability in scenarios involving real shore-based disturbances and ship dynamics constraints, and the existing berthing control methods have low versatility in heterogeneous waters. Summary of the Invention

[0005] To address the shortcomings of existing automatic berthing control methods in adapting to shore-based interference scenarios and ship dynamics constraints, and their low versatility in heterogeneous waters, this invention provides a ship berthing control method, system, computer equipment, and medium.

[0006] To achieve the above objectives, the present invention provides the following technical solution:

[0007] An automatic berthing control system for ships includes:

[0008] Acquire the vessel's position and attitude status, berth target status, and distance from shore environment status;

[0009] Based on the ship's pose state, real-time motion features of the ship are extracted; based on the berth target state, target relationship features are extracted; and based on the shore-based environmental state, shore-based constraint features are extracted. Based on the real-time motion features of the ship, target relationship features, and shore-based constraint features, a motion probability distribution is generated. The mean of the motion probability distribution is taken as the nominal control action, and the nominal control command is constructed by inverse normalization.

[0010] Safety constraints are determined based on nominal control commands, the maneuver parameters of the vessel to be moored, and risk quantification values. When the nominal control commands are unsafe, they are modified to obtain the action control commands after secondary planning. The risk quantification values ​​are determined based on the real-time status of the vessel to be moored and the geometric information of the wharf shoreline.

[0011] The present invention also provides an automatic berthing control system for ships, comprising:

[0012] The vessel information acquisition module is used to acquire the vessel's position and attitude status, berth target status, and distance from shore environment status of the vessel to be moored.

[0013] The control command generation module is used to extract real-time motion features of the ship based on its pose state, target relationship features based on the berth target state, and shore-based constraint features based on the distance from the shore environment. It generates a motion probability distribution based on the real-time motion features of the ship, target relationship features, and shore-based constraint features. The mean of the motion probability distribution is taken as the nominal control action, and the nominal control command is constructed by inverse normalization.

[0014] The control command adjustment module is used to determine safety constraints based on the nominal control command, the maneuver parameters of the vessel to be moored, and the risk quantification value. When the nominal control command is unsafe, the nominal control command is modified to obtain the action control command after secondary planning. The risk quantification value is determined based on the real-time status of the vessel to be moored and the geometric information of the wharf shoreline.

[0015] The present invention also provides a computer device, including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement any of the steps in an automatic berthing control method for ships.

[0016] The present invention also provides a computer-readable storage medium storing a computer program that, when loaded by a processor, can execute any step of an automatic berthing control method for a ship.

[0017] The automatic berthing control method, system, computer equipment, and medium provided by this invention have the following beneficial effects:

[0018] The automatic berthing control method provided by this invention can acquire the real-time pose state of the vessel to be berthed, the target berth state, and the state of the environment far from the shore. It can then extract state features to generate a probability distribution of actions, construct nominal control commands, and combine risk quantification values ​​to determine safety constraints. When the nominal control commands are unsafe, they can be modified according to ship dynamics to generate action control commands after quadratic programming. This significantly improves the adaptability to real shore-based disturbances and the robustness of ship dynamics constraints, overcoming the problem of low versatility of existing technologies in complex berthing scenarios. It ensures that the method can stably and reliably achieve automatic berthing control under various dock environments and dynamic changing conditions. Attached Figure Description

[0019] To more clearly illustrate the embodiments and design schemes of the present invention, the accompanying drawings required for this embodiment will be briefly described below. The drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 This is a schematic diagram of an automatic berthing control method for ships provided in an embodiment of the present invention;

[0021] Figure 2 This is a diagram of the PPO-CBF algorithm architecture provided in an embodiment of the present invention;

[0022] Figure 3 This is an example of the current Actor network and target Actor network structure provided in an embodiment of the present invention;

[0023] Figure 4 This is an example of a Critic network structure provided in an embodiment of the present invention;

[0024] Figure 5 This is an example of a three-degree-of-freedom ship motion coordinate system provided in an embodiment of the present invention;

[0025] Figure 6 This is a comparison diagram of motion trajectories under simple interference provided in an embodiment of the present invention;

[0026] Figure 7 This is a comparison diagram of motion trajectories under strong interference provided in an embodiment of the present invention;

[0027] Figure 8 This is a comparison diagram of motion trajectories under severe interference provided in an embodiment of the present invention;

[0028] Figure 9 A comparison chart of the number of collision rounds and success rounds in the first 1000 rounds of training under tiered difficulty provided in this embodiment of the invention;

[0029] Figure 10 This is a trajectory diagram of a vehicle moving in the same direction in the Osaka waters, provided as an embodiment of the present invention.

[0030] Figure 11 This is a trajectory diagram of non-unidirectional motion in the Osaka waters provided in an embodiment of the present invention. Detailed Implementation

[0031] To enable those skilled in the art to better understand and implement the technical solutions of the present invention, the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments. The following embodiments are only used to more clearly illustrate the technical solutions of the present invention and should not be construed as limiting the scope of protection of the present invention.

[0032] Given the widespread application of Control Barrier Functions (CBFs) in safety-critical systems such as robots, vehicles, and ships in recent years, based on set invariance theory, CBFs impose constraints on the system state during policy execution by constructing continuously differentiable safety functions, thereby achieving real-time avoidance of collisions or boundary violations. This provides a theoretical basis for integrating dynamic safety constraints into deep reinforcement learning. This invention utilizes an end-to-end berthing control framework that integrates the PPO algorithm and the CBF safety constraint mechanism. The PPO serves as the policy core, providing adaptive control capabilities to adapt to complex berthing scenarios; while the CBF implements constraints during the decision-making process, dynamically suppressing dangerous actions and ensuring the safety of the entire berthing process, thus achieving automatic berthing control for ships. This invention provides an automatic berthing control method for ships. On the one hand, it proposes an end-to-end berthing control method that integrates ship dynamics models and deep reinforcement learning, directly learning the underlying control strategy from the state space. This method combines strategy exploration capabilities with fine-grained control performance, making it suitable for automatic berthing tasks in complex shore-based interference environments. On the other hand, it introduces a CBF mechanism to impose dynamic safety constraints on the strategy output, improving space utilization efficiency during berthing and safety during the training phase. Furthermore, this invention constructs a reward function that fits the real berthing scenario, integrating factors such as shore-based constraints and heading changes to effectively enhance the generalization and stability of the strategy.

[0033] Example 1

[0034] This invention provides a method for automatic berthing control of ships, specifically as follows: Figure 1 As shown, it includes the following steps:

[0035] Step 11: Obtain the vessel's position and attitude status, berth target status, and distance from shore environment status.

[0036] Based on the above embodiments, the present invention provides a method for controlling automatic berthing of ships, wherein step 11 includes:

[0037] Step 111: Obtain the geometric information of the wharf shoreline and the real-time status of the vessels to be moored. Construct the direction vector and relative position vector of the shoreline segment based on the geometric information of the wharf shoreline and the real-time status of the vessels to be moored. Calculate the projection scale coefficient based on the direction vector and relative position vector of the shoreline segment and truncate it to obtain the nearest point of the vessels to be moored on the shoreline segment. Calculate the Euclidean distance based on the nearest point. Determine the closest distance between the vessel and the shoreline boundary based on the minimum value among multiple Euclidean distances. Determine the heading angle and the normal angle between the nearest shoreline boundary based on the closest distance between the vessel and the shoreline boundary.

[0038] Specifically, in the deep reinforcement learning control framework, the construction of the state space has a decisive impact on the effectiveness and convergence of the agent's policy learning. In order to accurately reflect the motion state, target guidance and environmental constraints involved in the ship berthing process, the automatic ship berthing control method provided in this invention divides the state space into three information subsets, namely the ship pose state, the berth target state and the distance from shore environment state, thereby forming a high-dimensional perception space that integrates autonomous navigation and risk perception capabilities, as shown in formula (1):

[0039] (1)

[0040] in, For higher-dimensional perception space, The ship's position and attitude. The target status of the berth. This refers to the environmental conditions offshore. Specifically, the ship's position and attitude. The information used to characterize the current dynamics and attitude of a ship is shown in formula (2):

[0041] (2)

[0042] in, Including the position of the ship's center of mass in the global coordinate system Bow angle and longitudinal velocity in the ship's coordinate system lateral velocity and bow angular velocity In addition, to reflect the trend of course smoothness caused by long-term heading adjustments, the cumulative heading change was introduced. This records the total change in the ship's bow angle during the current round. These variables comprehensively characterize the ship's attitude evolution and motion trends during autonomous berthing, providing fundamental dynamic input support for strategy optimization. The berth target state... The formula used to describe the relative position and attitude deviation between the ship and the pre-set berth is shown in formula (3):

[0043] (3)

[0044] in, Includes berth location coordinates The heading angle of the target berth Current distance of the vessel from the target berth The difference between the ship's current heading angle and the heading angle of the target berth Specifically, the current distance of the vessel from the target berth. The difference between the ship's current heading angle and the target berth heading angle can be calculated using the Euclidean distance shown in formula (4). It can be determined using the formula shown in formula (5):

[0045] (4)

[0046] (5)

[0047] The ability to guide an intelligent agent to gradually achieve both spatial position and attitude alignment is a crucial basis for deep reinforcement learning strategies to achieve berth convergence. The offshore environmental state... This is used to provide geometric perception capabilities of the shore-based structures in the berthing area. In actual berth scenarios, shore-based structures are typically arranged in a polygonal form close to the berth. Ships need to have high-precision boundary distance judgment capabilities to achieve safe berthing close to the shore without collision. In this embodiment, the closest distance between the ship and the shore-based boundary is used. The angle between the heading angle and the normal of the nearest shoreline boundary The offshore environmental state is described as shown in formula (6):

[0048] (6)

[0049] Specifically, this embodiment employs a calculation method based on Euclidean nearest projection to identify the nearest shoreline segment corresponding to the current ship's center of mass, and further obtains its orientation angle, thereby accurately extracting the relative geometric information between the ship's current position and the shoreline boundary. The current position of the ship's center of mass is shown in formula (7):

[0050] (7)

[0051] The shoreline consists of a series of ordered vertices. Composition, pairing adjacent points Defined as a line segment along the shoreline. For each line segment, construct its direction vector. With relative position vector As shown in formulas (8) and (9) respectively:

[0052] (8)

[0053] (9)

[0054] Next, the projection scale factor is calculated as shown in formula (10):

[0055] (10)

[0056] And by truncating the projection point to fall inside the line segment as shown in formula (11), the nearest point of the ship on the line segment is obtained, and the Euclidean distance from the nearest point to the ship is calculated as shown in formulas (12) and (13):

[0057] (11)

[0058] (12)

[0059] (13)

[0060] Finally, iterate through all shoreline segments and select the corresponding smallest one. The line segment is taken as the nearest shoreline to the ship's current position. The angle between the bow angle and the normal to the nearest shoreline boundary is... The calculation is shown in formula (14):

[0061] (14)

[0062] offshore environment status By fully considering the finite length constraint of the shoreline segment, the method avoids the errors and instabilities that may occur in the traditional point-to-line distance model when approaching shoreline corners or endpoints. In irregular dock geometry environments, this method has higher robustness and computational accuracy, and can be widely applied to obstacle detection, attitude assessment, and directional control index extraction in berthing tasks. In addition to the state space, this embodiment also provides an example of constructing an action space. Since the action space A determines the output structure of the policy network and is the key bridge from policy learning to actual control, this embodiment designs the action space as a two-dimensional continuous control vector as shown in formula (15):

[0063] (15)

[0064] in, Main thruster speed, This refers to the rudder angle control command, which directly corresponds to the ship's actual control input interface. This design possesses good physical feasibility, directly driving pre-built ship dynamics, and also improves the action space exploration efficiency of reinforcement learning strategies.

[0065] Step 12: Extract real-time motion features of the ship based on its pose state, extract target relationship features based on the berth target state, and extract shore-based constraint features based on the distance from the shore environment state; generate action probability distributions based on the real-time motion features of the ship, target relationship features, and shore-based constraint features; take the mean of the action probability distributions as the nominal control actions, and construct nominal control commands by inverse normalization.

[0066] Step 13: Determine safety constraints based on nominal control commands, the maneuver parameters of the vessel to be moored, and risk quantification values. When the nominal control commands are unsafe, modify the nominal control commands to obtain the action control commands after secondary planning. The risk quantification values ​​are determined based on the real-time status of the vessel to be moored and the geometric information of the dock shoreline.

[0067] Based on the above implementation methods, this embodiment can use the PPO-CBF model to process the ship's pose state, berth target state, and distance-from-shore environmental state to obtain the motion control command after quadratic planning. The PPO-CBF model includes a PPO algorithm architecture and a CBF safety filter. The PPO algorithm architecture includes a feature extraction layer, a motion generation layer, and an output layer connected in sequence. The CBF safety filter includes a safety decision layer and a QP optimizer connected in sequence. This invention achieves automatic ship berthing control through a PPO-CBF automatic berthing control algorithm that integrates policy gradient reinforcement learning and a control barrier function. The algorithm architecture is as follows: Figure 2 As shown, based on the ability of Proximal Policy Optimization (PPO) to learn policies in a continuous action space, efficient and robust berthing motion planning is achieved. Simultaneously, a Collision-Constraint Function (CBF) is introduced as a safety constraint module to construct a safe feasible region based on the policy output, and to filter and correct potentially high-risk actions in real time, thereby enhancing collision safety capabilities. In summary, this algorithm significantly improves the safety and practicality of the control strategy in shore-based interference environments while maintaining motion planning performance, providing theoretical and algorithmic support for practical deployment.

[0068] Specifically, to achieve end-to-end berthing control without relying on traditional path planning and trajectory tracker-based hierarchical control, this embodiment utilizes the Markov Decision Process (MDP) framework in deep reinforcement learning to construct a deep policy network that can directly learn low-level control commands from environmental state perception. Since the essence of end-to-end control policy is to construct a direct mapping from state perception to action control, this embodiment formalizes the ship berthing process as a five-tuple MDP. ,in, This is the state space, representing the ship's current perception of its environment. The action space represents the control variables for the policy output. The environmental state transition function is determined based on the ship motion model. This is the reward function, used to measure the progress of the current control action in the berthing task. This serves as a discount factor, used to balance immediate rewards and long-term benefits. During modeling, the agent uses a policy function to transition from the state space S to the action space A. Current state Mapped to the corresponding control action It interacts with the environment to receive new states. With instant rewards This forms a closed-loop control process, and the policy network parameters are then processed using the PPO algorithm. Continuous optimization is performed to maximize the expected long-term return. This includes the policy network parameters. The calculation is shown in formula (16):

[0069] (16)

[0070] The strategy learning process is a typical end-to-end control paradigm, which can achieve direct mapping from raw state inputs to physical control commands without relying solely on environmental interaction or requiring external expert demonstrations or path samples.

[0071] Based on the above embodiments, the CBF safety filter also includes a safety function calculator. In this invention, step 13 of an automatic ship berthing control method includes:

[0072] Step 131: Obtain the real-time status of the vessel to be moored and the geometric information of the dock foundation; calculate the risk quantification value by using the safety function calculator to obtain the real-time status of the vessel to be moored and the geometric information of the dock foundation; construct the feasible region based on the nominal control command and the dynamic model of the vessel to be moored; search for candidate solutions that satisfy the safety constraints in the feasible region using the QP optimizer; adjust the nominal control command based on the candidate solutions to obtain the action control command after quadratic programming.

[0073] like Figure 2 As shown, the PPO-CBF model provided by this invention introduces a discrete-time control barrier function (CBF) framework to adjust the control commands output by the deep reinforcement learning agent online to avoid entering dangerous areas. In this embodiment, the CBF constraint is embedded into the automatic berthing control system, and the optimal control action is projected by constructing a safety filter based on quadratic programming (QP). Specifically, the current ship state is shown in formula (17):

[0074] (17)

[0075] Where x and y are the ship's position coordinates, Let n be the heading angle and n be the heading angle. These represent the main propeller speed and rudder angle, respectively. A safety function (Barrier Function) is defined based on the relative distance between the hull and the dock, as shown in formula (18):

[0076] (18)

[0077] in, This is the shortest distance between the ship's current position and the shore base, after deducting the ship's hull radius. The minimum safe distance is set by the user. The sign of the safety function value is used to determine whether the current state is in a safe region. To ensure the positive invariance of the system state within the safe set, the CBF must satisfy the condition shown in formula (19):

[0078] (19)

[0079] Furthermore, in order to estimate the Lie derivative term in formula (19) L f h(s t ) and L g h(s t ) This can be approximated using a one-step finite difference method. First, the standard action output by the deep reinforcement learning agent is... Mapped to physical action increments and And combine the ship's maneuver mathematical model to predict the state at the next moment. Thus, we obtain the approximate relationships shown in formulas (20) and (21):

[0080] (20)

[0081] (twenty one)

[0082] in, and These represent the system states after applying small positive and negative perturbations to the current control input, respectively, and are used to estimate the sensitivity of the control input to the safety function. When the system state enters a potentially hazardous region, the CBF safety filter will be activated, and the safety function will begin to be continuously calculated. It also assesses the feasibility of the current policy control input and triggers a control correction mechanism if necessary. If the nominal control input... If the safety constraints cannot be met, the QP optimizer is entered, and the minimum deviation safety control input is obtained by solving the constrained quadratic programming problem as shown in formula (22):

[0083] (twenty two)

[0084] in, This represents the nominal control input generated by a deep reinforcement learning policy and then physically mapped. Positive coefficients in the K-like function are used to adjust the strictness of constraints, ensuring that safety constraints are sufficiently conservative. The optimization problem is solved efficiently using the OSQP solver; if the problem has no feasible solution in a certain state, the system will prioritize executing the original action to avoid control interruption. To achieve seamless integration with deep reinforcement learning control systems, the generated physical control actions will be converted back to the standard action space through inverse mapping. Ensure that the format of the action signals remains consistent with the original agent. The proposed CBF safety filter can provide an explicit safety guarantee mechanism without interfering with policy training, making it particularly suitable for automated berthing tasks with complex berth structures and severe collision consequences. Compared with traditional obstacle expansion methods, its formal constraint structure and continuously adjustable safety boundary construction enable ships to have stronger space utilization and boundary safety during shore-hulled berthing.

[0085] Based on the above embodiments, in the automatic berthing control method for ships provided by the present invention, the PPO-CBF model can be trained through the following steps before step 12:

[0086] Step 14: Input the training state information into the untrained PPO-CBF model to obtain training instructions and simulate ship berthing, collecting current environmental feedback; calculate the berthing success constraint reward based on the environmental feedback to obtain the berthing success reward value; calculate the distance reward and heading reward based on the environmental feedback to obtain the process optimization reward value; calculate the collision risk penalty based on the environmental feedback to obtain the safety risk penalty value; calculate the total reward corresponding to the training instructions based on the berthing success reward value, process optimization reward value, and safety risk penalty value, and adjust the PPO-CBF model based on the total reward.

[0087] Specifically, considering that the design of the reward function in deep reinforcement learning algorithms directly affects the policy convergence direction and agent behavior patterns, especially in high-risk berthing tasks where both target guidance and safety inhibition must be considered, this embodiment also constructs a hierarchical reward function structure activated during model training. This structure consists of a berthing success constraint, a procedural incentive, and a collision risk penalty, enabling the policy to optimize berthing efficiency while possessing good safety tolerance. The reward function structure includes a hard constraint reward term defining success or failure, namely the berthing success constraint, used to provide explicit feedback on whether the berthing task was successfully completed. For example, when the ship meets the berthing accuracy constraint, for instance… and Positive rewards are given when the ship is too close to the shore boundary, for example, less than 0.5m, it is considered a collision and a strong penalty is given, as shown in formula (23):

[0088] (twenty three)

[0089] The reward function structure also includes process-based basic reward items, such as distance rewards and heading rewards, to guide the ship to continuously optimize its approach to the target throughout the berthing process. Specifically, the process-based basic reward items include a combined penalty for distance, heading deviation, and cumulative heading adjustment, as shown in formula (24):

[0090] (twenty four)

[0091] in, and The standardized values ​​represent the distance difference and heading angle difference between the vessel and the target berth, where the distance and heading angle rewards are always negative. The larger the distance and heading angle deviations, the smaller the reward value; conversely, the smaller the deviations or the closer they are to zero, the larger the reward value. To achieve high heading smoothness and reduce operational risk during berthing, this embodiment introduces a heading integral term; however, the larger the cumulative heading change during berthing, the smaller the reward value. Furthermore, the reward function structure also includes a collision risk penalty function constructed based on geometric and dynamic indicators. This enhances a ship's ability to avoid near-shore collision risks. Specifically, the collision risk penalty function... Considering the bow angle between the ship and the nearest shoreline boundary Distance from the center point to the shore and velocity inertia index These three influencing factors are shown in formula (25):

[0092] (25)

[0093] in, This is used to describe whether the bow of a ship is pointing directly at the nearest shoreline boundary. The risk of collision is greatest when the relative angle between the ship and the shore is 90 degrees. This reflects how close a ship is to the shore; the smaller the closest distance to the shore, the greater the risk of collision. The inertia of the ship at its current speed is measured, where u is the longitudinal velocity and r is the bow angular velocity. Through this collision risk penalty function mechanism, the agent can effectively and autonomously adjust its speed during the overall berthing process, thereby controlling the ship's inertia under different levels of risk. Furthermore, it geometrically decomposes the collision risk of the shore-based polygon into angular risk, distance risk, and inertial risk, no longer confined to a specific shore-based polygon, thus exhibiting a certain degree of environmental adaptability and control robustness. In summary, the reward function structure in the automatic berthing control method provided by this invention is shown in formula (26):

[0094] (26)

[0095] By organically integrating task completion criteria, process behavior optimization, and environmental interaction feedback through a hierarchical structure design, the strategy can be effectively guided to generate berthing behaviors that are both task-oriented and environmentally adaptable.

[0096] Furthermore, in the automatic berthing control method for ships provided by this invention, the training of the PPO-CBF model further includes the following steps:

[0097] Step 15: Extract state features from the training state information and calculate the state value estimate and actual reward; calculate the advantage value based on the state value estimate and actual reward, and construct the action optimization direction signal; update the gradient of the PPO-CBF model based on the action optimization direction signal to obtain the PPO-CBF model.

[0098] Specifically, such as Figure 2 As shown, the DRL network architecture corresponding to the PPO algorithm in the PPO-CBF model includes three neural network architecture layers: the current Actor network, the target Actor neural network, and the Critic neural network. The input layer and output layer of the neural network correspond to the state space and action space mentioned in the above implementation method, respectively.

[0099] The structures of the current Actor network and the target Actor network are as follows: Figure 3 As shown, the input data shape in the Actor network is 11*1. The specific input content is the motion status information of the agent. First, it is processed by 64 neurons in Dense Layer 1 and then by the ReLU function. After that, it is processed by 32 neurons in Dense Layer 2 and then by the ReLU function. Finally, it is processed by 10 neurons in Dense Layer 3 and then by the Tanh function. The final output data shape is 2*1, and it outputs parameters such as the ship's rotational speed (RPM) and steering angle (RA).

[0100] Critic network structure as follows Figure 4 As shown, the input data shape in the Critic network is 11*1, and the specific input content is the agent's motion status information. First, it is processed by 64 neurons in Dense Layer 1 and then by the ReLU function. Next, it is processed by 32 neurons in Dense Layer 2 and then by the ReLU function. Finally, it is processed by 10 neurons in Dense Layer 3 and then by the Tanh function. The final output data shape is 1*1, outputting the value of status. During model training, the Critic network estimates the state value and calculates the advantage value from the training data, thereby achieving gradient updates for the PPO-CBF model.

[0101] Example 2

[0102] Building upon Example 1, this invention further provides a specific example of a ship berthing simulation experiment and verifies the effectiveness of the PPO-CBF model through an ablation experiment. In this example, considering the real-world port terminal environment and conventional ship configuration, an automatic berthing control scenario with shore-based interference is constructed, achieving automatic berthing control independent of empirical pre-aiming points. Furthermore, to balance the economy and environmental friendliness of the automatic berthing system, path length minimization and path smoothness optimization are used as auxiliary performance indicators. Specifically, considering the most representative ship type characteristics in the current shipping market, this example selects a single-propeller, single-rudder conventional ship model as the research object, which conforms to the power configuration characteristics of current mainstream merchant ships and has good engineering deployability. Due to its underactuated characteristics, this type of ship has limited control freedom, and path adjustment and attitude control during berthing are significantly challenging, thus placing higher demands on the intelligence and robustness of the control strategy. The simulation is conducted based on the model ship "KVLCC2_l7". Table 1 shows the main hull numbers of this ship. By referencing previous publicly available research, this example defines the acceptable error for successful berthing as... , .

[0103] Table 1. Main hull system numbers of the model ship "KVLCC2_l7"

[0104]

[0105] Subsequently, this embodiment also provides a method for modeling ship maneuverability, thereby facilitating the use of the ship motion model as the physical driver of an intelligent agent. The next state value of the agent is calculated from the previous state value and the selected values ​​of the previous action space through the ship numerical model. Specifically, considering that the ship's center of gravity is below the overall ship structure, the horizontal motion of the hull parallel to the water surface is mainly considered during actual berthing. In this embodiment, the pitch motion of the hull, the vertical motion perpendicular to the water surface, and the rolling motion centered on the ship's axis are ignored, and a model is established as follows: Figure 5 The diagram shows a three-degree-of-freedom ship motion coordinate system. Two coordinate systems are used to describe each step of the ship's motion: a fixed spatial coordinate system and a fixed spatial coordinate system. and ship fixed coordinate system The heading angle ψ of a ship is defined as... The angle with x. The velocity components of the ship along x and y are represented by u and y, respectively. The actual speed of the ship is represented as shown in formula (27), and the yaw angle is... The definition is shown in formula (28):

[0106] (27)

[0107] (28)

[0108] Yaw rate r is the percentage of the ship's yaw rate around the yaw point. The angular velocity of the shaft rotation. Based on the establishment of the ship's three degrees of freedom, this embodiment represents the ship's kinematic model as shown in formula (29), and the ship's dynamic model as shown in formula (30):

[0109] (29)

[0110] (30)

[0111] Among them, m, , These represent the mass of the ship, and the additional mass of the ship along the x-axis and y-axis due to inertia, respectively. The longitudinal coordinate of the ship's center of gravity. and These represent the ship's moment of inertia at its center of gravity and the additional moment of inertia caused by the ship's inertia, respectively. X, Y, These represent the longitudinal force, lateral force, and bow moment acting on the hull, respectively. Considering that the actual environment for automatic berthing control applications is usually in inland waterways or near-shore areas protected by breakwaters, waves and surges have a relatively small impact on ships, while wind disturbance is the main factor affecting ship safety in automatic berthing control. Therefore, in this embodiment, the forces and moments acting on the hull in formula (30) are further decomposed based on the motion characteristics of a single-propeller, single-rudder ship, resulting in formula (31):

[0112] (31)

[0113] In this model, the subscripts H, P, R, and W represent the hull, propeller, rudder, and wind disturbance, respectively. The specific calculation methods are existing publicly available technologies and will not be elaborated upon in this invention. After the ship's maneuverability modeling is completed, this embodiment uses the PyTorch framework and Python in the PyCharm environment to build a simulation platform and verify the algorithm. The Pygame module is used to visualize the training process. The visualization interface mainly consists of a 1000*1000 rectangular window, the ship's initial pose, the ship's real-time pose, and the target berth. The actual shape of the constructed shore base is mainly based on the shapes of two shore bases: the Xiamen waters at latitude and longitude 24°35'50.5"N 118°04'42.8"E and the Osaka waters at latitude and longitude 34°49'22.0"N 135°31'28.8"E.

[0114] The hyperparameter settings for algorithm training are shown in Table 2:

[0115] Table 2 Hyperparameters for Algorithm Training

[0116]

[0117] To ensure the rationality of the network output, the tanh activation function was selected, with an output range of [-1, 1], and linearly transformed to the range of normal rotational speed and rudder angle. The Actor network update frequency was set to perform policy optimization only after every K updates, and the PPO cutoff coefficient was set accordingly. The value was set to 0.2. Furthermore, to improve model stability, the algorithm employs various optimization techniques such as advantage function normalization, state normalization, reward scaling, and gradient pruning. The Adam optimizer is used for model optimization, and orthogonal initialization is used for weight initialization. After the simulation environment is set up, ablation experiments of the PPO-CBF algorithm with varying difficulties are conducted to comprehensively evaluate the performance gain of the control barrier function mechanism in automatic ship berthing tasks. To ensure the scientific rigor and quantifiability of the experimental difficulty classification, this embodiment proposes a spatial constraint assessment method based on the degree of lateral interference in the waterway. Specifically, it is assumed that the ship can navigate along the line connecting the starting point and the target berth point under no shore-based constraints, and an ideal passageway is constructed accordingly. This path is defined as a rectangular channel with the starting and ending points as the midpoints of the two short sides, and its width is equal to the actual width of the ship. Subsequently, this ideal path is spatially superimposed on the actual shore-based geometry, and based on the intersection area, the projection width of the overlapping area in the direction perpendicular to the course, i.e., the lateral direction of the path, is calculated, thus obtaining the path lateral overlap ratio index. This indicator is defined as the ratio of the maximum lateral obstruction width of the shore base on the path to the ship's beam, reflecting the degree of pressure exerted by the shore base on the ship's passageway. The larger this overlap width ratio, the more restricted the lateral space of the path, the more difficult the passage to the berth, and the higher the difficulty of berthing.

[0118] Based on this metric, this embodiment divides the experimental scenario into three typical difficulty levels: simple interference with 0-20% overlap, strong interference with 20-60% overlap, and severe interference with 60-100% overlap. Examples include an easy path with an overlap of 12.7%, a medium path with an overlap of 57.1%, and a hard path with an overlap of 100.0%. These three typical difficulty levels correspond to three representative starting point settings, as shown in Table 3. The same target berth is used, with coordinates (195, 160) and a heading angle of 0°.

[0119] Table 3. Ship Starting Position under Different Difficulty Levels

[0120]

[0121] The comparison images of PPO-CBF algorithm ablation experiments under simple interference, moderate interference, and severe interference scenarios are shown below. Figure 6 , Figure 7and Figure 8 As shown in Table 4, the experimental comparison results of the ablation algorithms under the three interference scenarios are as follows:

[0122] Table 4 Comparison of Algorithm Ablation Experiment Results in Three Scenarios

[0123]

[0124] As shown in the table above, comparing the original PPO and the PPO-CBF algorithm with CBF in the three interference scenarios, the results show that under simple and slight interference, PPO-CBF reduces the final yaw by 34.47% and 39.43% respectively compared to the original PPO, significantly reducing heading oscillations, resulting in a smoother path and higher space utilization. Simultaneously, the path length is shortened by 4.67% and 11.48% respectively, significantly improving control efficiency. In the severe interference scenario, PPO frequently collides, fails to train, and cannot complete the task, while PPO-CBF exhibits strong adaptability and can complete obstacle avoidance and precise berthing. Training statistics are as follows: Figure 9 As shown, the collision rate of PPO-CBF is reduced by 59.9% compared to PPO, while the success rate is increased to 43.7%. The results verify the control advantages and policy robustness of the CBF mechanism in complex berthing. In summary, the PPO-CBF algorithm not only performs superiorly in berthing path quality and trajectory stability, but also exhibits higher robustness and deployability in the training and task execution processes under complex spatial constraints, demonstrating good potential for engineering applications. Finally, a generalization experiment was conducted in heterogeneous waters. In this embodiment, the Osaka waterway terminal, based on existing research by Shoma Shimizu, was used for generalization verification. Compared to the shore-based environment of the Xiamen waterway terminal, the space of the Osaka waterway terminal is smaller, but its shore-based geometric constraints have less interference with the overall motion control process of berthing, and the situation of complete interference in the berthing process in the Xiamen waterway does not occur. This embodiment refers to the relative situation design in Shoma Shimizu's experiment, and adds berthing simulation under the condition of large difference in heading angle. In this embodiment, the experimental simulation environment of Shoma Shimizu is referenced to model the shore-based physical environment, and a similar berthing situation with no need for large adjustments to the heading angle is set. The berthing start pose and berth pose in the berthing scenario are shown in Table 5, and the key data extraction for berthing situation analysis is shown in Table 6.

[0125] Table 5 Simulated Co-directional Trend in Osaka Waters

[0126]

[0127] Table 6. Key Data Extraction Table for Analysis of Same-Direction Berthing Patterns in Osaka Waters

[0128]

[0129] The same direction berthing trajectory in Osaka waters is as follows Figure 10 As shown, compared to the Xiamen waters, the shore-based interference in the Osaka waters is weaker. The agent navigates along a clear channel, requiring only slight avoidance at the end of the berthing phase. Although the overall interference is small, the concave shore-based structure of Osaka still limits early policy learning, making the agent prone to limitations or stagnation during initial exploration. At this point, the inertial penalty function based on the relative heading angle effectively guides the agent to avoid risks and optimize its path. According to Table 6, during berthing in the same direction, within 0-43 seconds... from Rise to Achieving gradual growth, corresponding to shoreline 2, and maintaining stable control; while during the 44-54s period, shoreline 3 was switched to shoreline 3. Jump to The minimum distance to the shore decreases, increasing the risk of collision, and the intelligent agent initiates avoidance maneuvers; when it approaches the berth after 54 seconds... Down to The control strategy shifts to precise alignment to achieve smooth berthing. Table 7 shows the initial berthing pose and berth pose in the non-co-directional berthing scenario, and Table 8 shows the key data extraction for non-co-directional berthing situation analysis.

[0130] Table 7 Simulated Non-Co-directional Situations in Osaka Waters

[0131]

[0132] Table 8. Key Data Extraction Table for Non-Co-directional Berthing Situation Analysis in Osaka Waters

[0133]

[0134] Non-co-directional berthing tracks in Osaka waters, such as Figure 11 As shown, compared to homing missions, non-homing berthing paths are longer and require greater course adjustments, significantly increasing control difficulty. The inertial penalty function exhibits four distinct jumps, allowing the overall process to be divided into five stages for analysis: In the initial stage (0-52s), the ship moves away from shoreline 1. Stable at The status is safe; at 53 seconds, switch to shoreline 2. Sudden increase to The first jump occurred; at 59 seconds, it switched to shoreline number 6 on the left. Rise to This triggers a second jump; at 100 seconds, it cuts to shore number 5. Soaring to A third sudden change occurred, but control stabilized; at 125 seconds, it entered the final berth area. sudden drop The single, significant descent abrupt change marks the completion of course alignment. This abrupt change demonstrates the agent's rapid response and adjustment capabilities to changes in shore-based geometry. Overall, the results show that the designed reward mechanism effectively enhances the policy's adaptability and course control accuracy in complex environments, exhibiting good robustness and potential for wider application.

[0135] In summary, the simulation and experimental results of this embodiment demonstrate that the PPO-CBF control framework outperforms the original PPO algorithm in various complex interference scenarios. In the case of slight interference, the heading change and path length are reduced by approximately 39% and 11%, respectively, resulting in a smoother and more efficient berthing process. In the case of fully interfering berths, the original PPO training fails, while the PPO-CBF achieves a success rate of 43.7% and a collision rate reduction of 59.9%, exhibiting significant control robustness and environmental adaptability. Furthermore, in tests involving completely obstructed berths and heterogeneous waterways, the strategy effectively addresses interference from different shore-based structures, validating the practicality and deployability of the method.

[0136] Example 3

[0137] The present invention also provides an automatic berthing control system for ships, comprising:

[0138] The ship information acquisition module is used to acquire the ship's pose state, berth target state, and shore-based environmental state of the vessel to be berthed. The control command generation module is used to extract real-time ship motion features based on the ship's pose state, target relationship features based on the berth target state, and shore-based constraint features based on the shore-based environmental state; it generates a motion probability distribution based on the ship's real-time motion features, target relationship features, and shore-based constraint features; it takes the mean of the motion probability distribution as the nominal control action and inversely normalizes it to construct the nominal control command. The control command adjustment module is used to determine safety constraints based on the nominal control command, the maneuver parameters of the vessel to be berthed, and the risk quantification value. When the nominal control command is unsafe, it corrects the nominal control command to obtain a quadratic-planned motion control command. The risk quantification value is determined based on the real-time state of the vessel to be berthed and the shore-based geometric information of the wharf.

[0139] The present invention also provides a computer device, including a memory, a processor, and a computer program stored in the memory. The processor executes the computer program to implement the steps in an embodiment of an automatic ship berthing control method. Specific implementation methods can be found in the method embodiments, and will not be repeated here.

[0140] Furthermore, the present invention also provides a non-transitory computer-readable storage medium containing instructions, on which a computer program is stored. For example, a memory containing instructions that can be executed by a processor of a computer device to perform the above-described method. For example, the non-transitory computer-readable storage medium may be a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device, etc. When the computer program is executed by the processor, it can implement the steps in an embodiment of an automatic ship berthing control method. Specific implementation methods can be found in the method embodiments, which will not be repeated here.

[0141] It should be noted that the specific embodiments described above enable those skilled in the art to more fully understand the present invention, but do not limit the present invention in any way. Therefore, although the present invention has been described in detail in this specification and embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the present invention; and all technical solutions and improvements that do not depart from the spirit and scope of the present invention are covered within the protection scope of the present invention patent. No reference numerals in the claims should be construed as limiting the scope of the claims. Any simple variations or equivalent substitutions of technical solutions that can be readily obtained by those skilled in the art within the scope of the technology disclosed in the present invention are within the protection scope of the present invention.

Claims

1. A method for controlling automatic berthing of ships, characterized in that, include: Acquire the vessel's position and attitude status, berth target status, and distance from shore environment status; The ship's pose state, berth target state, and shore-distance environment state are processed by the PPO-CBF model to obtain the motion control command after secondary planning. The PPO-CBF model includes a PPO algorithm architecture and a CBF safety filter. The PPO algorithm architecture includes a feature extraction layer, a motion generation layer, and an output layer connected in sequence. The CBF safety filter includes a safety decision layer and a QP optimizer connected in sequence. The feature extraction layer extracts real-time motion features of the ship based on its pose state, target relationship features based on the berth target state, and shore-based constraint features based on the distance from shore environment. The action generation layer generates an action probability distribution based on the real-time motion features of the ship, target relationship features, and shore-based constraint features. The output layer takes the mean of the action probability distribution as the nominal control action and denormalizes it to construct the nominal control command. The safety determination layer determines safety constraints based on the nominal control command, the maneuver parameters of the vessel to be moored, and the risk quantification value. When the nominal control command is unsafe, a feasible region is constructed based on the nominal control command and the dynamic model of the vessel to be moored. The QP optimizer searches for candidate solutions that satisfy the safety constraints in the feasible region. The nominal control command is adjusted based on the candidate solutions to obtain the action control command after quadratic programming. The risk quantification value is determined based on the real-time status of the vessel to be moored and the geometric information of the dock shoreline.

2. The automatic berthing control method for ships according to claim 1, characterized in that, The offshore environmental conditions include the closest distance between the vessel and the shoreline boundary and the angle between the bow angle and the normal of the nearest shoreline boundary. Obtaining the offshore environmental conditions of a vessel to be moored includes: Obtain the geometric information of the wharf shore base and the real-time status of the vessel to be moored, and construct the direction vector and relative position vector of the shore line segment based on the geometric information of the wharf shore base and the real-time status of the vessel to be moored; The projection scaling factor is calculated and truncated based on the direction vector and relative position vector of the shoreline segment to obtain the nearest point of the vessel to be moored on the shoreline segment. The Euclidean distance is calculated based on the nearest point, and the closest distance between the ship and the shore-based boundary is determined based on the minimum value among multiple Euclidean distances. The heading angle and the normal angle between the ship and the shore-based boundary are determined based on the closest distance between the ship and the shore-based boundary.

3. The automatic berthing control method for ships according to claim 1, characterized in that, The CBF safety filter also includes a safety function calculator, which, before determining safety constraints based on the nominal control command, the maneuvering parameters of the vessel to be moored, and the risk quantification value, further includes: Obtain the real-time status of the vessels to be moored and the geometric information of the dock shoreline; The risk quantification value is obtained by calculating the real-time status of the vessel to be moored and the geometric information of the dock shoreline using the safety function calculator.

4. The automatic berthing control method for ships according to claim 1, characterized in that, Before extracting state features from the ship's pose state, berth target state, and offshore environment state, the process also includes: The training state information is input into the untrained PPO-CBF model to obtain training instructions and simulate ship berthing, collecting current environmental feedback. Based on the environmental feedback, the reward for successful berthing is calculated to obtain the successful berthing reward value. Based on the environmental feedback, distance and heading rewards are calculated to obtain the process optimization reward value; Based on the environmental feedback, a collision risk penalty calculation was performed to obtain a safety risk penalty value; The total reward corresponding to the training instruction is calculated based on the berth success reward value, process optimization reward value, and safety risk penalty value, and the PPO-CBF model is adjusted based on the total reward.

5. The automatic berthing control method for ships according to claim 4, characterized in that, After inputting the training state information into the untrained PPO-CBF model, the following is also included: Extract state features from the training state information and calculate state value estimates and actual returns; Based on the state value estimate and actual return, the advantage value is calculated, and an action optimization direction signal is constructed. The PPO-CBF model is updated by gradient based on the action optimization direction signal to obtain the PPO-CBF model.

6. An automatic berthing control system for ships, characterized in that, include: The vessel information acquisition module is used to acquire the vessel's position and attitude status, berth target status, and distance from shore environment status of the vessel to be moored. A control command generation module is used to process the ship's pose state, berth target state, and shore-based environmental state through a PPO-CBF model to obtain motion control commands after quadratic planning. The PPO-CBF model includes a PPO algorithm architecture and a CBF safety filter. The PPO algorithm architecture includes a feature extraction layer, a motion generation layer, and an output layer connected in sequence. The CBF safety filter includes a safety decision layer and a QP optimizer connected in sequence. The feature extraction layer extracts real-time ship motion features based on the ship's pose state, target relationship features based on the berth target state, and shore-based constraint features based on the shore-based environmental state. The motion generation layer generates a motion probability distribution based on the real-time ship motion features, target relationship features, and shore-based constraint features. The output layer takes the mean of the motion probability distribution as the nominal control action and inversely normalizes it to construct the nominal control command. The control command adjustment module is used to determine safety constraints through the safety determination layer based on the nominal control command, the maneuver parameters of the vessel to be moored, and the risk quantification value. When the nominal control command is unsafe, a feasible region is constructed based on the nominal control command and the dynamic model of the vessel to be moored. The QP optimizer searches for candidate solutions that satisfy the safety constraints in the feasible region. The nominal control command is adjusted based on the candidate solutions to obtain the action control command after the quadratic programming. The risk quantification value is determined based on the real-time status of the vessel to be moored and the geometric information of the dock shoreline.

7. A computer device, comprising a memory, a processor, and a computer program stored in the memory, characterized in that, The processor executes the computer program to implement the steps of the automatic berthing control method for ships according to any one of claims 1 to 5.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is loaded by the processor, it is able to execute the steps of the automatic berthing control method for ships as described in any one of claims 1 to 5.

Images