Ship autonomous navigation and obstacle avoidance method

Abstract

The invention discloses a ship autonomous navigation and obstacle avoidance method, which comprises the steps of constructing a simulation channel environment model and generating a channel image, constructing a deep reinforcement learning model improved based on a near-end strategy optimization algorithm, inputting the channel image into the deep reinforcement learning model to generate a rudder angle signal for controlling a ship, and performing ship autonomous navigation and obstacle avoidance. And correcting the course of the ship according to the rudder angle signal, constructing a nonlinear ship motion mathematical model adapted to the inland waterway, inputting the rudder angle signal into the nonlinear ship motion mathematical model to update the state information of the ship, and feeding back the updated state information of the ship to the simulated waterway environment model to form closed-loop control. Through the process, the autonomous navigation and obstacle avoidance capability of the ship in the complex channel environment of the inland river is realized.

Description

Technical Field

[0001] The present invention relates to a method for ship navigation and obstacle avoidance, and in particular to a method for autonomous ship navigation and obstacle avoidance. Background Art

[0002] With the rapid development of the shipping industry and the widespread application of intelligent technologies, autonomous ship navigation has gradually become a hot topic in academic research and engineering practice. However, traditional ship control methods, which mostly rely on preset route planning and rule-based control, have poor adaptability to complex and changing environments and are unable to cope with dynamic obstacles or rapidly changing waterway conditions. Furthermore, such methods often require extensive human intervention and expert experience, making it difficult to achieve truly intelligent autonomous control.

[0003] In recent years, deep reinforcement learning technology has demonstrated tremendous potential in the field of intelligent control. By simulating real-world environments, intelligent agents can continuously learn and optimize control strategies through interaction with the environment. However, existing deep reinforcement learning methods for autonomous ship navigation applications still face numerous challenges, such as insufficiently realistic environmental modeling, inefficient algorithm training, and insufficient integration of dynamic characteristics with control algorithms. Furthermore, in actual navigation environments, ship control must balance obstacle avoidance, navigation efficiency, and route centrality, placing higher demands on the algorithm's stability and adaptability. Therefore, the industry needs to develop a method for intelligent autonomous navigation and obstacle avoidance for ships that integrates image processing and dynamic modeling. Summary of the Invention

[0004] In view of the above-mentioned deficiencies in current ship navigation and obstacle avoidance, the present invention provides a method for autonomous ship navigation and obstacle avoidance that integrates visual perception and nonlinear modeling, so as to solve the problems in the existing technology of insufficient adaptability to complex waterway environments, reliance on human intervention, and low algorithm stability and efficiency.

[0005] To achieve the above-mentioned objectives, the first aspect of the present invention provides a method for autonomous navigation and obstacle avoidance of ships, a method for autonomous navigation and obstacle avoidance of ships, characterized in that it includes: constructing a simulated channel environment model and generating a channel image, constructing a deep reinforcement learning model improved based on a proximal policy optimization algorithm, inputting the channel image into the deep reinforcement learning model to generate a rudder angle signal for controlling the ship, correcting the ship's heading according to the rudder angle signal, constructing a nonlinear ship motion mathematical model adapted to inland waterways, inputting the rudder angle signal into the nonlinear ship motion mathematical model to update the ship's status information, and feeding back the updated ship status information into the simulated channel environment model to form a closed-loop control.

[0006] In some embodiments according to the first aspect of the present invention, the simulated channel image is generated by Bezier curve fitting and includes channel boundaries, obstacles and relative position information of the ship, and the ship is initially located at the center of the starting position of the channel.

[0007] In some embodiments of the first aspect of the present invention, the deep reinforcement learning model is constructed using an Actor-Critic network, including a feature extraction network, an Actor network and a Critic network. The feature extraction network is used to extract features from the input channel image and output a multi-channel feature map, and then convert the multi-channel feature map into a one-dimensional feature vector through a flattening layer. The Actor network and the Critic network both use the one-dimensional feature vector output by the feature extraction network as input data. The Actor network outputs a rudder angle signal for controlling the ship, and the Critic network is used to evaluate the actions performed by the Actor network.

[0008] In some embodiments of the first aspect of the present invention, the feature extraction network is constructed by an LCNN neural network, including two convolutional layers, the first convolutional layer uses 16 8*8 convolution kernels, a stride of 4, and a padding of 0, and the second convolutional layer uses 32 3*3 convolution kernels, a stride of 2, and a padding of 0.

[0009] In some embodiments of the first aspect of the present invention, in the Actor-Critic network, the GAE algorithm is used to generate a value function estimate to evaluate the expected accumulated reward after taking a certain rudder angle in the current state, thereby guiding the strategy update. The reward function sets the reward item in combination with the ship's navigation status information and channel information, and DLR is used for model training to improve training efficiency.

[0010] In some embodiments according to the first aspect of the present invention, the reward item includes the absolute value θ of the difference between the deviation angle θ2 of the channel and the true north direction and the yaw angle θ1 of the ship. abs The minimum distance d between the center point of the ship and all the center points of the channel, according to the θ abs The size of d gives a reward value R1, and the size of d gives a reward value R2.

[0011] In some embodiments according to the first aspect of the present invention, the calculation formula of the reward value R1 is: abs If the minimum distance d is less than 5 degrees, the reward value is 3; if it is between 5 and 12 degrees, the reward value is 1; otherwise, the reward value is -3; the calculation formula of the reward R2 is: if the minimum distance d is less than 5 units, the reward value is 3; if it is between 5 and 12 units, the reward value is 2; if it is between 12 and 20 units, the reward value is 0; otherwise, the reward value is -3.

[0012] In some embodiments according to the first aspect of the present invention, the nonlinear ship motion mathematical model is constructed by a Fossen dynamics ship motion mathematical model according to input data.

[0013] In some embodiments according to the first aspect of the present invention, the input data includes shape parameters, propulsion equipment parameters, and maneuvering equipment parameters of the vessel.

[0014] In some embodiments of the first aspect of the present invention, the deep reinforcement learning model also includes a sub-evaluation network, the original network of the deep reinforcement learning model is a main training network, the network structure of the sub-evaluation network is the same as the network structure of the main training network, the main training network is responsible for generating actions and updating them according to environmental feedback, and the sub-evaluation network is used to calculate importance sampling weights, evaluate the quality of the main training network and limit the amplitude of policy updates to avoid excessive policy changes.

[0015] As described above, the present invention's autonomous navigation and obstacle avoidance method for ships has the following beneficial effects: First, the present invention uses a lightweight convolutional neural network (LCNN) for feature extraction, providing high-quality input features for the deep reinforcement learning model. By introducing dynamic learning rates (DRL) and generalized advantage estimation (GAE), the model achieves rapid convergence while also improving generalization and reducing parameter tuning complexity. Second, by combining policy gradient optimization with value function estimation, it generates real-time and accurate rudder angle control commands, ensuring efficient navigation and obstacle avoidance capabilities for ships in dynamic environments. Third, by designing a positive incentive reward function, it guides the ship's optimized control strategy, rapidly mastering path-keeping, obstacle avoidance, and smooth navigation capabilities, significantly improving navigation performance in complex waterway environments. Finally, by combining an improved nonlinear mathematical model of ship motion, it accurately simulates dynamic characteristics such as ship heading angle, velocity, and position changes, ensuring high consistency between control decisions and actual ship behavior, and ensuring the credibility and transferability of simulation results. Furthermore, the present invention adopts a modular design, independently separating environmental modeling, algorithm optimization, and dynamic modeling, resulting in excellent adaptability and scalability. It is applicable not only to inland waterways but also to a variety of scenarios such as ports and marine environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0017] Figure 1 This is a flow chart of the method for autonomous navigation and obstacle avoidance of a ship according to the present invention;

[0018] Figure 2 This is a schematic diagram of the network structure of the deep reinforcement learning model described in the present invention;

[0019] Figure 3 Schematic diagram of the overall architecture of the deep reinforcement learning model with a secondary evaluation network according to the present invention;

[0020] Figure 4 This is a panoramic schematic diagram of the simulated waterway with obstacles according to the present invention;

[0021] Figure 5 Schematic diagram of the reward and punishment strategy principle of the present invention. DETAILED DESCRIPTION

[0022] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0023] Before further explaining the present invention in detail, the nouns and terms involved in the embodiments of the present invention are explained. The nouns and terms involved in the embodiments of the present invention are subject to the following interpretations:

[0024] Dynamic Learning Rate (DLR) is an important technique in neural networks and other machine learning algorithms that dynamically adjusts the learning rate during training. The learning rate is a key parameter in optimization algorithms, determining the step size of each parameter update. Dynamic learning rate adjustment aims to adaptively adjust the learning rate based on the model's training state, improving the model's convergence speed, stability, and generalization ability.

[0025] Generalized Advantage Estimation (GAE) is a method for efficiently estimating the advantage function in reinforcement learning. Proposed by John Schulman et al. in 2016, GAE aims to reduce estimation variance and improve sample efficiency by balancing the uncertainty of immediate rewards and future returns. It is widely used in deep reinforcement learning (e.g., PPO and TRPO). It is particularly useful in policy gradient methods, such as Proximal Policy Optimization (PPO) in the actor-critic framework. By incorporating multi-step temporal difference (TD) errors, GAE strikes a balance between bias and variance, thereby optimizing the policy learning process.

[0026] Actor-Critic is a reinforcement learning algorithm that combines the Policy Gradient method with the Value Function method. It improves learning efficiency and stability by simultaneously training a policy network (the actor) and a value network (the critic). The Actor-Critic algorithm has been widely used in fields such as robotic control, game intelligence, autonomous driving, and autonomous ship navigation.

[0027] Lightweight Convolutional Neural Networks (LCNNs) are convolutional neural networks designed to reduce computational complexity and model size while maintaining high performance. They aim to achieve a balance between high accuracy and computational efficiency. These networks are particularly well-suited for deployment in resource-constrained environments, such as mobile devices, embedded systems, or Internet of Things (IoT) devices.

[0028] The first embodiment of the present invention:

[0029] Figure 1 A flow chart of a method for autonomous navigation and obstacle avoidance of a ship according to an embodiment of the present invention is shown, which includes the following steps:

[0030] Step S1: Construct a simulated waterway environment model and generate a waterway image.

[0031] This embodiment of the present invention uses a two-dimensional planar waterway from a top-down perspective as the simulation environment. The channel model is constructed using a Canvas plane based on the Python basic library. Bezier curves are also incorporated to generate complex and diverse channel shapes, providing highly flexible and diverse environmental data for model training. The ship is initially positioned at the center of the channel's starting position to ensure uniformity and reproducibility during the simulation training process.

[0032] The simulated channel environment model can dynamically generate a local channel image centered on the ship based on the ship's environmental information. In the embodiment of the present invention, the resolution of the channel image is 116*116 pixels, and the channel image includes key environmental features such as channel boundaries, obstacles, and relative positions of ships.

[0033] Figure 4 A panoramic image of a simulated waterway with obstacles is shown. The image shows the waterway image, which includes both the shore and waterway environments. The red dots in the waterway environment represent simulated obstacles, which are randomly generated. Obstacles are replaced with a central circle, simplifying the geometric representation to reduce computational complexity. The blue area represents the navigable inland waterway, while the orange area represents the impassable area.

[0034] The Bézier curve is a mathematical curve widely used in computer graphics, animation design, and geometric modeling to draw smooth curves and paths. It was proposed by French engineer Pierre Bézier in the 1960s and was originally used in automobile body design. Defined by a set of control points, the Bézier curve is smooth, flexible, and easy to calculate.

[0035] Step S2: Build a deep reinforcement learning model based on the improved proximal policy optimization algorithm.

[0036] In an embodiment of the present invention, the deep reinforcement learning model adopts a lightweight convolutional neural network LCNN to improve the proximal policy optimization algorithm (PPO), and introduces generalized advantage estimation (GAE) and dynamic learning rate (DLR).

[0037] In an embodiment of the present invention, the deep reinforcement learning model includes a feature extraction network and a main training network.

[0038] The feature extraction network is built based on the LCNN convolutional neural network, which is used to extract features from the input channel image. The features include key feature information such as channel boundaries, obstacle distribution and relative positions of ships, and finally output a multi-channel feature map including key feature information. Figure 2 As shown in FIG, the feature extraction network also sets a flattening layer in the output segment. The flattening layer converts the output multi-channel feature map into a one-dimensional feature vector, which is used as the input data of the subsequent fully connected network.

[0039] In this embodiment of the present invention, the feature extraction network includes two convolutional layers. The first convolutional layer uses 16 8*8 convolution kernels with a stride of 4 and a padding of 0. The second convolutional layer uses 32 3*3 convolution kernels with a stride of 2 and a padding of 0.

[0040] The main training network is constructed using the Actor-Critic algorithm framework. The input layers of the main training network are all fully connected networks. The one-dimensional feature vectors output by the feature extraction network are used as input feature data, and the GAE algorithm is used to generate value function estimates. The value function includes an action value function and a state value function. This function estimates the expected accumulative reward after adopting a certain steering angle in the current state, thereby guiding policy updates and improving the reliability and efficiency of control decisions.

[0041] The advantage function calculated using GAE in the embodiment of the present invention is as follows:

[0042]

[0043] In the calculation formula, γ is the discount factor of the estimated reward, and λ is the trade-off parameter of the GAE function, which is usually set to a value close to 1. is the adjacent timing difference error.

[0044] in, where r t+1 is the environmental reward obtained at time t+1, V π (s t+x ) is the estimated value of the state at time t+x, that is, the critic network's score for the action performed at time t+x. To prevent excessive policy updates, a truncated probability ratio mechanism is used. Through extensive training and comparison, the optimized hyperparameter ∈ is determined to limit the range of the probability ratio so that it does not deviate too far from the old policy. A dual-objective loss function is adopted: one using the original probability ratio ρ and the other using the truncated probability ratio. The final loss function is the minimum of these two objectives:

[0045] L clip (θ)=E[min(ρG t ,clip(ρ,1-,1+)G t )]

[0046] Among them, G t It is the advantage function calculated by GAE, which effectively reduces variance and bias, provides a more stable and accurate advantage function, and improves sample efficiency and accelerates convergence, making the algorithm more efficient and stable in complex tasks.

[0047] The dynamic learning rate (DLR) used in the deep reinforcement learning model of the present invention is used for model training to improve the efficiency and effectiveness of training. Specifically, the learning rate is dynamically adjusted by setting a threshold for relative entropy, also known as the Kullback-Leibler Divergence (KL) value. When the KL value exceeds the threshold, the learning rate is increased to accelerate convergence. Conversely, the learning rate is reduced to avoid over-optimization. This mechanism improves the training efficiency and stability of the model.

[0048] The reward function of the Actor-Critic algorithm combines the ship's navigation status and channel information, and sets reward items to guide the ship to stay in the center of the channel and avoid collisions with obstacles. At the same time, a reward function that integrates channel information, the ship's physical characteristics, navigation status and positive incentives is introduced to ensure the ship's stable navigation in complex environments.

[0049] according to Figure 5 As shown, the embodiment of the present invention sets two reward items to guide the training of the ship. One is the absolute value θ of the difference between the deviation angle θ2 of the channel and the true north direction and the yaw angle θ1 of the ship. abs , with θ abs As a reference, the reward value R1 is given. The calculation formula of R1 is as follows:

[0050]

[0051] If θ abs If the value is less than 5 degrees, the reward is 3; if it is between 5 and 12 degrees, the reward is 1; otherwise, the reward is -3.

[0052] The second is the minimum distance d between the center point of the ship and all the center points of the channel. The value of d is:

[0053] d=min(d1,d2,d3,...,d n )

[0054] The reward value R2 is given according to the size of d. The R2 calculation formula is as follows:

[0055]

[0056] If the minimum distance d is less than 5 units, the reward is 3; if it is between 5 and 12 units, the reward is 2; if it is between 12 and 20 units, the reward is 0; otherwise, the reward is -3.

[0057] In this embodiment, GAE (Generalized Advantage Estimation) is introduced to adaptively adjust the baseline and accelerate the convergence of the loss function. LDR (Dynamic Learning Rate) is used to flexibly adjust the step size at different training stages, improving convergence speed and stability, enhancing generalization, escaping local optima, and reducing the complexity of parameter adjustment, thereby significantly optimizing model performance.

[0058] The deep reinforcement learning model is continuously trained and optimized in a simulation environment. When the training results meet the requirements, the intelligent model is exported and deployed to the target device.

[0059] Step S3: Input the channel image into the deep reinforcement learning model to generate a rudder angle signal for controlling the ship.

[0060] The feature extraction network in the deep reinforcement learning model extracts features from the input channel image. The feature data includes key environmental features such as channel boundaries, obstacle distribution, and relative positions of ships.

[0061] In the embodiment of the present invention, combined with the parameter setting of the feature extraction network introduced above, for a waterway image with a resolution of 116*116, the first convolution layer outputs a 16-channel feature map of 28*28, the second convolution layer outputs a 32-channel feature map of 13*13, and after the flattening layer, the output is a one-dimensional feature vector of 13*13*32.

[0062] The one-dimensional feature vector is input into the main training network and the sub-evaluation network, and finally the rudder angle signal for controlling the ship is generated through policy gradient optimization.

[0063] The process of feature extraction, rudder angle information output and value function estimation generation has been introduced in detail in the construction process of the deep reinforcement learning model and will not be repeated here.

[0064] Step S4: Correct the ship's heading according to the rudder angle signal.

[0065] The ship manipulation agent model, trained through reinforcement learning, is continuously optimized in a simulation environment, enabling it to adapt to complex waterway conditions. Based on the rudder angle signals provided by the deep reinforcement learning prediction model, the agent can autonomously adjust the ship's course, effectively avoid obstacles, and maintain stable navigation in the center of the waterway. This enables the ship's autonomous navigation and obstacle avoidance capabilities, ensuring safe and accurate navigation. In this way, the agent not only ensures that the ship stays on the planned route but also responds promptly to potential hazards, ensuring smooth and safe navigation.

[0066] Step S5: Construct a nonlinear ship motion mathematical model suitable for inland waterways.

[0067] The embodiment of the present invention constructs a nonlinear ship motion mathematical model adapted to inland waterways based on the ship motion mathematical model of Fossen dynamics. The formula for calculating the ship coordinates of this mathematical model is as follows:

[0068]

[0069] Among them, (x t+1 ,y t+1 ) is (x t ,y t ) in Δ t The new coordinates calculated under the interval, is the calculated heading. μ is the correction factor for inland waterways and can be set between 0.95 and 1.10. This improved calculation formula maintains reasonable accuracy while being suitable for real-time simulation of inland waterway scenarios.

[0070] When constructing a mathematical model of ship motion based on Fossen dynamics, it is necessary to systematically input the physical characteristics data of the ship based on a multi-dimensional parameter system. This process covers three core parameter categories: ship shape parameters (such as waterline length, cross-sectional area, and hull linear coefficient), propulsion equipment parameters (propeller diameter, thrust curve, and torque characteristics), and control equipment parameters (rudder efficiency coefficient and nonlinear characteristics of steering torque). By combining high-precision measurement with simulation calibration, the dynamic response characteristics of the ship in still water and waves are fully characterized. On this basis, the model is adaptively modified for the special working conditions of inland waterways (such as shallow water effects and bend water flow interference), and finally a nonlinear ship motion mathematical model suitable for inland waterways is constructed.

[0071] The Fossen dynamics mathematical model of ship motion is a mathematical model widely used in the field of ship dynamic modeling and control. It was proposed by Norwegian scholar Jan Fossen. Based on hydrodynamic theory, this model takes into account the mass, buoyancy, resistance, and nonlinear factors of the hull, and can accurately describe the motion characteristics of unmanned vessels or ships in water.

[0072] Step S6: inputting the rudder angle signal into the nonlinear ship motion mathematical model to update the ship's state information, and feeding back the updated ship state information into the simulated channel environment model to form a closed-loop control.

[0073] Specifically, the dynamic simulation of a nonlinear mathematical model of ship motion adapted to inland waterways can predict the ship's trajectory based on the input rudder angle signal and update the ship's status information, including heading angle and position, in real time based on the predicted trajectory. This updated status information is fed back to the simulated channel environment model, which generates a local channel image centered on the ship based on the input ship status information, forming a closed-loop control mechanism. This enables continuous movement and precise maneuvering of the ship in a dynamic environment, thereby improving the real-time performance and reliability of the autonomous navigation system.

[0074] It can be understood that through the continuous cycle of the above steps S1 to S6, the ship control intelligent agent model can continuously correct the ship's navigation according to the rudder angle signal output by the deep reinforcement learning model to achieve autonomous navigation.

[0075] The second embodiment of the present invention:

[0076] In order to improve the stability of the deep reinforcement learning model during the policy update process, the second embodiment of the present invention adds a secondary evaluation network to the deep reinforcement learning model based on the first embodiment. The specific solution is as follows:

[0077] The secondary evaluation network is constructed with reference to the main training network. The network structure of the secondary evaluation network is the same as that of the main training network, that is, the secondary evaluation network also uses the Actor-Critic algorithm framework, the input layer of the secondary evaluation network is also a fully connected network, and the number of network layers and neurons is consistent with that of the main training network.

[0078] By introducing a secondary evaluation network, the ship autonomous navigation and obstacle avoidance method of the present invention forms a deep reinforcement learning algorithm combination of DLR-GAE-DPPO. Here, DPPO represents a PPO architecture mode using a dual network of a main training network and a secondary evaluation network.

[0079] like Figure 3 As shown in the figure, after adding the secondary evaluation network, the main training network is responsible for generating actions and updating them based on environmental feedback. The secondary evaluation network is used to calculate importance sampling weights, evaluate the quality of the main training network, and limit the amplitude of policy updates to avoid excessive policy changes.

[0080] In an embodiment of the present invention, the main training network and the secondary evaluation network each have their own network parameters. After the main training network reaches the set exploration generation during the training process, the main training network parameters are frozen, and the validity of the explored network parameters is evaluated in the secondary evaluation network. θ is the network parameter explored in the latest round, and θ' is the optimal network parameter explored in the previous round. In the secondary evaluation network, if the reward value evaluated using the network parameter θ is higher than the reward of the θ' parameter, it is considered that the exploration value of the new round is higher and the trained parameters are more effective. The network parameters θ trained by the main training network are saved in the secondary evaluation network. Otherwise, it is considered that the θ parameter is not as good as the θ' parameter, that is, the training effect of the latest round is not good, and it is discarded, and the network parameters θ' of the secondary evaluation network are continued to be used for training the main network.

[0081] By constructing two Actor-Critic frameworks for deep reinforcement learning models, in an embodiment of the present invention, these two Actor-Critic frameworks correspond to the main training network and the secondary evaluation network, respectively, helping the intelligent agent to better explore different strategies. During the training process, by balancing exploration and exploitation, the model maintains a certain degree of diversity, preventing the model from converging to a local optimal solution too early, thereby finding a better solution.

[0082] In summary, the present invention firstly uses a lightweight convolutional neural network (LCNN) for feature extraction to provide high-quality input features for the deep reinforcement learning model. By introducing a dynamic learning rate (DRL) and generalized advantage estimation (GAE), the model achieves rapid convergence while also improving generalization and reducing parameter tuning complexity. Secondly, by combining policy gradient optimization with value function estimation, it generates real-time and accurate rudder angle control commands, ensuring efficient navigation and obstacle avoidance capabilities for ships in dynamic environments. Thirdly, by designing a positive incentive reward function, it guides the ship's optimized control strategy, rapidly mastering path-keeping, obstacle avoidance, and smooth navigation capabilities, significantly improving navigation performance in complex waterway environments. Finally, by combining an improved nonlinear mathematical model of ship motion, it accurately simulates dynamic characteristics such as ship heading angle, velocity, and position changes, ensuring high consistency between control decisions and actual ship behavior, and ensuring the credibility and transferability of simulation results. Furthermore, the present invention adopts a modular design that independently separates environmental modeling, algorithm optimization, and dynamic modeling, resulting in excellent adaptability and scalability. It is applicable not only to inland waterways but also to a variety of scenarios such as ports and marine environments. In summary, the technical solution of the present invention significantly improves the autonomous navigation and control capabilities of ships in complex inland waterway environments, and has important application value and broad promotion prospects.

[0083] The above is a detailed introduction to the method for autonomous navigation and obstacle avoidance of ships that integrates visual perception and nonlinear modeling provided by the present invention. Specific examples are used in this article to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core ideas. At the same time, for those skilled in the art, according to the ideas of the present invention, there will be changes in the specific implementation methods and application scopes. In summary, the content of this specification should not be understood as limiting the present invention.

Claims

1. A method for autonomous navigation and obstacle avoidance of a ship, characterized in that: include: Construct a simulated waterway environment model and generate waterway images; Build a deep reinforcement learning model based on an improved proximal policy optimization algorithm; Inputting the channel image into the deep reinforcement learning model to generate a rudder angle signal for controlling the ship; Correcting the ship's heading according to the rudder angle signal; Construct a nonlinear mathematical model of ship motion suitable for inland waterways; The rudder angle signal is input into the nonlinear ship motion mathematical model to update the ship's state information, and the updated ship state information is fed back into the simulated channel environment model to form a closed-loop control.

2. The method for autonomous navigation and obstacle avoidance of a ship according to claim 1, characterized in that: The simulated channel image is generated by Bezier curve fitting and contains channel boundaries, obstacles and ship relative position information. The ship is initially located at the center of the starting position of the channel.

3. The method for autonomous navigation and obstacle avoidance of a ship according to claim 1, characterized in that: The deep reinforcement learning model is constructed using an Actor-Critic network, including a feature extraction network, an Actor network, and a Critic network; The feature extraction network is used to extract features from the input channel image and output a multi-channel feature map, and then convert the multi-channel feature map into a one-dimensional feature vector through a flattening layer; The Actor network and the Critic network both use the one-dimensional feature vector output by the feature extraction network as input data; The Actor network outputs a rudder angle signal for controlling the ship; The Critic network is used to evaluate the actions performed by the Actor network.

4. The method for autonomous navigation and obstacle avoidance of a ship according to claim 3, characterized in that: The feature extraction network is constructed using an LCNN neural network, which includes two convolutional layers. The first convolutional layer uses 16 8*8 convolution kernels with a stride of 4 and a padding of 0, and the second convolutional layer uses 32 3*3 convolution kernels with a stride of 2 and a padding of 0.

5. The method for autonomous navigation and obstacle avoidance of a ship according to claim 3, characterized in that: In the Actor-Critic network: Use the GAE algorithm to generate a value function estimate to evaluate the expected cumulative reward after taking a certain steering angle in the current state, thereby guiding the strategy update; The reward function sets the reward item based on the ship's navigation status information and channel information; DLR is used for model training to improve training efficiency.

6. The method for autonomous navigation and obstacle avoidance of a ship according to claim 5, characterized in that: The reward item includes the absolute value θ of the difference between the deviation angle θ2 of the channel and the true north direction and the yaw angle θ1 of the ship. abs The minimum distance d between the center point of the ship and all the center points of the channel, according to the θ abs The size of d gives a reward value R1, and the size of d gives a reward value R2.

7. The method for autonomous navigation and obstacle avoidance of a ship according to claim 6, characterized in that: The calculation formula of the reward value R1 is: abs If it is less than 5 degrees, the reward value is 3; if it is between 5 and 12 degrees, the reward value is 1; otherwise, the reward value is -3; The calculation formula of the reward R2 is: if the minimum distance d is less than 5 units, the reward value is 3, if it is between 5 and 12 units, the reward value is 2, if it is between 12 and 20 units, the reward value is 0, otherwise, the reward value is -3.

8. The method for autonomous navigation and obstacle avoidance of a ship according to any one of claim 1, characterized in that: The nonlinear ship motion mathematical model is constructed by the ship motion mathematical model of Fossen dynamics according to input data.

9. The method for autonomous navigation and obstacle avoidance of a ship according to claim 8, characterized in that: The input data includes the ship's shape parameters, propulsion equipment parameters and control equipment parameters.

10. The method for autonomous navigation and obstacle avoidance of a ship according to any one of claims 1 to 9, characterized in that: The deep reinforcement learning model also includes a secondary evaluation network, and the original network of the deep reinforcement learning model is a main training network; The network structure of the secondary evaluation network is the same as the network structure of the main training network; The main training network is responsible for generating actions and updating them based on environmental feedback; The secondary evaluation network is used to calculate importance sampling weights, evaluate the quality of the main training network and limit the amplitude of policy updates to avoid excessive policy changes.

Images