Unmanned ship autonomous navigation method and decision making system based on laser radar real-time perception

Abstract

The invention discloses an unmanned ship autonomous navigation method and decision making system based on laser radar real-time perception, and relates to the field of unmanned ship autonomous navigation.The unmanned ship autonomous navigation method comprises the steps that unmanned ship state information, terminal position information and two-dimensional laser radar data are collected, and position information of a terminal relative to an unmanned ship is calculated; normalizing the state information of the unmanned ship and the position information of the terminal point relative to the unmanned ship, and fusing with radar feature information extracted from the normalized two-dimensional laser radar data; inputting the fused information into a trained unmanned ship autonomous navigation model, outputting an expected yawing angle and an expected speed of the unmanned ship, and calculating the propelling speed of each propeller by combining a kinematic model and a kinetic model of the unmanned ship; and judging whether the unmanned ship arrives at a terminal point, if so, ending navigation, and otherwise, continuing the navigation process. According to the method, autonomous learning navigation can be realized in an unknown or dynamic environment, and the method has higher environment adaptability and decision optimization capability.

Description

Technical Field

[0001] The present invention relates to the technical field of autonomous navigation of unmanned vessels, and more particularly to an autonomous navigation method and decision-making system for unmanned vessels based on real-time perception of lidar. Background Art

[0002] With the increasing depth of ocean resource development, the complexity and risk of maritime tasks have increased significantly, posing unprecedented challenges to the autonomy and intelligence levels of maritime unmanned systems. Unmanned vessels, as a key component of maritime unmanned systems, demonstrate great potential in high-risk and high-difficulty tasks such as port cruising and environmental monitoring, relying on their high degree of automation, excellent maneuverability, and multifunctional characteristics. However, in complex marine environments, unmanned vessels not only need to accurately navigate to preset targets but also must perceive the surrounding environment in real time and effectively avoid various maritime obstacles to ensure the safety and efficiency of task execution. Therefore, studying an efficient and reliable autonomous navigation method is of great significance for improving the application efficiency and practical deployment value of unmanned vessels.

[0003] Currently, the autonomous navigation of unmanned vessels mainly relies on path planning methods based on global information, such as the ant colony algorithm (ACO) and the artificial potential field method (APF). The ant colony algorithm simulates the collaborative behavior of ant colonies to find the optimal path based on global environmental information and is suitable for path planning in static or known environments; the artificial potential field method constructs a navigation potential field using the attraction of the target point and the repulsion of obstacles, causing the unmanned vessel to move along the direction of gradient descent. However, these methods usually rely on a complete or partially known environmental map and are difficult to adapt to dynamic and uncertain marine environments. For example, in complex waters, environmental information may change at any time, and the path planning method based on the global map is difficult to adjust the path in a timely manner, thereby affecting navigation efficiency and safety.

[0004] Therefore, how to enable unmanned vessels to navigate autonomously without knowing the map information, achieve flexible and intelligent obstacle avoidance and target navigation, and improve environmental adaptability and task execution reliability are technical problems that need to be urgently solved by those skilled in the art. Summary of the Invention

[0005] In view of this, the present invention provides an autonomous navigation method and decision-making system for unmanned vessels based on real-time perception of lidar, which solves the problems existing in the background art.

[0006] To achieve the above object, the present invention provides the following technical solutions:

[0007] An autonomous navigation method for unmanned vessels based on real-time perception of lidar, comprising the following steps:

[0008] S1: Collect the state information of the unmanned vessel, the end position information, and the two-dimensional lidar data;

[0009] S2: Calculate the position information of the end point relative to the unmanned ship based on the state information of the unmanned ship and the end point position information;

[0010] S3: Normalize the two-dimensional lidar data, and use a deep neural network to extract features from the normalized two-dimensional lidar data to obtain lidar feature information;

[0011] S4: Normalize the state information of the unmanned ship and the position information of the end point relative to the unmanned ship, and fuse them with the lidar feature information;

[0012] S5: Input the fused information into the trained unmanned ship autonomous navigation model, and output the desired yaw angle and desired speed of the unmanned ship;

[0013] S6: Input the desired yaw angle and desired speed of the unmanned ship into the low-level controller of the unmanned ship, and calculate the propulsion speed of each thruster in combination with the kinematic model and dynamic model of the unmanned ship;

[0014] S7: Determine whether the unmanned ship has reached the end point. If it has reached the end point, the navigation ends. Otherwise, return to S1 to continue the navigation process.

[0015] Optionally, in S1, the state information of the unmanned ship includes the coordinates (x u , y u ), yaw angle ψ u and the speed of the unmanned ship (u u , v u , r u ); the end point position information is the coordinates of the end point (x t , y t ); the two-dimensional lidar data is represented as is the measurement data of a lidar with an angular range of n s . Each scan provides n L sets of angle-distance pairs. Among them, n L represents the number of ranging beams of the lidar, and the maximum detection distance of the lidar is L max .

[0016] Optionally, in S2, the position information of the end point relative to the unmanned ship includes the distance d between the unmanned ship and the end point t and the azimuth angle ψ of the end point relative to the unmanned ship t . The specific calculation formula is:

[0017]

[0018] The deviation between the yaw angle ψ of the unmanned ship u and the azimuth angle ψ of the end point relative to the unmanned ship t is the relative course angle δψ is:

[0019] δ Ψ = Ψ u - Ψ t

[0020] where: (x t , y t ) is the coordinate of the end point, and (x u , y u ) is the coordinate of the unmanned ship.

[0021] Optionally, in S3, the two-dimensional lidar data is normalized as follows:

[0022] The distance information is normalized by dividing by the maximum detection distance L of the lidar max , and the angle information is normalized by dividing by the angular range n of the lidar s .

[0023] Optionally, in S4, the unmanned ship state information and the position information of the end point relative to the unmanned ship are normalized as follows:

[0024] The speed (u u , v u , r u ) and the yaw angle ψ u of the unmanned ship are normalized by dividing by the corresponding maximum values respectively. The coordinate (x u , y u ) of the unmanned ship is normalized by dividing by the width W and length L of the map. The distance d t between the unmanned ship and the end point is normalized by dividing by the diagonal length of the map, and the relative course angle δ ψ is normalized by dividing by 2π.

[0025] Optionally, in S5, the unmanned ship autonomous navigation model is trained based on the double-delayed deep deterministic policy gradient framework, which specifically includes the following steps:

[0026] S51. State space definition: The state space is the input of the unmanned ship autonomous navigation model and contains all the information on which decisions depend. It is defined as follows:

[0027] s = <o u , o t , o a , o l >

[0028] where: o u represents the real-time state information of the unmanned ship, including the position information (x u , y u , ψ u) and speed information (u u , v u , r u );o t Represents the relative position information of the end point, that is, the distance d between the unmanned ship and the end point t and the relative course angle δ ψ ;o a Represents the execution action of the unmanned ship in the previous time step o l Represents the lidar feature information after dimensionality reduction processing;

[0029] S52. Action space definition: The action space is the output of the unmanned ship autonomous navigation model, which is compatible with the characteristics of the underlying controller, and is set as follows:

[0030] a = <u d , ψ d >

[0031] In the formula: u d Represents the desired surge speed, ψ d Represents the desired yaw angle;

[0032] Considering the constraint characteristics of the underlying controller, the following limiting conditions are imposed on the action space:

[0033]

[0034] In the formula: u max Represents the maximum surge speed of the unmanned ship, ψ max Represents the maximum yaw angle of the unmanned ship;

[0035] S53. Set the reward mechanism: The reward function is a decision-driven mechanism that guides the unmanned ship to execute the expected behavior pattern.

[0036] Optionally, in S53, the reward mechanism is set according to the state space and the action space, specifically including:

[0037] 1) Target point reward mechanism: Set the target point reward to encourage the unmanned ship to navigate towards the target point; The target point reward satisfies the following conditions:

[0038]

[0039] In the formula: r safe Represents the safety zone radius of the unmanned ship, which is used to judge whether the unmanned ship reaches the target point or collides; When the distance d between the unmanned ship and the end point t is less than or equal to r safe , it is determined that the unmanned ship has successfully reached the target point, and a positive reward r arrival is given;

[0040] 2) Collision penalty mechanism: Set a collision penalty to prompt the unmanned ship to avoid obstacles and maintain safe navigation; the collision penalty meets the following conditions:

[0041]

[0042] In the formula: d o represents the distance between the unmanned ship and the obstacle; when the distance d o between the unmanned ship and the obstacle is less than or equal to the safety distance r safe , it is determined that a collision occurs and a negative reward r collision is imposed;

[0043] 3) Timeout penalty mechanism: Set a timeout penalty to prevent the unmanned ship from executing inefficient strategies; the timeout penalty meets the following conditions:

[0044]

[0045] In the formula: t o represents the set time threshold; if the current time step t is greater than or equal to t o , it is determined that a timeout occurs and a negative reward r timeout is imposed;

[0046] 4) Out-of-bounds penalty mechanism: Set an out-of-bounds penalty to ensure that the unmanned ship sails within the specified area; the out-of-bounds penalty meets the following conditions:

[0047]

[0048] In the formula: |x e | and |y e | are respectively the boundary ranges of the navigation area; when the current position (x u , y u ) of the unmanned ship exceeds the boundary range, a negative reward r boundary is imposed;

[0049] 5) Potential field reward mechanism: Set a potential field reward to guide the unmanned ship to actively avoid obstacles while approaching the end point; the potential field reward meets the following conditions:

[0050]

[0051] R p = R att + R rep

[0052] In the formula: k att represents the gravitational potential field coefficient, k rep represents the repulsive potential field coefficient; the unmanned ship obtains a positive reward R att provided by the gravitational field and a negative reward R rep provided by the repulsive field at each time step;

[0053] 6) Smooth motion reward mechanism: Set the smooth motion reward to suppress the sudden change of the actions of the unmanned ship between adjacent time steps; the smooth motion reward satisfies the following conditions:

[0054]

[0055] where: u d and ψ d respectively represent the actions taken at the current time step, u′ d and ψ′ d respectively represent the actions taken at the previous time step, and σ u and σ ψ represent the adjustment parameters;

[0056] Based on the above reward and punishment mechanisms, the final reward function is defined as follows:

[0057] R = λ0R d + λ1R o + λ2R e + λ3R b + λ4R p + λ5R a

[0058] where: λ0, λ1, λ2, λ3, λ4, λ5 respectively represent the corresponding weight coefficients.

[0059] Optionally, in S6, the bottom controller of the unmanned ship adopts PID, and the expression is as follows:

[0060]

[0061] where: K p 、K i and K d respectively represent the proportional coefficient, integral coefficient and differential coefficient; e(t) and w(t) respectively represent the input and output of PID, and are expressed as:

[0062]

[0063] The propulsion force of the left and right sides of the unmanned ship is obtained through differential drive calculation of w(t), and then the overall propulsion force τ of the unmanned ship is obtained:

[0064]

[0065] where: T p and T s respectively represent the propulsion forces of the left and right sides, and B represents the center distance of the side body of the unmanned ship;

[0066] The three-degree-of-freedom model of the unmanned ship is as follows:

[0067]

[0068] where: η = [x, y, ψ] T and v = [u, v, r] T respectively represent the pose vector and velocity vector of the unmanned ship; J(η) represents the rotation matrix used to transform variables in the geographical coordinate system to the ship-fixed coordinate system; M represents the inertia matrix, C(v) represents the Coriolis-centrifugal force matrix of the unmanned ship, D(v) represents the damping matrix, G(η) represents the hydrostatic matrix, τ wind and τ wave respectively represent the wind and wave interference moments;

[0069] Based on the propulsive force τ of the whole unmanned ship and the three-degree-of-freedom model of the unmanned ship, solve the kinematic equation of the unmanned ship to achieve motion control.

[0070] An unmanned ship autonomous decision-making system based on real-time lidar perception, which executes an unmanned ship autonomous navigation method based on real-time lidar perception described in any one of the above, includes:

[0071] An acquisition module for acquiring the state information of the unmanned ship, the end position information, and two-dimensional lidar data;

[0072] A calculation module for calculating the position information of the end relative to the unmanned ship through the state information of the unmanned ship and the end position information;

[0073] A first data processing module for normalizing the two-dimensional lidar data and extracting features from the normalized two-dimensional lidar data using a deep neural network to obtain radar feature information;

[0074] A second data processing module for normalizing the state information of the unmanned ship and the position information of the end relative to the unmanned ship and fusing them with the radar feature information;

[0075] A mapping module for inputting the fused information into a trained unmanned ship autonomous navigation model and outputting the desired yaw angle and desired speed of the unmanned ship;

[0076] A motion control module for inputting the desired yaw angle and desired speed of the unmanned ship into the underlying controller of the unmanned ship and calculating the propulsion speed of each thruster in combination with the kinematic model and dynamic model of the unmanned ship;

[0077] A judgment module for confirming whether the unmanned ship has reached the end. If it has reached the end, the navigation ends; otherwise, the navigation process continues.

[0078] As can be seen from the above technical solutions, compared with the prior art, the present invention discloses an autonomous navigation method and decision-making system for an unmanned ship based on real-time lidar perception, which gets rid of the dependence on global environmental information, enables the unmanned ship to achieve autonomous navigation in unknown or dynamic environments, and has stronger adaptability and robustness. Secondly, by introducing deep reinforcement learning, the unmanned ship can autonomously learn efficient navigation strategies and achieve real-time path optimization in complex environments without relying on traditional heuristic rules. BRIEF DESCRIPTION OF THE DRAWINGS

[0079] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only the embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained according to the provided drawings without creative efforts.

[0080] Figure 1 It is a flowchart of the autonomous navigation method for an unmanned ship based on real-time lidar perception provided by the present invention;

[0081] Figure 2 It is a schematic diagram of the conversion of relative end point information and radar scanning provided by the present invention;

[0082] Figure 3 It is a three-degree-of-freedom mathematical model of the unmanned ship provided by the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0083] The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts fall within the scope of protection of the present invention.

[0084] To improve the autonomous navigation ability of an unmanned ship in a complex marine environment, an embodiment of the present invention discloses an autonomous navigation method for an unmanned ship based on real-time lidar perception, as Figure 1 shown, including the following steps:

[0085] S1: Collect the state information of the unmanned ship, the end point position information, and the two-dimensional lidar data;

[0086] Specifically, the state information of the unmanned ship includes the coordinates (x u , y u ) of the unmanned ship, the yaw angle ψ u , and the speed (u u , v u , ru ); The end - point position information is the coordinates (x t , y t ) of the end - point; The two - dimensional lidar data is represented as being the measurement data of a lidar with an angular range of n s . Each scan provides n L sets of angle - distance pairs, where n L represents the number of ranging beams of the radar, and the maximum detection distance of the lidar is L max .

[0087] S2: Based on the unmanned - ship state information and the end - point position information, calculate the position information of the end - point relative to the unmanned ship;

[0088] Among them, the position information of the end - point relative to the unmanned ship includes the distance d t between the unmanned ship and the end - point and the azimuth angle ψ t of the end - point relative to the unmanned ship. Refer to Figure 2 , and the specific calculation formula is:

[0089]

[0090] The deviation between the yaw angle ψ u of the unmanned ship and the azimuth angle ψ t of the end - point relative to the unmanned ship, that is, the relative course angle δ ψ is:

[0091] δ ψ = ψ u - ψ t

[0092] In the formula: (x t , y t ) are the coordinates of the end - point, and (x u , y u ) are the coordinates of the unmanned ship.

[0093] S3: Normalize the two - dimensional lidar data, and use a deep neural network to extract features from the normalized two - dimensional lidar data to obtain radar feature information, realizing the dimensionality reduction of high - dimensional radar data, retaining key environmental information, and improving the calculation efficiency;

[0094] In this embodiment, the distance information is normalized by dividing by the maximum detection distance L max of the lidar, and the angle information is normalized by dividing by the angular range n s of the lidar to eliminate the influence of the measurement scale.

[0095] S4: Normalize the unmanned - ship state information and the position information of the end - point relative to the unmanned ship, and fuse them with the radar feature information;

[0096] In this embodiment, the speed (x u , v u , r u ) and the yaw angle ψ u of the unmanned ship are normalized by dividing them by their respective maximum values to ensure that the speed and angle values are normalized to a fixed range; the coordinates (x u , y u ) of the unmanned ship are normalized by dividing them by the width W and length L of the map to ensure the scale consistency of the position features; the distance d t between the unmanned ship and the end point is normalized by dividing it by the diagonal length of the map, and the relative heading angle δ ψ is normalized by dividing it by 2π. The normalization operation aims to process the data into the range of [-1, 1], ensuring that the input data has a consistent numerical scale, thereby improving the training stability and convergence effect of the neural network model.

[0097] S5: Input the fused information into the trained unmanned ship autonomous navigation model to output the desired yaw angle and desired speed of the unmanned ship;

[0098] In this embodiment, the unmanned ship autonomous navigation model is trained based on the Twin Delayed Deep Deterministic Policy Gradient (TD3) framework, which specifically includes the following steps:

[0099] S51. State space definition: The state space is the input of the unmanned ship autonomous navigation model, containing all the information relied on for decision-making, and is defined as follows:

[0100] s = <o u , o t , o a , o l 〉

[0101] where: o u represents the real-time state information of the unmanned ship, including the position information (x u , y u , ψ u ) and the speed information (u u , v u , r u ); o t represents the relative position information of the end point, that is, the distance d t between the unmanned ship and the end point and the relative heading angle δ ψ ; o a represents the execution action of the unmanned ship at the previous time step o l represents the lidar feature information after dimensionality reduction processing;

[0102] S52. Action Space Definition: The action space is the output of the unmanned ship autonomous navigation model, which is compatible with the characteristics of the underlying controller and is set as follows:

[0103] a = <u d , ψ d >

[0104] Where: u d represents the desired surge velocity, and ψ d represents the desired yaw angle;

[0105] Considering the constraint characteristics of the underlying controller, the following limiting conditions are imposed on the action space:

[0106]

[0107] Where: u max represents the maximum surge velocity of the unmanned ship, which is set to 1; ψ max represents the maximum yaw angle of the unmanned ship, which is set to π;

[0108] S53. Reward Mechanism Setting: The deep reinforcement learning model is used for the autonomous navigation of the unmanned ship. The reward function is a decision-driven mechanism, aiming to guide the unmanned ship to execute the expected behavior pattern.

[0109] Among them, the reward mechanism is set according to the state space and the action space to optimize the training effect and ensure safety, specifically including:

[0110] 1) Target Point Reward Mechanism: Set the target point reward to encourage the unmanned ship to navigate towards the target point; the target point reward satisfies the following conditions:

[0111]

[0112] Where: r safe represents the safety area radius of the unmanned ship, used to judge whether the unmanned ship reaches the target point or collides; when the distance d t between the unmanned ship and the end point is less than or equal to r safe , it is determined that the unmanned ship successfully reaches the target point, and a positive reward r arrival is given;

[0113] 2) Collision Penalty Mechanism: Set the collision penalty to prompt the unmanned ship to avoid obstacles and maintain safe navigation; the collision penalty satisfies the following conditions:

[0114]

[0115] Where: d o represents the distance between the unmanned ship and the obstacle; when the distance d o between the unmanned ship and the obstacle is less than or equal to the safety distance r safeWhen a collision is determined to occur, a negative reward r is applied. collision ;

[0116] 3) Timeout penalty mechanism: Set a timeout penalty to prevent the unmanned ship from executing inefficient strategies; the timeout penalty satisfies the following conditions:

[0117]

[0118] In the formula: t o represents the set time threshold; if the current time step t is greater than or equal to t o , it is determined that the timeout has occurred and a negative reward r is applied. timeout ;

[0119] 4) Out-of-bounds penalty mechanism: Set an out-of-bounds penalty to ensure that the unmanned ship sails within the specified area and prevent it from sailing aimlessly; the out-of-bounds penalty satisfies the following conditions:

[0120]

[0121] In the formula: |x e | and |y e | are the boundary ranges of the sailing area respectively; when the current position (x u , y u ) of the unmanned ship exceeds the boundary range, a negative reward r is applied. boundary ;

[0122] 5) Potential field reward mechanism: Set a potential field reward to guide the unmanned ship to actively avoid obstacles while approaching the end point, thereby improving the sailing safety and efficiency; the potential field reward satisfies the following conditions:

[0123]

[0124] R p = R att + R rep

[0125] In the formula: k att represents the gravitational potential field coefficient, k rep represents the repulsive potential field coefficient; the unmanned ship obtains a positive reward R provided by the gravitational field and a negative reward R provided by the repulsive field at each time step. att and a negative reward R provided by the repulsive field. rep ;

[0126] 6) Action smoothing reward mechanism: Set an action smoothing reward to suppress the action mutation of the unmanned ship between adjacent time steps and ensure the stability of the output action; the action smoothing reward satisfies the following conditions:

[0127]

[0128] In the formula: ud and ψ d They represent the action taken at the current time step, u′ d and ψ′ d represent the action taken in the previous time step, σ u and σ ψ Indicates adjustment parameters;

[0129] Combining the above reward and punishment mechanisms, the final reward function is defined as follows:

[0130] R=λ0R d +λ1R o +λ2R e +λ3R b +λ4R p +λ5R a

[0131] Wherein: λ0, λ1, λ2, λ3, λ4, λ5 represent the corresponding weight coefficients respectively.

[0132] S6: input the desired bow pitch angle and desired speed of the unmanned ship into the bottom controller of the unmanned ship, calculate the propulsion speed of each propeller in combination with the kinematic model and dynamic model of the unmanned ship, and realize precise control of the unmanned ship;

[0133] In this embodiment, the bottom controller of the unmanned ship adopts PID, and the expression is as follows:

[0134]

[0135] Where: K p , K i and K d They represent the proportional coefficient, integral coefficient and differential coefficient respectively; e(t) and w(t) represent the input and output of PID respectively, which are expressed as:

[0136]

[0137] The propulsion force of the unmanned ship on the left and right sides is calculated by differential drive, and then the overall propulsion force τ of the unmanned ship is obtained:

[0138]

[0139] Where: T p and T s They represent the propulsion force on the port side and the starboard side respectively, and B represents the side center distance of the unmanned ship;

[0140] Reference Figure 3 , the three-degree-of-freedom model of the unmanned ship is as follows:

[0141]

[0142] where: η = [x, y, ψ] T and v = [u, v, r] T respectively represent the pose vector and velocity vector of the unmanned ship; J(η) represents the rotation matrix used to transform the variables in the geographical coordinate system to the ship-fixed coordinate system; M represents the inertia matrix, C(v) represents the Coriolis-centrifugal force matrix of the unmanned ship, D(v) represents the damping matrix, G(η) represents the hydrostatic matrix, τ wind and τ wave respectively represent the wind and wave interference moments; based on the propulsion force τ of the overall unmanned ship and the three-degree-of-freedom model of the unmanned ship, the kinematic equation of the unmanned ship is solved to achieve motion control.

[0143] S7: Determine whether the unmanned ship has reached the end point. If it has reached the end point, the navigation ends; otherwise, return to S1 to continue the navigation process.

[0144] And Figure 1 corresponding to the method described above, an unmanned ship autonomous decision-making system based on real-time lidar perception provided by an embodiment of the present invention is used to Figure 1 For the specific implementation of the method in, an unmanned ship autonomous decision-making system based on real-time lidar perception provided by an embodiment of the present invention can be applied to a computer terminal or various mobile devices, and specifically includes:

[0145] An acquisition module for acquiring the unmanned ship state information, end point position information, and two-dimensional lidar data;

[0146] A calculation module for calculating the position information of the end point relative to the unmanned ship through the unmanned ship state information and the end point position information;

[0147] A first data processing module for normalizing the two-dimensional lidar data and extracting features from the normalized two-dimensional lidar data using a deep neural network to obtain radar feature information;

[0148] A second data processing module for normalizing the unmanned ship state information and the position information of the end point relative to the unmanned ship and fusing them with the radar feature information;

[0149] A mapping module for inputting the fused information into the trained unmanned ship autonomous navigation model and outputting the desired yaw angle and desired speed of the unmanned ship;

[0150] A motion control module for inputting the desired yaw angle and desired speed of the unmanned ship into the unmanned ship low-level controller, and calculating the propulsion speeds of each thruster in combination with the kinematic model and dynamic model of the unmanned ship;

[0151] A judgment module is used to confirm whether the unmanned ship has reached the end point. If it has reached the end point, the navigation ends; otherwise, the navigation process continues.

[0152] Compared with traditional methods that rely on global information such as the ant colony algorithm and the artificial potential field method, the autonomous navigation method based on deep reinforcement learning is adopted in this embodiment, enabling the unmanned ship to achieve autonomous navigation without knowing the map information. By perceiving the environment in real time and combining it with the deep reinforcement learning model for decision optimization, the unmanned ship can dynamically adjust its trajectory in a complex and uncertain marine environment, achieving flexible and intelligent obstacle avoidance and target navigation. In addition, to further improve the environmental perception ability, a two-dimensional lidar is used in this embodiment to obtain high-precision point cloud data, enabling the unmanned ship to make autonomous decisions relying on local perception information without the support of global information, thereby improving its environmental adaptability and the reliability of task execution.

[0153] The various embodiments in this specification are described in a progressive manner. Each embodiment focuses on the differences from other embodiments. For the same or similar parts among the various embodiments, reference can be made to each other. For the system disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple, and reference can be made to the description in the method part for the relevant parts.

[0154] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present invention. Various modifications to these embodiments will be obvious to those skilled in the art. The general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to these embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. An autonomous navigation method for an unmanned ship based on real-time lidar perception, characterized in that, It includes the following steps: S1: Collect the state information of the unmanned ship, the end position information, and the two-dimensional lidar data; S2: Based on the state information of the unmanned ship and the end position information, calculate the position information of the end relative to the unmanned ship; S3: Normalize the two-dimensional lidar data, and use a deep neural network to extract features from the normalized two-dimensional lidar data to obtain radar feature information; S4: Normalize the state information of the unmanned ship and the position information of the end relative to the unmanned ship, and fuse them with the radar feature information; S5: Input the fused information into the trained unmanned ship autonomous navigation model, and output the desired yaw angle and desired speed of the unmanned ship; S6: Input the desired yaw angle and desired speed of the unmanned ship into the low-level controller of the unmanned ship, and calculate the propulsion speed of each thruster in combination with the kinematic model and dynamic model of the unmanned ship; S7: Determine whether the unmanned ship has reached the end point. If it has reached the end point, the navigation ends; otherwise, return to S1 to continue the navigation process.

2. The autonomous navigation method of an unmanned ship based on real-time perception of lidar according to claim 1, characterized in that, In S1, the unmanned ship status information includes the coordinates (x u , y u ) of the unmanned ship, the yaw angle ψ u and the speed (u u , v u , r u ) of the unmanned ship; the end position information is the coordinates (x t , y t ) of the end point; the two-dimensional lidar data is represented as is the measurement data of the lidar with an angular range of n s , and each scan provides n L sets of angle-distance pairs, where n L represents the number of ranging beams of the radar, and the maximum detection distance of the lidar is L max .

3. A method for autonomous navigation of an unmanned ship based on real-time perception of lidar according to claim 2, characterized in that In S2, the position information of the end point relative to the unmanned ship includes the distance d between the unmanned ship and the end point t and the azimuth angle ψ of the end point relative to the unmanned ship t , and the specific calculation formula is as follows: Heading angle ψ of the unmanned boat u Azimuth angle ψ of the unmanned boat relative to the end point t The deviation between them is the relative course angle δ ψ That is: δ ψ = ψ u - ψ t ; Where: (x t , y t ) are the coordinates of the end point, (x u , y u ) are the coordinates of the unmanned boat.

4. A method for autonomous navigation of an unmanned ship based on real-time perception of lidar according to claim 1, characterized in that, In S3, the two-dimensional lidar data is normalized as follows: the distance information is normalized by dividing it by the maximum detection distance L of the lidar. max The angle information is normalized by dividing it by the angular range n of the lidar. s And then it is normalized.

5. The autonomous navigation method of an unmanned ship based on real-time perception of lidar according to claim 3, characterized in that In S4, the state information of the unmanned ship and the position information of the end point relative to the unmanned ship are normalized. Specifically: the speed (u u , v u , r u ) and the yaw angle ψ u of the unmanned ship are normalized by dividing them by their respective maximum values. The coordinates (x u , y u ) of the unmanned ship are normalized by dividing them by the width W and length L of the map. The distance d t between the unmanned ship and the end point is normalized by dividing it by the diagonal length of the map. The relative course angle δ ψ is normalized by dividing it by 2π.

6. The autonomous navigation method of an unmanned boat based on real-time lidar perception according to claim 3, characterized in that, In S5, the unmanned ship autonomous navigation model is trained based on the double-delayed deep deterministic policy gradient framework, specifically including the following steps: S51. State space definition: The state space is the input of the unmanned ship autonomous navigation model, which contains all the information relied on for decision-making, and is defined as follows: s = <o u , o t , o a , o l > where: o u represents the real-time status information of the unmanned ship, including the position information (x u , y u , ψ u ), and the speed information (u u , v u , r u ); o t represents the relative position information of the end point, that is, the distance d between the unmanned ship and the end point t and the relative course angle δ ψ ; o a represents the execution action of the unmanned ship in the previous time step o l represents the lidar feature information after dimensionality reduction processing; S52. Action space definition: The action space is the output of the unmanned ship autonomous navigation model, which is compatible with the characteristics of the low-level controller, and is set as follows: a = <u d , ψ d > where: u d represents the desired surge velocity, and ψ d represents the desired yaw angle; Considering the constraint characteristics of the low-level controller, the following limiting conditions are imposed on the action space: where: u max represents the maximum surge speed of the unmanned ship, and ψ max represents the maximum yaw angle of the unmanned ship; S53. Set the reward mechanism: The reward function is a decision-driven mechanism that guides the unmanned ship to execute a behavior pattern that meets expectations.

7. A method for autonomous navigation of an unmanned ship based on real-time perception of lidar according to claim 6, characterized in that In S53, the reward mechanism is set according to the state space and the action space, specifically including: 1) Target point reward mechanism: Set the target point reward to encourage the unmanned ship to navigate towards the target point; The target point reward meets the following conditions: Where: r safe represents the safety zone radius of the unmanned ship, which is used to determine whether the unmanned ship reaches the target point or collides; when the distance d t from the unmanned ship to the end point is less than or equal to r safe , it is determined that the unmanned ship has successfully reached the target point, and a positive reward r arrival is given; 2) Collision penalty mechanism: Set the collision penalty to prompt the unmanned ship to avoid obstacles and maintain safe navigation; The collision penalty meets the following conditions: where: d o represents the distance between the unmanned ship and the obstacle; when the distance d o between the unmanned ship and the obstacle is less than or equal to the safety distance r safe , it is determined that a collision has occurred and a negative reward r collision is applied; 3) Timeout penalty mechanism: Set the timeout penalty to prevent the unmanned ship from executing inefficient strategies; The timeout penalty meets the following conditions: where: t o represents the set time threshold; if the current time step t is greater than or equal to t o , it is determined that the time is out and a negative reward r timeout is applied; 4) Out-of-bounds penalty mechanism: Set the out-of-bounds penalty to ensure that the unmanned ship sails within the specified area; The out-of-bounds penalty meets the following conditions: where: |x e | and |y e | are the boundary ranges of the navigation area respectively; when the current position (x u , y u ) of the unmanned ship exceeds the boundary range, a negative reward r boundary is imposed; 5) Potential field reward mechanism: Set the potential field reward to guide the unmanned ship to actively avoid obstacles while approaching the end point; The potential field reward meets the following conditions: R p = R att + R rep Where: k att represents the gravitational potential field coefficient, and k rep represents the repulsive potential field coefficient; the unmanned ship obtains the positive reward R provided by the gravitational field at each time step att and the negative reward R provided by the repulsive field rep ; 6) Action smoothing reward mechanism: Set the action smoothing reward to suppress the sudden change of the unmanned ship's actions between adjacent time steps; The action smoothing reward meets the following conditions: where: u d and ψ d represent the actions taken at the current time step respectively, u′ d and ψ′ d represent the actions taken at the previous time step respectively, σ u and σ ψ represent the adjustment parameters; Combining the above reward and punishment mechanisms, the final reward function is defined as follows: R = λ0R d + λ1R o + λ2R e + λ3R b + λ4R p + λ5R a In the formula: λ0, λ1, λ2, λ3, λ4, λ5 respectively represent the corresponding weight coefficients.

8. A method for autonomous navigation of an unmanned ship based on real-time perception of lidar according to claim 1, characterized in that, In S6, the low-level controller of the unmanned ship adopts PID, and the expression is as follows: Where: K p , K i and K d represent the proportional coefficient, integral coefficient, and differential coefficient, respectively; e(t) and w(t) represent the input and output of the PID, respectively, and are expressed as: w(t) obtains the propulsion forces on the left and right sides of the unmanned ship through differential drive calculation, and then obtains the overall propulsion force τ of the unmanned ship: Where: T p and T s represent the propulsive forces on the port side and starboard side respectively, and B represents the distance between the centerlines of the side hulls of the unmanned ship; The three-degree-of-freedom model of the unmanned ship is as follows: where: η = [x, y, ψ] T and v = [u, v, r] T represent the pose vector and velocity vector of the unmanned ship respectively; J(η) represents the rotation matrix used to transform the variables in the geographical coordinate system to the ship-fixed coordinate system; M represents the inertia matrix, C(v) represents the Coriolis-centrifugal force matrix of the unmanned ship, D(v) represents the damping matrix, G(η) represents the hydrostatic matrix, τ wind and τ wave represent the wind and wave interference moments respectively; Based on the propulsion force τ of the overall unmanned ship and the three-degree-of-freedom model of the unmanned ship, the kinematic equation of the unmanned ship is solved to achieve motion control.

9. An autonomous decision-making system for an unmanned ship based on real-time lidar perception, characterized in that, Implementing an unmanned ship autonomous navigation method based on real-time lidar perception as described in any one of claims 1-8, including: The acquisition module is used to acquire the unmanned ship status information, the end point position information, and the 2D lidar data; The calculation module is used to calculate the position information of the end point relative to the unmanned ship based on the unmanned ship status information and the end point position information; The first data processing module is used to normalize the 2D lidar data, and extract features from the normalized 2D lidar data using a deep neural network to obtain radar feature information; The second data processing module is used to normalize the unmanned ship status information and the position information of the end point relative to the unmanned ship, and fuse them with the radar feature information; The mapping module is used to input the fused information into the trained unmanned ship autonomous navigation model, and output the desired yaw angle and desired speed of the unmanned ship; The motion control module is used to input the desired yaw angle and desired speed of the unmanned ship into the low-level controller of the unmanned ship, and calculate the propulsion speed of each thruster in combination with the kinematic model and dynamic model of the unmanned ship; The judgment module is used to confirm whether the unmanned ship has reached the end point. If it has reached the end point, the navigation ends; otherwise, the navigation process continues.

Images