Unmanned ship route planning method improved by gradient reward and punishment measures based on RRT algorithm
By introducing gradient reward and punishment measures in the RRT algorithm, the problems of low search efficiency and insufficient path feasibility in unmanned ship path planning are solved, and more efficient and safer path planning is achieved, thus reducing the risk of obstacle collision in complex environments .
Patent Information
- Application Number
- CN202411402221.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-09
- Publication Date
- 2025-01-10
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
The existing RRT algorithms have problems such as low search efficiency and insufficient path feasibility in unmanned ship path planning, especially in complex environments that may cause unmanned ships to collide with obstacles.
Gradient reward and punishment measures are introduced, and by setting gradient punishment away rules and gradient reward attraction rules, the binary reward function in the RRT algorithm is improved, and the unmanned ships are guided to explore state space more effectively in complex environments and generate feasible and close to the optimal path.
It improves the efficiency and success rate of unmanned ship path planning, reduces the path length and obstacle collision probability, provides more optimized and adaptable path planning, and improves the response speed of the navigation system.
Smart Images
Figure CN119289993A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field related to unmanned ship route planning, and in particular to an unmanned ship route planning method improved by using gradient reward and punishment measures based on an RRT algorithm. Background Art
[0002] An unmanned ship is a ship system that does not require human piloting. It uses autonomous navigation and control technology to perform tasks in different waters. The path planning of an unmanned ship is one of the core technologies of its autonomous navigation, which aims to provide an optimized navigation path for the unmanned ship to reach the target quickly and safely.
[0003] Traditional unmanned ship path planning algorithms include A* algorithm, Dijkstra algorithm, etc., which generate the shortest path by searching the state nodes in the space. However, in complex environments, these traditional algorithms have some limitations, such as low search efficiency and local optimal solutions, which result in the generated paths failing to meet actual needs. In order to solve these problems, researchers have proposed a path planning method based on the RRT algorithm in recent years. The RRT algorithm is a sampling optimization technique that searches the state space through random sampling and tree expansion. Compared with traditional algorithms, the RRT algorithm can better explore the state space in complex environments and generate feasible paths that are close to the optimal solution.
[0004] However, the existing RRT algorithm still has some limitations, such as low search efficiency and insufficient path feasibility. In the perception system of unmanned ships, although unmanned ships can detect and identify obstacles, if the system lacks sufficient avoidance behavior, under the influence of dynamic environments such as wind and waves, the unmanned ship may collide with obstacles. Summary of the invention
[0005] The purpose of the present invention is to solve at least one of the deficiencies of the prior art and to provide a gradient reward and punishment measure based on the RRT algorithm to improve the route planning method of an unmanned ship.
[0006] In order to achieve the above object, the present invention adopts the following technical solutions: Specifically, a gradient reward and punishment measure based on the RRT algorithm is proposed to improve the route planning method of the unmanned ship, including the following: Step 110: Initialize the target map, determine the initial state of the unmanned ship, and set the root node and step size of the RRT algorithm; Step 120, obtaining the starting point position and the end point position; Step 130: constructing an RRT path planning curve connecting the starting position and the end position based on the RRT algorithm and a gradient reward and punishment mechanism; Step 140: Use the RRT path planning curve as the unmanned boat route planning result to control the unmanned boat to travel accordingly.
[0007] Further, specifically, the initialization settings include: Step 111, establish a virtual environment digital platform, and initialize the size of the target map, the size and coordinate position of static obstacles, the coordinate position of the channel and both sides, add the starting position, end position, gradient reward area and gradient penalty area of the unmanned ship in the target map, and initialize the motion state of the unmanned ship; Step 112: Take the starting position of the unmanned ship as the root node of the RRT algorithm, and put the root node into the node set of the growth tree; Step 113: Set the initial growth step size.
[0008] Further, specifically, in constructing the RRT path planning curve, constructing the RRT algorithm tree includes: Step 131, randomly sample a state point from the target map, and record the sampled state point as Xrand; Step 132: pre-construct a tree with only one node, the starting point, and find the node Xnear closest to the sampling point Xrand from the constructed tree; Step 133, start the tree growth process, first connect Xnear and Xrand to get a connecting line, the direction of this connecting line is the direction of tree growth, set a step size Stepsize as the step size of the tree growth once, grow a step size in this direction of tree growth, and then a new node Xnew will be generated at the end of the growth; Step 134, determine whether there is an obstacle from Xnear to Xrand, if so, abandon the Xnew node, if not, add the Xnew node to the tree; Step 135, looping through steps 111 to 134 until the distance from the newly generated node Xnew to the end point is less than a step size Stepsize, then terminating the growth of the tree; Step 136: directly connect the newly generated node Xnew to the end point to obtain an RRT path planning curve connecting the start point and the end point.
[0009] Further, specifically, in constructing the RRT path planning curve, the gradient reward and punishment mechanism is: The traditional binary reward function in the original RRT algorithm is replaced by setting the gradient penalty away rule and the gradient reward attraction rule.
[0010] Further, specifically, the gradient penalty of the gradient reward and punishment mechanism is away from the rule, including, When the unmanned ship enters the gradient penalty area, the ordinate of the current position of the unmanned ship is calculated To the target obstacle ordinate The difference H; Calculate the horizontal coordinate of the current position of the unmanned ship The horizontal coordinate of the target obstacle The difference ; According to the following formula, select the value of η: ; The unmanned ship is 1 gradient penalty, where 1According to the formula Calculated.
[0011] Further, specifically, the gradient reward attraction rules of the gradient reward and punishment mechanism include: When the unmanned ship enters the gradient reward area, calculate the ordinate of the current position of the unmanned ship The ordinate of the point on the target RRT path curve that is closest to the current position The difference H; Calculate the minimum value of the Euclidean distance from the current position of the unmanned ship to the target RRT path curve , the specific formula is as follows:
[0012] in, is the horizontal coordinate of the current position of the unmanned ship, is the horizontal coordinate of the target RRT path curve, According to the following formula, select the value of η:
[0013] Then the unmanned ship gets The gradient reward is According to the formula Calculated.
[0014] The present invention also proposes a gradient reward and punishment measure based on the RRT algorithm to improve the unmanned ship route planning device, including: The initialization module is used to initialize the target map, determine the initial state of the unmanned ship, and set the root node and step size of the RRT algorithm; A location acquisition module is used to obtain the starting point location and the end point location; A path planning module, used to construct an RRT path planning curve connecting the starting position and the end position based on the RRT algorithm with a gradient reward and punishment mechanism; The unmanned boat control module is used to control the unmanned boat to travel accordingly by using the RRT path planning curve as the unmanned boat route planning result.
[0015] The beneficial effects of the present invention are: The present invention proposes a method for improving the route planning of an unmanned ship based on a gradient reward and punishment measure of an RRT algorithm. By introducing a technical solution for improving the gradient reward and punishment measure, the efficiency of the path planning of an unmanned ship based on the RRT algorithm can be improved. By utilizing environmental gradient information, the search process is guided to extend toward areas with larger gradients, thereby quickly finding a feasible optimized path. Compared with traditional path planning algorithms, this improved solution can generate efficient paths more quickly. When unmanned ships perform tasks in different waters and regions, they often face complex environmental conditions such as complex terrain and water flow. By introducing gradient reward and punishment measures, this technical solution can better adapt to the navigation needs of different environments and provide more optimized and adaptable path planning. It can more accurately search for the optimal path, reduce the complexity of path planning and improve the response speed of the navigation system. BRIEF DESCRIPTION OF THE DRAWINGS
[0016] The above and other features of the present disclosure will become more obvious by describing in detail the embodiments shown in the accompanying drawings. The same reference numerals in the accompanying drawings of the present disclosure represent the same or similar sampling monitoring points. Obviously, the accompanying drawings described below are only some embodiments of the present disclosure. For those skilled in the art, other accompanying drawings can be obtained based on these accompanying drawings without creative work. In the accompanying drawings: Figure 1 It is a flow chart of the method for improving the route planning of unmanned ships based on the gradient reward and punishment measures of the RRT algorithm of the present invention; Figure 2 Shown is a schematic diagram of the RRT algorithm planning algorithm used in the present invention; Figure 3 The figure shows the principle diagram of the gradient penalty stay-away rule for the unmanned ship around obstacles in the present invention; Figure 4 Shown is a schematic diagram of the gradient reward attraction rule for the unmanned ship on the left and right of the target RRT path in the present invention. DETAILED DESCRIPTION
[0017] The following will be combined with the embodiments and drawings to clearly and completely describe the concept, specific structure and technical effects of the present invention, so as to fully understand the purpose, scheme and effect of the present invention. It should be noted that the embodiments in this application and the features in the embodiments can be combined with each other without conflict. The same reference numerals used throughout the drawings indicate the same or similar parts.
[0018] Example 1, reference Figure 1The present invention proposes a method for improving the route planning of an unmanned ship based on a gradient reward and punishment measure of an RRT algorithm, including the following: Step 110: Initialize the target map, determine the initial state of the unmanned ship, and set the root node and step size of the RRT algorithm; Step 120, obtaining the starting point position and the end point position; Step 130: constructing an RRT path planning curve connecting the starting position and the end position based on the RRT algorithm and a gradient reward and punishment mechanism; Step 140: Use the RRT path planning curve as the unmanned boat route planning result to control the unmanned boat to travel accordingly.
[0019] In this embodiment 1, by introducing a technical solution improved by gradient reward and punishment measures, the efficiency of unmanned ship path planning based on the RRT algorithm can be improved. By utilizing environmental gradient information, the search process is guided to extend toward areas with larger gradients, so as to quickly find a feasible optimized path. Compared with traditional path planning algorithms, this improved solution can generate efficient paths more quickly. When unmanned ships perform tasks in different waters and regions, they often face complex terrain, water flow and other environmental conditions. By introducing gradient reward and punishment measures, this technical solution can better adapt to the navigation needs of different environments and provide more optimized and adaptable path planning. It can search for the best path more accurately, reduce the complexity of path planning and improve the response speed of the navigation system.
[0020] In this preferred embodiment, specifically, the initialization settings include: Step 111, establish a virtual environment digital platform, and initialize the size of the target map, the size and coordinate position of static obstacles, the coordinate position of the channel and both sides, add the starting position, end position, gradient reward area and gradient penalty area of the unmanned ship in the target map, and initialize the motion state of the unmanned ship; Step 112: Take the starting position of the unmanned ship as the root node of the RRT algorithm, and put the root node into the node set of the growth tree; Step 113: Set the initial growth step size.
[0021] Reference Figure 2 In this preferred embodiment, specifically, in constructing the RRT path planning curve, constructing the tree of the RRT algorithm includes: Step 131, randomly sample a state point from the target map, and record the sampled state point as Xrand; Step 132: pre-construct a tree with only one node, the starting point, and find the node Xnear closest to the sampling point Xrand from the constructed tree; Step 133, start the tree growth process, first connect Xnear and Xrand to get a connecting line, the direction of this connecting line is the direction of tree growth, set a step size Stepsize as the step size of the tree growth once, grow a step size in this direction of tree growth, and then a new node Xnew will be generated at the end of the growth; Step 134, determine whether there is an obstacle from Xnear to Xrand, if so, abandon the Xnew node, if not, add the Xnew node to the tree; Step 135, looping through steps 111 to 134 until the distance from the newly generated node Xnew to the end point is less than a step size Stepsize, then terminating the growth of the tree; Step 136: directly connect the newly generated node Xnew to the end point to obtain an RRT path planning curve connecting the start point and the end point.
[0022] In this preferred embodiment, sampling points are obtained, directions are determined according to the sampling points and the nearest nodes, and the tree is grown according to the step size to determine whether the newly grown nodes collide with obstacles, thereby constructing the RRT tree.
[0023] In this preferred embodiment, specifically, in constructing the RRT path planning curve, the gradient reward and punishment mechanism is: The traditional binary reward function in the original RRT algorithm is replaced by setting the gradient penalty away rule and the gradient reward attraction rule.
[0024] In this preferred embodiment, considering that in path planning, the traditional binary reward function is set as follows: the reward for reaching the target point is 1, the reward for encountering an obstacle or exceeding the boundary is -1, and the reward is 0 in other cases. Although this method can detect and identify obstacles, if the system lacks sufficient avoidance behavior, under the influence of dynamic environments such as wind and waves, the unmanned ship may collide with obstacles. Therefore, the reward and punishment mechanism is improved in the above manner, so that the improved RRT algorithm can better explore the state space in a complex environment and generate a feasible path that is close to the optimal solution.
[0025] The experimental verification is as follows: When conducting a simulated reward and punishment process in complex waters, the path length of the traditional method was 300 meters, and after improvement it was 220 meters; the success rate of the traditional method was 70% (collisions occurred in 30% of the attempts), and the success rate after improvement was 90% (collisions occurred in 10% of the attempts); the reward value of the traditional method remained almost unchanged when approaching an obstacle, but after improvement, the reward value dropped rapidly when approaching an obstacle, prompting the algorithm to adjust the path, and the reward value increased significantly when approaching the target.
[0026] It can be seen from the actual simulation data that the improved RRT algorithm based on the gradient reward and punishment mechanism shows obvious advantages in path planning. Through the dynamic adjustment of the reward value, it not only reduces the path length, but also improves the success rate (reduces the probability of collision with obstacles), and more effectively guides the unmanned ship to avoid obstacles and quickly reach the target point. This proves that the mechanism proposed by the present invention is significantly better than the existing mechanism and helps to improve the navigation ability of unmanned ships in complex sea environments.
[0027] Reference Figure 3 In this preferred embodiment, specifically, the gradient penalty away from the rule of the gradient reward and punishment mechanism includes: When the unmanned ship enters the gradient penalty area, the ordinate of the current position of the unmanned ship is calculated To the target obstacle ordinate The difference H; Calculate the horizontal coordinate of the current position of the unmanned ship The horizontal coordinate of the target obstacle The difference ; According to the following formula, select the value of η: ; The unmanned ship is 1 gradient penalty, where 1According to the formula Calculated.
[0028] Reference Figure 4 In this preferred embodiment, specifically, the gradient reward attraction rules of the gradient reward and punishment mechanism include: When the unmanned ship enters the gradient reward area, calculate the ordinate of the current position of the unmanned ship The ordinate of the point on the target RRT path curve that is closest to the current position The difference H; Calculate the minimum value of the Euclidean distance from the current position of the unmanned ship to the target RRT path curve , the specific formula is as follows:
[0029] in, is the horizontal coordinate of the current position of the unmanned ship, is the horizontal coordinate of the target RRT path curve, According to the following formula, select the value of η:
[0030] Then the unmanned ship gets The gradient reward is According to the formula Calculated.
[0031] Embodiment 2, the present invention also proposes a gradient reward and punishment measure based on the RRT algorithm to improve the unmanned ship route planning device, including: The initialization module is used to initialize the target map, determine the initial state of the unmanned ship, and set the root node and step size of the RRT algorithm; A location acquisition module is used to obtain the starting point location and the end point location; A path planning module, used to construct an RRT path planning curve connecting the starting position and the end position based on the RRT algorithm with a gradient reward and punishment mechanism; The unmanned boat control module is used to control the unmanned boat to travel accordingly by using the RRT path planning curve as the unmanned boat route planning result.
[0032] In addition, each functional module in each embodiment of the present invention may be integrated into one processing module, or each module may exist physically separately, or two or more modules may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of software functional modules.
[0033] If the integrated module is implemented in the form of a software function module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present invention implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or system that can carry the computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electrical carrier signal, telecommunication signal and software distribution medium, etc.
[0034] Although the description of the present invention has been quite detailed and specifically describes several described embodiments, it is not intended to be limited to any of these details or embodiments or any particular embodiment, but should be regarded as providing a broad possible interpretation of these claims in view of the prior art by reference to the appended claims, thereby effectively covering the intended scope of the present invention. In addition, the above description of the present invention is based on the embodiments foreseeable by the inventor, and its purpose is to provide a useful description, and those non-substantial changes to the present invention that have not yet been foreseen may still represent equivalent changes to the present invention.
[0035] The above is only a preferred embodiment of the present invention. The present invention is not limited to the above implementation. As long as the technical effect of the present invention is achieved by the same means, it should belong to the protection scope of the present invention. Within the protection scope of the present invention, its technical scheme and / or implementation method can have various modifications and changes.
Claims
1. The gradient reward and punishment measures based on the RRT algorithm are used to improve the route planning method of unmanned ships, which is characterized by: Includes the following: Step 110: Initialize the target map, determine the initial state of the unmanned ship, and set the root node and step size of the RRT algorithm; Step 120, obtaining the starting point position and the end point position; Step 130: constructing an RRT path planning curve connecting the starting position and the end position based on the RRT algorithm and a gradient reward and punishment mechanism; Step 140: Use the RRT path planning curve as the unmanned boat route planning result to control the unmanned boat to travel accordingly.
2. The method for improving route planning of unmanned ship based on gradient reward and punishment measures of RRT algorithm according to claim 1 is characterized in that: Specifically, the initialization settings include: Step 111, establish a virtual environment digital platform, and initialize the size of the target map, the size and coordinate position of static obstacles, the coordinate position of the channel and both sides, add the starting position, end position, gradient reward area and gradient penalty area of the unmanned ship in the target map, and initialize the motion state of the unmanned ship; Step 112: Take the starting position of the unmanned ship as the root node of the RRT algorithm, and put the root node into the node set of the growth tree; Step 113: Set the initial growth step size.
3. The method for improving unmanned ship route planning based on gradient reward and punishment measures of RRT algorithm according to claim 2 is characterized in that: Specifically, in constructing the RRT path planning curve, the tree of the RRT algorithm is constructed, including: Step 131, randomly sample a state point from the target map, and record the sampled state point as Xrand; Step 132: pre-construct a tree with only one node, the starting point, and find the node Xnear closest to the sampling point Xrand from the constructed tree; Step 133, start the tree growth process, first connect Xnear and Xrand to get a connecting line, the direction of this connecting line is the direction of tree growth, set a step size Stepsize as the step size of the tree growth once, grow a step size in this direction of tree growth, and then a new node Xnew will be generated at the end of the growth; Step 134, determine whether there is an obstacle from Xnear to Xrand, if so, abandon the Xnew node, if not, add the Xnew node to the tree; Step 135, looping through steps 111 to 134 until the distance from the newly generated node Xnew to the end point is less than a step size Stepsize, then terminating the growth of the tree; Step 136: directly connect the newly generated node Xnew to the end point to obtain an RRT path planning curve connecting the start point and the end point.
4. The method for improving unmanned ship route planning based on gradient reward and punishment measures of RRT algorithm according to claim 2 is characterized in that: Specifically, in constructing the RRT path planning curve, the gradient reward and punishment mechanism is: The traditional binary reward function in the original RRT algorithm is replaced by setting the gradient penalty away rule and the gradient reward attraction rule.
5. The method for improving unmanned ship route planning based on gradient reward and punishment measures of RRT algorithm according to claim 4 is characterized in that: Specifically, the gradient penalty of the gradient reward and punishment mechanism is far away from the rules, including: When the unmanned ship enters the gradient penalty area, the ordinate of the current position of the unmanned ship is calculated To the target obstacle ordinate The difference H; Calculate the horizontal coordinate of the current position of the unmanned ship The horizontal coordinate of the target obstacle The difference ; According to the following formula, select the value of η: ； The unmanned ship is 1 gradient penalty, where 1According to the formula Calculated.
6. The method for improving the route planning of unmanned ships by using gradient reward and punishment measures based on the RRT algorithm according to claim 4 is characterized in that: Specifically, the gradient reward attraction rules of the gradient reward and punishment mechanism include: When the unmanned ship enters the gradient reward area, calculate the ordinate of the current position of the unmanned ship The ordinate of the point on the target RRT path curve that is closest to the current position The difference H; Calculate the minimum value of the Euclidean distance from the current position of the unmanned ship to the target RRT path curve , the specific formula is as follows: in, is the horizontal coordinate of the current position of the unmanned ship, is the horizontal coordinate of the target RRT path curve, According to the following formula, select the value of η: Then the unmanned ship gets The gradient reward is According to the formula Calculated.
7. The unmanned ship route planning device is improved based on the gradient reward and punishment measures of the RRT algorithm, which is characterized by: include: The initialization module is used to initialize the target map, determine the initial state of the unmanned ship, and set the root node and step size of the RRT algorithm; A location acquisition module is used to obtain the starting point location and the end point location; A path planning module, used to construct an RRT path planning curve connecting the starting position and the end position based on the RRT algorithm with a gradient reward and punishment mechanism; The unmanned boat control module is used to control the unmanned boat to travel accordingly by using the RRT path planning curve as the unmanned boat route planning result.