Path planning method, terminal and storage medium for a multi-robot system

Through hierarchical path planning combined with global and local path optimization, RVO algorithm and deep neural network are used to solve the path planning problems in multi-robot systems, and the robot avoids collisions and deadlocks in a dynamic environment and optimizes the path to reach the target.

CN118092413BActive Publication Date: 2025-07-22HEFEI UNIV OF TECH
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202311468674.X
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2023-11-06
Publication Date
2025-07-22
Estimated Expiration
2043-11-06

AI Technical Summary

Technical Problem

The prior art cannot effectively coordinate the path planning between robots in multi-robot systems, resulting in collision and deadlock problems, especially when constant-speed robots are unable to cross obstacles when parallel.

Method used

The hierarchical path planning method is adopted, combined with global path planning and local path planning based on reinforcement learning, and through RVO dynamic obstacle avoidance algorithm and deep neural network, conflict points are determined and robot paths are optimized, and the optimal path is optimized using long and short-term memory networks and reward functions.

Benefits of technology

It effectively avoids obstacle collisions and deadlocks in multi-robot systems, optimizes the robot path to reach the target in the shortest time, and solves the problems of crossing obstacles and anti-collision interactions in parallel.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118092413B_ABST
    Figure CN118092413B_ABST
Patent Text Reader

Abstract

The present invention relates to the technical field of mobile robot navigation, and discloses a path planning method, a terminal and a storage medium for a multi-robot system. The method first obtains a grid map and basic information of all robots in the multi-robot system, determines a starting point and a target point in the grid map, and represents the grid map in the form of a weighted directed graph. Then, it plans a global path for each robot and finds the intersection points of all the global paths of the robots. Next, it determines the intersection points that belong to the conflict points, and based on the basic information of the robots corresponding to the conflict points and the expected collision time, plans a local path for the robots based on the RVO dynamic obstacle avoidance algorithm. When a hierarchical path planning of the multi-robot system is completed, time information is extracted from the hierarchical path, and based on the time information extracted from the hierarchical path, an optimal path planning scheme for the multi-robot system is found based on the reinforcement learning planning method. The present invention solves the problems that it is difficult for robots to cross obstacles and the problem of anti-collision interaction.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of mobile robot navigation, and specifically to a path planning method, a terminal, and a storage medium for a multi-robot system. Background Art

[0002] Autonomous navigation of robots has broad application prospects and is a promising research field. Robot navigation is a design process of moving towards a target position while avoiding obstacles. This process has four basic components: (1) perception, where the robot uses its sensors to extract meaningful information; (2) localization, where the robot determines its position in the workspace; (3) cognition and path planning, where the robot decides how to turn to achieve its goal; (4) motion control, where the robot adjusts its motion to achieve the desired trajectory.

[0003] With the advancement of industrial intelligence and the continuous development of computer technology, reinforcement learning has become one of the important methods in the field of autonomous navigation of mobile robots. Currently, many autonomous navigation methods based on reinforcement learning have been proposed, but there are still many practical problems: First, the representation of interactive environmental states. How to develop a suitable form of environmental state representation to clearly describe the anti-collision interaction between these robots remains an open question. Second, the effective mapping from continuous environmental states to continuous control actions. Given the current sequential environmental state, how to use neural networks to achieve optimal and accurate anti-collision control actions with low computational cost is still challenging. Third, the reward design for anti-collision behavior supervision. There is currently no systematic method to design rewards based on observations to accurately represent the collision risk and guide the robots to achieve mutual anti-collision behavior.

[0004] Currently, relevant researchers have proposed a path planning method based on globally-guided reinforcement learning, which utilizes environmental spatio-temporal information. Different from existing learning-based methods, it proposes a hierarchical framework that combines global guidance and local reinforcement learning-based planning to achieve end-to-end learning in a dynamic environment. Introducing global guidance allows the robot to learn to navigate to the destination through a fixed-size learning model, greatly saving the time for training the robot. The local RL planner uses the spatial and temporal information within the local area (e.g., the field of view) to avoid potential collisions and unnecessary detours, and then designs a new reward structure to encourage the robot to explore all potential solutions and can be generalized to any environment.

[0005] However, the above method is only for a single robot, and there is no coordination between multiple robots, so it cannot be extended to a multi-robot system. In a multi-robot system, an algorithm based on RVO or VO area and deep reinforcement learning is proposed to solve the problem of mutual collision between multiple robots. This method first models the dynamic subject and static obstacles based on VO and RVO vectors, and then extracts features and calculates actions in continuous space through deep learning neural networks. After that, the reward function of reinforcement learning is designed based on VO and RVO areas to optimize the action strategy of the mobile robot, thereby realizing interactive collision between multiple robots. However, this method has the problem of being unable to cross due to the parallel operation of multiple robots with equal speeds, as well as the large-scale deadlock of robots caused by mutual collision avoidance, which needs to be solved urgently in the field of autonomous navigation of multi-robot systems. Summary of the invention

[0006] In order to avoid and overcome the technical problems existing in the prior art, the present invention provides a path planning method, terminal and storage medium for a multi-robot system. The present invention solves the path planning problem of multiple constant speed robots by adopting a hierarchical structure combining global path planning and local path planning based on reinforcement learning.

[0007] To achieve the above object, the present invention provides the following technical solutions:

[0008] The present invention discloses a path planning method for a multi-robot system, comprising the following steps:

[0009] S1. Obtain a grid map and basic information of all robots in the multi-robot system; the basic information includes the position, radius, running speed and steering angle of the robots.

[0010] S2. Determine the starting point and target point of each robot in the grid map.

[0011] S3. Represent the entire grid map in the form of a weighted directed graph.

[0012] S4. Perform global path planning from the starting point to the target point for each robot.

[0013] S5. Find the intersection of all robot global paths.

[0014] S6. Determine the intersection points between the global paths of each robot that are conflict points, and plan local paths for the robots at the conflict points based on the RVO dynamic obstacle avoidance algorithm according to the basic information of the robots corresponding to the conflict points and the expected collision time.

[0015] S7. After completing the global path planning and local path planning, i.e., the hierarchical path planning of the multi-robot system, use the long short-term memory network to extract the time information of the hierarchical path.

[0016] S8. Based on the time information extracted from the hierarchical path, find the optimal path planning scheme for the multi-robot system based on the reinforcement learning planning method.

[0017] As a further improvement of the above scheme, step S8 includes the following specific steps:

[0018] (1) Data collection phase

[0019] Initialize the neural network parameters and run the old policies of each robot within T time intervals.

[0020] Collect the observation, action, and reward information within T time intervals and store them. Use the generalized advantage estimator to estimate the quality of each action.

[0021] (2) Policy update phase

[0022] Construct the robot objective function and value loss function.

[0023] Use the optimizer to optimize the parameters in the objective function and value loss function to obtain the new policy of each robot.

[0024] Calculate the KL divergence between the old policy and the new policy of each robot, and determine whether the KL divergence is greater than the preset threshold; if so, continue to update the policy, otherwise stop updating.

[0025] As a further improvement of the above scheme, the following specific steps are included in step S6:

[0026] S61. Calculate the relative positions of two robots A and B at the intersection:

[0027]

[0028] In the formula, represents the future position of robot A relative to robot B; v A represents the current speed of robot A; v B represents the current speed of robot B; t represents the expected collision time; represents the speed of robot A relative to robot B.

[0029] S62. Determine whether is less than the minimum interval d. If so, determine the intersection of robots A and B as a conflict point, and calculate the VO area of A generated by B, i.e., The calculation formula is:

[0030]

[0031] λ(p, v) = {p + tv | t ≥ 0}

[0032]

[0033] wherein, is the Minkowski sum of robots A and B; -A represents the reflection of robot A in the coordinate system; λ(p, v) is the ray of the robot along the direction of velocity v from the current position p; is an empty set.

[0034] S63. According to calculate the RVO region of A generated by B, that is The calculation formula is:

[0035]

[0036] S64. According to obtain the new velocity set of robot A

[0037] S65. Use the reward function to judge the quality of any velocity v in the new velocity set of robot A t to control the robot to update the action strategy, and generate the road calculation result accordingly to complete the local path planning of the robot.

[0038] As a further improvement of the above solution, in step S62, when is not less than the minimum interval d, then directly use the original global paths of robots A and B as the road calculation result.

[0039] As a further improvement of the above solution, in step S3, the weighted directed graph form G of the grid map is expressed as:

[0040] G = (V, E, W)

[0041] wherein, V is the set of vertices, and the vertices include the starting point, the ending point and the intersection points; E represents the set of edges; W represents the set of edge weights.

[0042] As a further improvement of the above solution, in step S4, the Dijkstra algorithm is used to perform global path planning for each robot.

[0043] As a further improvement of the above solution, in step S1, the SLAM is used to process the image information of the multi-robot system operation scenario to obtain the grid map.

[0044] As a further improvement of the above solution, when obtaining the grid map, the size of the map, the obstacle density, and the radius of the restricted area to prevent entry conflicts are also obtained respectively.

[0045] The present invention also discloses a computer terminal, which includes a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the program, the steps of the path planning method of the above multi-robot system are implemented.

[0046] The present invention also discloses a computer-readable storage medium, on which a computer program is stored. When the program is executed by a processor, the steps of the path planning method of the above multi-robot system are implemented.

[0047] Compared with the prior art, the beneficial effects of the present invention are:

[0048] 1. The path planning method of the multi-robot system disclosed by the present invention combines global path planning and local path planning based on reinforcement learning. By introducing global guidance, the robot can reach the destination based on a fixed learning model, and search for the conflict points where each robot will collide along its respective global path. Then, based on the time and space information of the local area and reinforcement learning, local path planning is carried out for the robots at the conflict points, so that the robots can avoid collisions with potential obstacles and unnecessary detours in a real-time dynamic environment. Thus, in the working condition where multiple robots run in parallel and at the same speed, the problems that robots are difficult to cross obstacles and avoid collisions are solved.

[0049] 2. The path planning method also determines the representation form of the robot and the environmental state. On the one hand, by establishing a suitable deep neural network model, the model takes the effective environmental state as input, and under the basic principle of global guidance, outputs it as a series of effective actions. If there are two path intersections, and if the intersection is a conflict point, other paths are sought to avoid the occurrence of the conflict point, and the deadlock phenomenon caused by path conflicts is solved. On the other hand, taking the output of the deep neural network model as the path of the robot, a new reward function is established based on the RVO or VO region, encouraging the robot to explore all potential paths, continuously optimizing the path planning of the robot, seeking to reach the target point in the shortest time, avoiding collisions between robots, and solving the phenomenon that constant-speed parallel robots cannot cross.

[0050] 3. The computer terminal and computer-readable storage medium disclosed by the present invention can produce the same beneficial effects as the above method by applying the path planning method of the multi-robot system, which will not be elaborated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1Flowchart of the path planning method for the multi-robot system in Embodiment 1 of the present invention.

[0052] Figure 2 Flowchart of the global path planning using the Dijkstra algorithm in Embodiment 1 of the present invention.

[0053] Figure 3 Specific flowchart of step S6 in Embodiment 1 of the present invention.

[0054] Figure 4 Flowchart of finding the optimal path planning scheme for the multi-robot system based on the reinforcement learning planning method in Embodiment 1 of the present invention. Detailed implementation manners

[0055] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0056] Embodiment 1

[0057] To solve the path planning problem of the multi-constant-speed robot system, this embodiment provides a path planning method for the multi-robot system. The robot is a circular robot car (the speed v of the robot is constant, and the direction can be changed, v ∈ [v min , v max ), the radius of the robot is R, and the steering angle is α, α ∈ [α min , α max . Environmental settings: The map size is 30 * 30, the obstacle density is μ, μ ∈ [0, 1]. The radius Δ entry of the restricted area to prevent entry into conflicts. The starting and target cells are randomly generated.

[0058] The present invention uses a hierarchical structure to solve the path planning problem of multi-constant-speed robots. This path planning method combines global path planning and local path planning based on reinforcement learning. By introducing global guidance, the car can reach the destination based on a fixed learning model. Then, through the local path planner based on reinforcement learning, the car can avoid collisions with potential obstacles and unnecessary detours in the real-time dynamic environment using the time and space information of the local area. Each time the car starts, it has to follow the global path planning, and the global path planning is only calculated once and remains unchanged.

[0059] In this embodiment, the method combines global path planning and local path planning based on reinforcement learning. By introducing global guidance, the vehicle (robot) can reach the destination based on a fixed learning model. Subsequently, through the local path planner based on reinforcement learning, using the time and space information of the local area, the vehicle can avoid collisions with potential obstacles and unnecessary detours in a real-time dynamic environment. At the beginning of each operation, the vehicle follows the global path planning, which is calculated only once and remains unchanged.

[0060] Please refer to Figure 1 , in this embodiment, the path planning method for the multi-robot system may include the following steps, namely steps S1 - S8.

[0061] S1. Obtain the grid map and the basic information of all robots in the multi-robot system; the basic information includes the position, radius, running speed, and steering angle of the robot.

[0062] In this embodiment, the grid map can be obtained by processing the image information of the running scenario of the multi-robot system through SLAM (simultaneous localization and mapping). SLAM means placing a robot in an unknown environment, allowing the robot to map the environment while moving and also know which direction to go next. For example, Mars rovers, floor cleaning robots, etc. The specific principle will not be elaborated here.

[0063] When obtaining the grid map, the basic information of the robot and the environmental information of the map can also be obtained. The robot is a circular robot vehicle (the speed v of the robot is constant, and the direction can be changed, v ∈ [v min , v max ), the radius of the robot is R, and the steering angle is α, α ∈ [α min , α max . Environmental information: The map size is, for example, 30 * 30, the obstacle density is μ, μ ∈ [0, 1]. The radius Δ of the restricted area to prevent entry into conflicts entry . The starting and target cells are randomly generated.

[0064] S2. Determine the starting point s and the target point g of each robot in the grid map.

[0065] S3. Represent the entire grid map in the form of a weighted directed graph. In this embodiment, the weighted directed graph form G of the grid map is represented as:

[0066] G = (V, E, W)

[0067] Wherein, V is the set of vertices, and the vertices include the starting point, the ending point, and the intersection points (the intersections of roads); E represents the set of edges; W represents the set of edge weights. Among them, e ij ∈E, and the edge e ij represents from vertex i to vertex j, denoted as e ij : = {v i , v j}. w ij (e ij ) represents the weight of the edge e ij , that is, the distance from vertex i to vertex j. The shortest distance from v i to v j is denoted as dist[v i , v i .

[0068] S4. Perform global path planning from the starting point to the target point for each robot.

[0069] Please refer to Figure 2 . In this embodiment, the Dijkstra algorithm can be used for global path planning. The specific steps are as follows, namely steps S41 - S45.

[0070] S41. Establish a Close table and store the value of dis[i]; dis[i] represents the shortest distance from the starting point to node i; vis[i] represents whether node i has been popped from the Open table.

[0071] S42. Establish an Open table to store records: (point x, the distance from the starting point to point x), and the Open table is sorted according to the distance.

[0072] S43. Let dis[starting point] = 0 (the distance from the starting point to other nodes is infinite), and (source point, 0) enters the Open table.

[0073] S44. Determine whether the Open table is empty; if so, execute step S45; otherwise, pop the minimum distance (point u, the distance from the starting point to point u) from the Open table, and determine whether vis[u] is true.

[0074] If vis[u] = true, repeat step S44 until the Open table is empty.

[0075] If vis[u] = false, let vis[u] = true, indicating that u has been popped; then find each edge of u. Assume that an edge goes to v, then the weight is w uv ; if vis[v] = false and dis[u] + w uv < dis[v], let dis[v] = dis[u] + w uv , and put (v, dis[u] + wuv ) Put it into the Open list. After processing each edge of u, repeat step S44 until the Open list is empty.

[0076] S45. The Close list records the shortest distances from the starting point to each node, and finds the shortest global path from the starting point to the source point.

[0077] S5. Find out the intersection points of all the robot global paths.

[0078] S6. Determine the intersection points that are conflict points (i.e., two robots will move to this point at the same time and collide) among the global paths of each robot. Based on the basic information of the robots corresponding to the conflict points and the expected collision time, perform local path planning for the robots at this place based on the RVO dynamic obstacle avoidance algorithm. Please refer to Figure 3 , which specifically includes the following steps, namely steps S61 - S65.

[0079] S61. Calculate the relative positions of two robots A and B at the intersection point:

[0080]

[0081] In the formula, represents the future position of robot A relative to robot B; v A represents the current speed of robot A; v B represents the current speed of robot B; t represents the expected collision time; represents the speed of robot A relative to robot B.

[0082] S62. Judge whether is less than the minimum interval d. If so, determine that the intersection point of robots A and B is a conflict point, and calculate the VO region of A generated by B, that is The calculation formula is:

[0083]

[0084] λ(p, v) = {p + tv | t ≥ 0}

[0085]

[0086] In the formula, is the Minkowski sum of robots A and B; -A represents the reflection of robot A in the coordinate system; λ(p, v) is the ray of the robot along the direction of velocity v with the current position p; is an empty set.

[0087] In step S62, when is not less than the minimum interval d, then directly use the original global paths of robots A and B as the road calculation results.

[0088] S63. According to calculate the RVO area of A generated by B, i.e., The calculation formula is:

[0089]

[0090] S64. According to obtain the new speed set of robot A

[0091] S65. Use the reward function to judge the quality of any speed v in the new speed set of robot A t and control the robot to update the action strategy, thereby generating the road calculation result and completing the local path planning of the robot.

[0092] Among them, the role of the reward function in step S65 is as follows: First, based on the global path (except for the conflict point range, which is related to the actual size and speed of the robot), if the robot is not on the global path, a small negative reward is given; when the robot collides with a static obstacle, a large negative reward is given; when the robot is in a global path cell, a large positive reward is given; at the conflict point, when the robot's speed is close to the required speed or the collision risk is low, a positive reward is given; when the robot collides with other robots, a large negative reward is given.

[0093] S7. After completing a global path planning and a local path planning, i.e., the hierarchical path planning of the multi-robot system, use a long short-term memory network (LSTM) to extract time information from the hierarchical path. Use the long short-term memory network to extract features from the grid map containing robots in a continuous space, and rearrange the external perception measurements at each step into sequential input data to generate a fixed-size environmental feature vector.

[0094] S8. According to the time information extracted from the hierarchical path above, find the optimal path planning scheme of the multi-robot system based on the reinforcement learning planning method. That is: input the time information in the form of a one-dimensional vector into the RL planner (RLP) (RLP, Reinforcement Learning Planner), and use the RL planner to find the optimal path planning scheme of the multi-robot system.

[0095] In the multi-robot system, first find the global path of each robot, find all the conflict points, use RVO to complete the local path planning at the conflict points to solve the multi-robot collision conflict problem, and finally set the reward function based on reinforcement learning to find the optimal speed that meets the expectations for a single robot in a collision-free scenario, thereby finding the optimal path planning scheme of the multi-robot system.

[0096] Please refer to Figure 4 Step S8 includes the following specific steps, namely two phases of (1) and (2).

[0097] (1) Data collection phase

[0098] Input a one-dimensional vector processed by a long short-term memory network.

[0099] Initialize the neural network parameters π θ and V ψ (θ and ψ are the parameters set during policy update), and run the old policy π of robot i within T time intervals t θold .

[0100] Collect the observation, action and reward information of robot i within T time intervals t, that is and store them in the Buffer cache space; use the generalized advantage estimator to estimate the quality of each action, that is The formula for calculating the quality of the action is:[[]]

[0101]

[0102] In the formula, γ is the discount rate, μ = 1, δ t = r t + γV(s t+1 ) - V(s t ); s t is the state of the robot at time t, and V(·) is the state value function.

[0103] (2) Policy update phase

[0104] Construct the robot objective function and value loss function.

[0105] Use the optimizer to optimize the parameters in the objective function and value loss function to obtain the new policy of each robot.

[0106] Calculate the KL divergence between the old policy and the new policy of each robot, and determine whether the KL divergence is greater than a preset threshold; if so, continuously update the policy, otherwise stop updating.

[0107] Embodiment 2

[0108] This embodiment provides a computer terminal, which includes a memory, a processor, and a computer program stored on the memory and executable on the processor.

[0109] The computer terminal can be a smart phone, a tablet computer, a laptop computer, etc. that can execute programs. In some embodiments, the processor can be a Central Processing Unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chips. The processor is generally used to control the overall operation of the computer device. In this embodiment, the processor is used to run the program code stored in the memory or process data. When the processor executes the program, it implements the steps of the path planning method of the multi-robot system in Embodiment 1.

[0110] Embodiment 3

[0111] This embodiment provides a computer-readable storage medium, on which a computer program is stored. When the program is executed by a processor, it implements the steps of the path planning method of the multi-robot system in Embodiment 1.

[0112] The computer-readable storage medium can include flash memory, a hard disk, a multimedia card, a card-type memory (such as an SD or DX memory, etc.), a random access memory (RAM), a static random access memory (SRAM), a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), a programmable read-only memory (PROM), a magnetic memory, a magnetic disk, an optical disk, etc. In some embodiments, the storage medium can be an internal storage unit of the computer device, such as the hard disk or memory of the computer device. In other embodiments, the storage medium can also be an external storage device of the computer device, such as a plug-in hard disk equipped on the computer device, a Smart Media Card (SMC), a Secure Digital (SD) card, a Flash Card, etc. Of course, the storage medium can also include both the internal storage unit and the external storage device of the computer device. In this embodiment, the memory is generally used to store the operating system and various application software installed on the computer device. In addition, the memory can also be used to temporarily store various data that have been output or will be output.

[0113] As described above, only the preferred specific embodiments of the present invention are provided, but the protection scope of the present invention is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present invention, according to the technical solution and inventive concept of the present invention, makes equivalent substitutions or changes, and all should be covered by the protection scope of the present invention.

Claims

1. A path planning method for a multi-robot system, characterized in that, It includes the following steps: S1. Obtain the grid map and the basic information of all robots in the multi-robot system; the basic information includes the position, radius, running speed, and steering angle of the robots; among them, the magnitudes of the running speeds of each robot are the same; S2. Determine the starting point and the target point of each robot in the grid map; S3. Represent the entire grid map in the form of a weighted directed graph; S4. Perform global path planning from the starting point to the target point for each robot; S5. Find the intersection points of all robots' global paths; S6. Determine the intersection points that belong to conflict points among the global paths of each robot, and based on the basic information of the corresponding robots at the conflict points and the expected collision time, perform local path planning for the robots at that location based on the RVO dynamic obstacle avoidance algorithm; the following specific steps are included in step S6: S61. Calculate the relative positions of two robots A and B at the intersection point: In the formula, represents the future position of robot A relative to robot B; v A represents the current speed of robot A; v B represents the current speed of robot B; t represents the expected collision time; represents the speed of robot A relative to robot B; S62. Judgment Determine whether it is less than the minimum interval d. If so, determine the intersection point of robots A and B as a conflict point, and calculate the VO area of A generated by B, that is The calculation formula is: λ(p,v)={p+tv|t≥0} wherein, is the Minkowski sum of robots A and B; -A represents the reflection of robot A in the coordinate system; λ(p, v) is the ray of the robot along the direction of velocity v at the current position p; is an empty set; S63. According to calculate the RVO area of A generated by B, that is The calculation formula is: S64. According to obtain a new speed set of robot A S65. Use a reward function to judge the quality of any velocity v in the new velocity set of robot A, control the robot to update the action strategy, generate a road calculation result accordingly, and complete the local path planning of the robot; t The quality of, control the robot to update the action strategy, generate a road calculation result accordingly, and complete the local path planning of the robot; S7. After completing a global path planning and a local path planning, that is, the hierarchical path planning of the multi-robot system, use a long short-term memory network to extract time information from the hierarchical path; S8. According to the time information extracted from the hierarchical path, find the optimal path planning scheme of the multi-robot system based on the reinforcement learning planning method.

2. The path planning method for a multi-robot system according to claim 1, characterized in that, Step S8 includes the following specific steps: (1) Data collection stage Initialize the neural network parameters and run the old policies of each robot within T time intervals; Collect the observation, action, and reward information within T time intervals and store them, and use the generalized advantage estimator to estimate the quality of each action; (2) Policy update stage Construct the robot objective function and the value loss function; Use an optimizer to optimize the parameters in the objective function and the value loss function to obtain the new policies of each robot; Calculate the KL divergence between the old policies and the new policies of each robot, and determine whether the KL divergence is greater than a preset threshold; if so, continuously update the policy, otherwise stop updating.

3. A path planning method for a multi-robot system according to claim 1, characterized in that In step S62, when is not less than the minimum interval d, the original global paths of robots A and B are directly used as the road calculation results.

4. The path planning method for a multi-robot system according to claim 1, characterized in that, In step S3, the weighted directed graph form G of the grid map is expressed as: G=(V,E,W) In the formula, V is the set of vertices, and the vertices include the starting point, the ending point, and the intersection points; E represents the set of edges; W represents the set of edge weights.

5. A path planning method for a multi-robot system according to claim 1, characterized in that, In step S4, the Dijkstra algorithm is used to perform global path planning for each robot.

6. The path planning method for a multi-robot system according to claim 1, characterized in that In step S1, the SLAM is used to process the image information of the running scenario of the multi-robot system to obtain the grid map.

7. A path planning method for a multi-robot system according to claim 6, characterized in that, When obtaining the grid map, the size of the map, the obstacle density, and the no-go zone radius to prevent entry into conflicts are also obtained respectively.

8. A computer terminal, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps of a path planning method for a multi-robot system as described in any one of claims 1 to 7.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the steps of a path planning method for a multi-robot system as described in any one of claims 1 to 7.

Citation Information

Patent Citations

  • Multi-unmanned aerial vehicle global and local path intelligent planning method and system

    CN115494866A

  • Multi-robot unknown environment path planning method based on geometric graph neural network

    CN115907248A