Multi-robot task allocation and path planning method and system based on evolutionary optimization and learning driving
By combining evolutionary optimization and reinforcement learning algorithms for task allocation and path planning in multi-robot systems, the limitations of existing systems in computing resource consumption and path conflict are solved, and more efficient task execution and resource utilization are achieved.
Patent Information
- Application Number
- CN202411700091.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-26
- Publication Date
- 2025-04-18
AI Technical Summary
The existing multi-robot system has limitations in task allocation and path planning, high computing resource consumption, poor system scalability and real-time performance, and fail to effectively consider dynamic interactions and path conflicts between robots, resulting in inefficient task completion and waste of resources.
A method based on evolutionary optimization and learning drive is adopted, and a genetic algorithm is used to allocate global tasks, and adaptive path planning is carried out in combination with reinforcement learning algorithms. A closed-loop optimization system is formed through evolutionary algorithms to optimize task allocation schemes and reinforcement learning optimization path planning.
The task allocation and path planning efficiency of multi-robot systems in complex dynamic environments is improved, the stagnation caused by path conflicts is reduced, and the system's productivity and resource utilization are improved.
Smart Images
Figure CN119847142A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of multi-robot collaboration and path optimization, and in particular to a multi-robot task allocation and path planning method and system based on evolutionary optimization and learning drive. Background Art
[0002] With the rapid development of robotics and artificial intelligence, multi-robot systems are increasingly used in warehousing logistics, industrial production and other fields. In order to improve production efficiency and flexibility, multiple robots collaborating to complete complex tasks has become an effective solution. However, existing multi-robot systems still have many limitations in task allocation and path planning. Traditional centralized path planning methods mainly rely on global information, which not only requires high computing resources, but also significantly reduces the scalability and real-time performance of the system as the task scale increases. In addition, the task allocation process does not consider the dynamic interaction and path conflict between robots, which easily leads to problems such as low efficiency in task completion and waste of resources.
[0003] In addition, in actual industrial scenarios, the efficiency of task allocation and path planning has a significant impact on the productivity of the overall system. Most existing methods are based on heuristic algorithms or linear programming. When the task complexity is high or the environment changes dynamically, it is difficult to quickly find the optimal solution and cannot fully adapt to the actual industrial needs. Therefore, how to effectively allocate tasks and plan paths taking into account potential conflicts has become a difficult problem that needs to be solved in multi-robot systems in industrial applications. Summary of the invention
[0004] The purpose of the present invention is to overcome the defects of the above-mentioned prior art and to provide a multi-robot task allocation and path planning method and system based on evolutionary optimization and learning-driven, which uses evolutionary algorithm for global task allocation and combines reinforcement learning for adaptive path planning, so as to improve the task allocation and path planning efficiency of multi-robot systems in complex dynamic environments and reduce stagnation caused by path conflicts.
[0005] The purpose of the present invention can be achieved by the following technical solution: A multi-robot task allocation and path planning method based on evolutionary optimization and learning drive, comprising the following steps:
[0006] S1. Set up a multi-robot task allocation problem;
[0007] S2. A genetic algorithm is used to solve the task allocation problem, and a reinforcement learning algorithm is used to generate a collision-free path for the solved task allocation solution.
[0008] Furthermore, the specific process of step S1 is as follows:
[0009] In a multi-robot system, given a set of tasks {T1,T2,…,T n}, each task contains the starting position S i =(x i ,y i ) and the end position G i =(x i ′ ,y i ′ ), we need n robots {R1, R2, …, R n} Assign tasks and plan the optimal path without collision so that all robots can complete their tasks in the shortest time;
[0010] The goal of task allocation is to find an optimal allocation solution σ, where σ(i) represents the task T i Assigned to robot R σ(i) ;
[0011] The goal of path planning is to generate a path for each robot from the starting point S while avoiding collisions. i To the end point G i The shortest path.
[0012] Furthermore, the step S2 specifically includes the following steps:
[0013] S21, initializing the task allocation population and randomly generating multiple initial task allocation schemes;
[0014] S22, calculating the fitness of multiple initial task allocation schemes, and generating a new population through selection, crossover and mutation operations in the genetic algorithm;
[0015] S23, using reinforcement learning algorithm to plan and generate collision-free paths for each task allocation scheme within the new population;
[0016] S24, update and calculate the corresponding fitness according to the path length, and then generate a new population through selection, crossover and mutation operations in the genetic algorithm, and then return to step S23 until convergence to obtain the optimal task allocation and path planning solution.
[0017] Furthermore, the calculation formula of the fitness is:
[0018]
[0019] Among them, F(σ) is the fitness function, which is the inverse of the total path cost, L(R σ(i) ,T i ) is the robot R σ(i) Execute Task T i is the number of time steps, and n is the total number of paths.
[0020] Furthermore, when generating a new population through selection, crossover and mutation operations in the genetic algorithm in step S22 and step S24, the set constraints include task uniqueness constraints and robot uniqueness constraints.
[0021] Furthermore, the task uniqueness constraint is specifically:
[0022]
[0023] That is, each task T i Can only be assigned to one robot R j ;
[0024] The robot uniqueness constraint is specifically:
[0025]
[0026] That is, each robot can only perform one task.
[0027] Furthermore, when the reinforcement learning algorithm is used in step S23 to plan and generate collision-free paths for each task allocation scheme in the new population, the constraint condition is set as a collision avoidance constraint.
[0028] Furthermore, the collision avoidance constraints are specifically:
[0029]
[0030] That is, at any time t, two robots cannot occupy the same position p at the same time.
[0031] Furthermore, the step S23 specifically adopts the Q learning algorithm in reinforcement learning, and gradually generates a collision-free optimal path by allowing each robot to select the next action according to local observation information, wherein the path planning problem is expressed as a Markov decision process (MDP), whose elements include:
[0032] State space: The state of each robot is its current position s t =(x t ,y t ) and the observation information of the surrounding environment t ;
[0033] Action space: The robot's optional actions are a t ∈{up,down,left,right,stay};
[0034] Reward function: Each robot receives a reward R = +30 when it completes a task, and each move consumes a small cost C = -0.75. If a collision occurs, the reward is R = -5.
[0035] The update rule for Q-learning is as follows:
[0036]
[0037] Among them, α is the learning rate, γ is the discount factor, and r t is the current reward, a ′ The best next action.
[0038] A multi-robot task allocation and path planning system based on evolutionary optimization and learning drive, including a task allocation module and a path planning module with interactive closed-loop connection. The task allocation module iteratively optimizes the task allocation scheme through a genetic algorithm. The population contains different task allocation schemes. After fitness calculation and genetic operation, efficient allocation results are optimized generation by generation, while considering conflict resolution time and total cost.
[0039] The path planning module receives the task assignment results output by the task assignment module, plans the corresponding path for each robot, realizes information sharing between robots through the communication module of the multi-head attention mechanism, uses GRU (Gated Recurrent Unit) to process time series information, and makes path decisions through a double-layer Q network to generate a low-conflict, high-efficiency path, and feeds back the cost of path planning to the task assignment module.
[0040] Compared with the prior art, the present invention has the following advantages:
[0041] The present invention combines evolutionary algorithms with reinforcement learning algorithms, uses evolutionary algorithms to globally optimize task allocation, and uses reinforcement learning to adjust collision-free path planning through adaptive learning. This enables robots to not only complete path planning autonomously, but also reduce path conflicts between robots and resource waste in task allocation, further improving the task processing efficiency of the multi-robot system.
[0042] In the cost estimation process of task allocation, the present invention specifically considers the impact of potential path conflicts on the total cost. When allocating tasks, the cost is not only estimated based on the path length, but also the prediction of the task conflict time is incorporated into the cost calculation, so as to more accurately evaluate the execution cost of the task. Compared with the traditional solution that only considers the path length, this method can effectively avoid high-cost task allocation caused by conflicts, thereby reducing the stagnation caused by path conflicts in multi-robot systems, improving the efficiency of overall task allocation and path planning, and enhancing the productivity and resource utilization of the system. BRIEF DESCRIPTION OF THE DRAWINGS
[0043] Figure 1 It is a schematic diagram of the method flow of the present invention;
[0044] Figure 2 It is a schematic diagram of the system framework of the present invention;
[0045] Figure 3 Schematic diagram of simulation experiment comparison of different schemes in the embodiment. DETAILED DESCRIPTION
[0046] The present invention is described in detail below with reference to the accompanying drawings and specific embodiments.
[0047] Example
[0048] like Figure 1 As shown, a multi-robot task allocation and path planning method based on evolutionary optimization and learning-driven includes the following steps:
[0049] S1. Set up a multi-robot task allocation problem;
[0050] S2. A genetic algorithm is used to solve the task allocation problem, and a reinforcement learning algorithm is used to generate a collision-free path for the solved task allocation solution.
[0051] Using the above method, a multi-robot task allocation and path planning system based on evolutionary optimization and learning-driven is constructed. Figure 2 As shown in the figure, it is mainly divided into two modules: the task allocation module iteratively optimizes the task allocation scheme through the evolutionary algorithm. The initial population contains different allocation methods. After fitness calculation and genetic operations (such as crossover and mutation), efficient allocation results are optimized generation by generation, while considering the conflict resolution time and total cost;
[0052] The path planning module receives the results of task allocation and plans a path for each agent (i.e., the robot involved in the task allocation plan). This module realizes information sharing between agents through the communication module of the multi-head attention mechanism, uses GRU to process time series information, and makes path decisions through a double-layer Q network to generate a low-conflict, high-efficiency path.
[0053] The two modules of task allocation and path planning form a closed-loop optimization, and the cost of path planning is fed back to the task allocation module, thereby further improving the overall optimization effect.
[0054] Applying the above solution to practice mainly includes:
[0055] 1. Problem Definition
[0056] In a multi-robot system, given a set of tasks {T1,T2,…,T n}, each task contains the starting position S i =(x i ,y i ) and the end position G i =(xi ′ ,y i ′ ), we need n robots {R1, R2, …, R n}Assign tasks and plan the optimal collision-free path so that all robots can complete their tasks in the shortest time.
[0057] The goal of task allocation is to find an optimal allocation solution σ, where σ(i) represents the task T i Assigned to robot R σ(i) The goal of path planning is to generate a path for each robot from the starting point S while avoiding collisions. i To the end point G i The shortest path.
[0058] 2. Task Allocation
[0059] This scheme uses Genetic Algorithm (GA) for task allocation. The genetic algorithm continuously optimizes the task allocation scheme by encoding the chromosomes of task allocation and using operations such as selection, crossover and mutation.
[0060] This scheme designs a fitness function to evaluate the quality of each task allocation scheme. The fitness function is defined as the inverse of the total path cost, that is:
[0061]
[0062] Among them, L(R σ(i) ,T i ) indicates robot R σ(i) Execute Task T i The number of time steps.
[0063] 3. Path Planning
[0064] Path planning uses the Q-learning algorithm in reinforcement learning to gradually generate a collision-free optimal path by allowing each robot to choose the next action based on local observation information. The path planning problem can be expressed as a Markov decision process, whose elements include:
[0065] State space: The state of each robot is its current position s t =(x t ,y t ) and the observation information of the surrounding environment t .
[0066] Action space: The robot's optional actions are a t ∈{up,down,left,right,stay}.
[0067] Reward function: Each robot receives a reward R = +30 when completing a task, and each movement consumes a small cost C = -0.75. If a collision occurs, the reward is R = -5.
[0068] The update rule for Q-learning is as follows:
[0069]
[0070] Among them, α is the learning rate, γ is the discount factor, and r t is the current reward, and a′ is the optimal action for the next step.
[0071] 4. Constraints
[0072] In order to ensure the rationality of task allocation and path planning, this scheme introduces the following constraints in the optimization process:
[0073] Task uniqueness constraint: Each task T i Can only be assigned to one robot R j ,Right now:
[0074]
[0075] Robot uniqueness constraint: Each robot can only perform one task, that is:
[0076]
[0077] Collision avoidance constraint: At any time t, two robots cannot occupy the same position p at the same time, that is:
[0078]
[0079] 5. Design Algorithm Process
[0080] The basic process of the task allocation and path planning algorithm of this solution is as follows:
[0081] 1. Initialize the task allocation population and randomly generate the initial task allocation plan.
[0082] 2. Evaluate the pros and cons of each solution based on the fitness function.
[0083] 3. Generate new populations through selection, crossover and mutation operations in genetic algorithms.
[0084] 4. Use reinforcement learning algorithm to generate collision-free paths for each task assignment scheme and update fitness based on the path length.
[0085] 5. Repeat the above process until task allocation and path planning converge to the optimal solution.
[0086] The effectiveness of this solution has not been verified. This embodiment compares the method of this solution (GA+DHC) with the existing task allocation cost estimation methods GA+EU and GA+A* through simulation experiments, and performs task allocation and path planning for multiple agents. Figure 3 As shown in the figure, by comparison, in a high-density obstacle environment, the GA+DHC algorithm proposed in this scheme effectively reduces the path intersection and conflict between robots, ensuring the smooth execution of the task. This shows that the cost estimation method of this scheme can better consider the conflict factors, thereby improving the coordination of the multi-agent system.
[0087] In summary, this solution uses evolutionary algorithms for global task allocation and combines it with the adaptive path planning capabilities of reinforcement learning to solve the problems of task allocation and path conflict in existing multi-robot systems in complex dynamic environments. It can effectively improve the efficiency of task allocation and path planning of multi-robot systems in complex dynamic environments, reduce stagnation caused by path conflicts, improve the overall productivity and resource utilization of the system, and significantly improve the task execution efficiency and path optimization effect of multi-robot systems in industrial scenarios such as warehousing logistics and intelligent manufacturing.
Claims
1. A multi-robot task allocation and path planning method based on evolutionary optimization and learning-driven, characterized in that: The following steps are involved: S1. Set up a multi-robot task allocation problem; S2. A genetic algorithm is used to solve the task allocation problem, and a reinforcement learning algorithm is used to generate a collision-free path for the solved task allocation solution.
2. The multi-robot task allocation and path planning method based on evolutionary optimization and learning drive according to claim 1 is characterized in that: The specific process of step S1 is as follows: In a multi-robot system, given a set of tasks {T1, T2, ..., T n }, each task contains the starting position S i =(x i ,y i ) and the end position G i =(x′ i , y′ i ), we need n robots {R1, R2, ..., R n } Assign tasks and plan the optimal path without collision so that all robots can complete their tasks in the shortest time; The goal of task allocation is to find an optimal allocation solution σ, where σ(i) represents the task T i Assigned to robot R σ(i) ; The goal of path planning is to generate a path for each robot from the starting point S while avoiding collisions. i To the end point G i The shortest path.
3. The multi-robot task allocation and path planning method based on evolutionary optimization and learning drive according to claim 2 is characterized in that: The step S2 specifically includes the following steps: S21, initializing the task allocation population and randomly generating multiple initial task allocation schemes; S22, calculating the fitness of multiple initial task allocation schemes, and generating a new population through selection, crossover and mutation operations in the genetic algorithm; S23, using reinforcement learning algorithm to plan and generate collision-free paths for each task allocation scheme within the new population; S24, update and calculate the corresponding fitness according to the path length, and then generate a new population through selection, crossover and mutation operations in the genetic algorithm, and then return to step S23 until convergence to obtain the optimal task allocation and path planning solution.
4. The multi-robot task allocation and path planning method based on evolutionary optimization and learning drive according to claim 3 is characterized in that: The calculation formula of the fitness is: Among them, F(σ) is the fitness function, which is the inverse of the total path cost, L(R σ(i) , T i ) is the robot R σ(i) Execute Task T i is the number of time steps, and n is the total number of paths.
5. The multi-robot task allocation and path planning method based on evolutionary optimization and learning drive according to claim 3 is characterized in that: When generating a new population through the selection, crossover and mutation operations in the genetic algorithm in step S22 and step S24, the set constraints include task uniqueness constraints and robot uniqueness constraints.
6. The multi-robot task allocation and path planning method based on evolutionary optimization and learning drive according to claim 5 is characterized in that: The task uniqueness constraint is specifically: That is, each task T i Can only be assigned to one robot R j ; The robot uniqueness constraint is specifically: That is, each robot can only perform one task.
7. The multi-robot task allocation and path planning method based on evolutionary optimization and learning drive according to claim 3 is characterized in that: When the reinforcement learning algorithm is used in step S23 to plan and generate collision-free paths for each task allocation scheme in the new population, the constraint condition is set as a collision avoidance constraint.
8. The multi-robot task allocation and path planning method based on evolutionary optimization and learning drive according to claim 7 is characterized in that: The collision avoidance constraints are specifically: That is, at any time t, two robots cannot occupy the same position p at the same time.
9. The multi-robot task allocation and path planning method based on evolutionary optimization and learning drive according to claim 3 is characterized in that: The step S23 specifically adopts the Q learning algorithm in reinforcement learning, and gradually generates a collision-free optimal path by allowing each robot to select the next action based on local observation information. The path planning problem is represented as a Markov decision process, and its elements include: State space: The state of each robot is its current position s t =(x t ,y t ) and the observation information of the surrounding environment t ; Action space: The robot's optional actions are a t ∈{up,down,left,right,stay}; Reward function: Each robot receives a reward R = +30 when it completes a task, and each move consumes a small cost C = -0.
75. If a collision occurs, the reward is R = -5. The update rule for Q-learning is as follows: Among them, α is the learning rate, γ is the discount factor, and r t is the current reward, a ′ The best next action.
10. A multi-robot task allocation and path planning system based on evolutionary optimization and learning drive, using the multi-robot task allocation and path planning method based on evolutionary optimization and learning drive as claimed in claim 1, characterized in that: It includes a task allocation module and a path planning module connected in an interactive closed loop. The task allocation module iteratively optimizes the task allocation scheme through a genetic algorithm. The population contains different task allocation schemes. After fitness calculation and genetic operation, efficient allocation results are optimized generation by generation, while considering conflict resolution time and total cost. The path planning module receives the task assignment results output by the task assignment module and plans the corresponding path for each robot. The module realizes information sharing between robots through the communication module of the multi-head attention mechanism, uses GRU to process time series information, and makes path decisions through a double-layer Q network to generate a low-conflict, high-efficiency path, and feeds back the cost of path planning to the task assignment module.