A multi-robot path planning algorithm based on priority game and A3C

Abstract

The application discloses a kind of multi-robot path planning algorithm based on priority game and A3C;The algorithm of the present application first initializes the state and priority of all robots, then path planning is carried out according to the priority.In the planning process, the robot collects experience by interacting with the environment, and uses A3C model for learning and optimization.The algorithm uses a specific reward function to evaluate the performance of the robot to guide its learning behavior.Through continuous iteration and optimization, the robot gradually learns the optimal path planning strategy.The present application has the characteristics of strong practicability, and can be widely used in scenarios requiring multi-robot collaboration, such as warehouse logistics, intelligent manufacturing and other fields.Compared with traditional path planning methods, the present application improves the efficiency and quality of path planning, reduces the collision rate, and has better adaptability and robustness.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of robot technology, artificial intelligence and reinforcement learning, and particularly relates to a multi-robot path planning algorithm based on priority game and A3C. BACKGROUND

[0002] With the continuous development of robot technology, multi-robot systems have been widely applied in many fields, such as warehouse logistics, intelligent manufacturing, environmental detection, etc. In these application scenarios, multiple robots often need to work together to complete complex tasks. However, how to effectively plan the paths of multiple robots and avoid collisions and conflicts between them is a challenge faced by current multi-robot systems.

[0003] Traditional path planning methods are usually based on graph search, potential field method, etc. These methods can achieve good results in simple scenarios. However, when the number of robots increases and the complexity of the environment increases, traditional methods often fail to meet the requirements. In addition, these methods usually lack consideration of the interaction between robots, and cannot well handle the coordination problem between multiple robots.

[0004] In recent years, with the development of artificial intelligence and reinforcement learning technology, more and more researchers have begun to explore the use of these technologies to solve the problem of multi-robot path planning. Reinforcement learning allows robots to continuously try and error, learn and optimize in the process of interacting with the environment, so as to find the optimal behavior strategy. A3C algorithm, as an important method in the field of reinforcement learning, has the advantages of good parallelism and high learning efficiency, and is very suitable for solving the problem of multi-robot path planning.

[0005] However, the current multi-robot path planning method based on reinforcement learning still faces some challenges. For example, how to effectively handle the priority problem between robots and avoid collisions and conflicts in the path planning process; how to design a suitable reward function to guide the learning behavior of robots; how to balance the relationship between exploration and utilization to improve learning efficiency, etc. SUMMARY

[0006] In view of the limitations and challenges of traditional path planning methods in handling multi-robot systems, the present application provides a multi-robot path planning algorithm based on priority game and A3C. It aims to solve the problems existing in the prior art and improve the path planning efficiency and safety of multi-robot systems.

[0007] The application allocates different priorities to multiple robots by constructing a priority game model, thereby effectively avoiding collisions and conflicts in path planning. At the same time, combined with the A3C reinforcement learning algorithm, the robot can autonomously learn and optimize the path planning strategy to adapt to complex and variable environments and task requirements. The application aims to improve the efficiency and safety of multi-robot system path planning and provide a more intelligent and efficient solution for robot collaborative work.

[0008] The application is implemented by the following technical solutions:

[0009] A multi-robot path planning algorithm based on priority game and A3C, comprising the following steps:

[0010] I. Initialization

[0011] At the beginning of algorithm startup, a series of initialization operations are essential to ensure smooth execution of the multi-robot path planning process. The specific steps are described as follows:

[0012] 1. Robot state initialization

[0013] Set the initial state of each robot, including position P i 0 , velocity V i 0 , direction , and target position G i . These state parameters will serve as the starting point for path planning and provide baseline data for subsequent calculations. Mathematically, the robot state initialization can be represented as:

[0014]

[0015] where, represents the initial state of robot i.

[0016] 2. Priority setting

[0017] To avoid collisions and conflicts, an initial priority Pr i 0 is set for each robot. The priority is calculated based on factors such as task importance W i , current position P i 0 , target position G i . The specific calculation formula is as follows:

[0018] Pr i 0 = σ(W i ) + ρ(f(P i 0 , G i ))

[0019] where σ and ρ are weight functions, f(P i 0 ,G i ) is an evaluation function of the position and target position relationship. This formula takes into account the task importance and the current state of the robot, ensuring the rationality and effectiveness of priority setting.

[0020] 3. A3C model parameter initialization

[0021] As the core algorithm, the A3C model needs to set the parameters of the actor network and the critic network during the initialization phase. The actor network parameters include weights θ a and biases b a , and the critic network parameters include weights θ c and biases b c . At the same time, the learning rate η also needs to be set. These parameters will directly affect the learning performance of the A3C model, so they need to be carefully designed. Mathematically, the A3C model parameter initialization can be represented as:

[0022]

[0023] where θ represents the initial parameter set of the A3C model.

[0024] Through the above initialization steps, the algorithm lays a solid foundation for multi-robot path planning. In the subsequent steps, the algorithm will use the initialized state and parameters to perform path planning through priority game and A3C algorithm, to achieve collaborative work and efficient path selection among robots.

[0025] II. Priority game

[0026] After completing the initialization step, the algorithm immediately enters the priority game phase. This phase aims to set a priority mechanism to ensure that robots with high priority can have priority in path use, thereby reducing the risk of collision and conflict.

[0027] At each time step, the algorithm first retrieves and updates the priority list of the robot. This list is dynamically adjusted based on the task importance W i , current position P i , target position G i and other relevant factors. The priority calculation formula is as follows:

[0028] Priority i = α * W i + β * f(P i , G i ) + γ * g(Environment i)

[0029] where α, β, γ are weight coefficients, f(P i , S i ) represents the function relationship between the current position and the target position of the robot, g(Environment i ) represents the evaluation function of the environment where the robot is located.

[0030] Subsequently, the algorithm plans paths for robots in order of priority from high to low. For each robot, the algorithm will combine its current state S i (including position, speed, direction, etc.) and environmental information E i (such as obstacle position, other robot state, etc.), and calculate the optimal path Path i from the current position to the target position through path planning algorithms (such as A* algorithm, RRT algorithm, etc.).

[0031] When planning paths, the algorithm will pay special attention to the conflict problem between paths. If there is a potential conflict between the path of a high-priority robot and the path of a low-priority robot, the algorithm will initiate an avoidance strategy. This strategy adjusts the cost function C i in the path planning algorithm so that the low-priority robot tends to avoid the high-priority robot when choosing a path. The adjustment of the cost function can be represented as:

[0032] C i′ = C i + λ * Overlap(Path high , Path i )

[0033] where Overlap(Path high , Path i ) represents the degree of overlap between the path of the high-priority robot and the current robot path, and λ is the avoidance coefficient.

[0034] Through priority game, the algorithm realizes ordered path planning in multi-robot systems. High-priority robots can obtain path usage rights first, while low-priority robots avoid conflicts by adjusting their own behavior, thereby ensuring the efficiency and safety of overall path planning.

[0035] It should be noted that during the priority game process, the algorithm needs to ensure the real-time and dynamic nature of path planning. With real-time updates of robot state and environmental information, the priority list and path planning results need to be adjusted synchronously. Therefore, the algorithm needs to have efficient data processing capability and flexible decision-making mechanism to cope with complex and changing environments and task requirements.

[0036] In summary, by introducing the priority game mechanism, the algorithm realizes the ordered path planning based on the priority of robots, effectively reduces the risk of collision and conflict, and provides strong support for the collaborative work of robots.

[0037] III. State input and action selection

[0038] After determining the path planning order of each robot in the priority game phase, the algorithm proceeds to the state input and action selection phase. The core of this phase is to input the current state of the robot into the A3C model and select the corresponding action based on the model output, thereby realizing the autonomous path planning of the robot.

[0039] Specifically, the algorithm first inputs the state information of the current robot, including the position P i , the velocity V i , the direction D i , the target position G i , etc., as input data X i to the actor network of the A3C model. The actor network generates a probability distribution of actions through a parameterized function f a (θ a , X i ), where θ a is the parameter of the actor network.

[0040] The action probability distribution can be represented as:

[0041] π(a|X i ; θ a ) = f a (θ a , X i )

[0042] where a represents the action that the robot can take.

[0043] Subsequently, the algorithm selects the specific action to be executed by the current robot based on the output of the actor network using the ε-greedy strategy. The ε-greedy strategy combines the ideas of exploration and exploitation, selecting a random action with probability ε for exploration and selecting the action with the highest probability with probability 1-ε for exploitation. Mathematically, the action selection process can be represented as:

[0044]

[0045] where rand() is a function that generates a random number in the range [0, 1].

[0046] It is worth noting that the A3C model employs an asynchronous update mechanism, allowing multiple threads to process different robot states in parallel, thereby improving the efficiency of path planning. During the state input and action selection phases, the algorithm fully utilizes this characteristic of the A3C model to achieve efficient parallel path planning.

[0047] Through the state input and action selection phases, the algorithm achieves autonomous path planning for the robot. Each robot selects and executes corresponding actions based on its own state information and the output of the A3C model, gradually approaching the target position. Simultaneously, due to the asynchronous learning and optimization characteristics of the A3C model, the robot can optimize its path planning strategy through continuous trial and learning, improving the quality and efficiency of path planning.

[0048] In summary, this invention, by introducing the A3C model and the ε-greedy strategy, realizes autonomous path planning for robots based on state input and action selection, providing effective decision support for collaborative robot work.

[0049] IV. Environmental Interaction and Experience Collection

[0050] After completing the state input and action selection steps, the algorithm moves on to the environment interaction and experience collection phase, which is a core component of the reinforcement learning process. This phase involves the robot's action execution, environmental interaction, and the collection and storage of experience, aiming to optimize the path planning strategy using environmental feedback data.

[0051] First, the robot performs the selected action a. i Perform operations, such as changing direction of movement or adjusting speed. During execution, the robot continuously receives environmental feedback, including new states P. i ′ and reward value r i Environmental feedback can be formalized as state transition probabilities and a reward function, i.e., P(P i ′|P i ,a i ) and R(P i ,a i ).

[0052] Next, the algorithm stores the experience of this interaction in tuples (P). i ,a i ,r i ,P i The data is stored in the form of a dataset (') in the experience replay buffer D. The experience replay buffer is a fixed-size dataset used to store historical experience data. When the buffer is full, the algorithm replaces the old data using a first-in-first-out (FIFO) or other strategy.

[0053] The management and maintenance of the experience replay buffer are crucial, as they ensure that the algorithm can learn from diverse experience data. By randomly sampling from the buffer during training, the algorithm can break the correlation between data and improve the stability and efficiency of learning.

[0054] The learning part of the algorithm, particularly the training of the A3C model, will be based on this collected experience data. The A3C model continuously updates its parameters to better guide the robot's action selection by optimizing objective functions, such as maximizing expected returns.

[0055] Mathematically, the expected return can be represented as:

[0056]

[0057] where θ is the parameter of the A3C model, and γ is the discount factor, which weighs the importance of near-term rewards and long-term rewards.

[0058] Through the environment interaction and experience collection phase, the algorithm provides rich data support for the training of the A3C model, promoting the continuous optimization of the robot's path planning strategy. These experience data not only reflect the history of the robot's interaction with the environment, but also contain important information about the merits of actions, providing strong support for the learning and optimization of the algorithm.

[0059] In summary, the environment interaction and experience collection phase is a key link in achieving autonomous path planning for robots. By combining the robot's action execution and environmental feedback, this phase provides the necessary data foundation for reinforcement learning, promoting the continuous improvement and enhancement of the robot's path planning strategy.

[0060] Five, asynchronous update

[0061] After the robot has accumulated sufficient experience data through environmental interaction, the algorithm enters the asynchronous update phase. This phase takes full advantage of the asynchronous learning characteristics of the A3C model, significantly improving learning efficiency and algorithm performance through multi-thread parallel updates.

[0062] First, the algorithm randomly samples a batch of experience data from the experience replay buffer D in multiple threads, denoted as where k represents the sample index in the sampling batch.

[0063] Next, the algorithm uses the critic network Q(P,a; θ c ) to evaluate the value of this batch of experience. The critic network predicts the expected reward based on the current state P i k and the target state P i k+1 , providing value guidance for the actor network. The predicted value of the expected reward can be represented as:

[0064]

[0065] Then, the algorithm uses the actor network π(a|P; θ a ) to update the robot's policy. Based on the value guidance provided by the critic network, the actor network adjusts its parameters θ through methods such as gradient descent to maximize the expected reward. The objective function of policy update can be represented as: a

[0066]

[0067] where γ is the discount factor, used to weigh the importance of current rewards and future rewards.

[0068] Finally, the algorithm synchronizes the updated network parameters θ a and θ c asynchronously into other threads. Since the A3C model uses asynchronous updating, the learning progress of different threads is independent. Through parameter synchronization, the algorithm ensures that all threads use the latest and optimal policy for path planning.

[0069] During the asynchronous updating process, the algorithm avoids data race and deadlock problems through effective thread synchronization and communication mechanisms, ensuring smooth collaboration between threads.

[0070] Through the asynchronous updating phase, the algorithm efficiently utilizes experience data through parallel learning and policy optimization, continuously improving the robot's path planning performance. This phase provides strong support for robot collaboration, enabling robots to achieve efficient and accurate path planning in complex and variable environments.

[0071] In summary, the invention introduces the asynchronous updating mechanism of the A3C model, achieving efficient and stable robot path planning learning. Through multi-thread parallel sampling, evaluation and updating, the algorithm can fully utilize experience data, improve learning efficiency and algorithm performance, and provide effective decision support for robot collaboration.

[0072] Six, reward function calculation

[0073] In the process of robot path planning, the reward function as the guide of learning behavior is crucial. The reward function in the algorithm of the invention considers multiple factors such as reaching the target, avoiding collision and reaching the target as soon as possible. The following details the calculation method of the reward function.

[0074] First, define the reward function R goal ​When the robot successfully reaches the target position, a positive reward is given. The reward value can be adjusted according to the complexity and importance of the task, for example:

[0075] R goal = C goal

[0076] where C goal is a fixed positive constant.

[0077] Secondly, the reward function R collision for collision avoidance is defined. To encourage the robot to avoid collisions during path planning, a certain reward value is given when the robot successfully avoids collisions. The reward value can be set according to the degree of collision risk, for example, in the form of a function inversely proportional to the collision distance:

[0078]

[0079] where C collision is the constant factor of collision reward, d collision is the current distance between the robot and the obstacle or other robots. When the distance is larger, the reward value is higher; when the distance is smaller, the reward value is lower or even negative.

[0080] In addition, to encourage the robot to reach the target position as soon as possible, the reward function R time for reaching the target as soon as possible is defined. The reward function can be calculated according to the time required for the robot to reach the target position, for example, in the form of a function inversely proportional to the arrival time:

[0081]

[0082] where C time is the constant factor of time reward, t arrival is the time required for the robot to reach the target position. The shorter the arrival time, the higher the reward value.

[0083] By combining the above three reward functions, the total reward function R total is obtained:

[0084] R total = R goal + R collision + R time

[0085] In practical applications, the three reward functions can be adjusted in weight according to the specific task requirements to balance the influence between different factors. Weight adjustment can be achieved by adjusting the constant factors C goal , C collision and C time .

[0086] By reasonably designing the reward function, the algorithm of the present application can guide the robot to consider multiple factors in the path planning process, and achieve efficient and safe path selection. The calculation of the reward value will be directly used to guide the learning behavior of the robot, and through continuous iteration and optimization, the robot can gradually learn better path planning strategies. This reward mechanism ensures that the robot can make reasonable decisions when facing complex environments, thereby improving the accuracy and efficiency of path planning.

[0087] Seven, iterative optimization

[0088] In the first six steps of the algorithm, the robot has initially learned certain path planning strategies through interaction with the environment, collection of experience, asynchronous updating, and calculation of the reward function. However, in order to obtain the optimal path planning strategy, the robot needs to further try and learn through the process of iterative optimization.

[0089] In the iterative optimization phase, the algorithm repeatedly executes steps three to six, forming a closed-loop learning process. Specifically, the robot selects and executes an action a according to the current state information s, then observes the environmental feedback and collects new experience data. These experience data are then used in the asynchronous updating process to adjust the robot's strategy parameters through the collaborative work of the critic network and the actor network. At the same time, according to the performance of the robot in executing the action, the corresponding reward value r t is calculated to guide subsequent learning behavior.

[0090] As the iteration process proceeds, the robot's strategy π(a|s) gradually becomes optimized. The optimization process of the strategy can be represented as finding a strategy π* that maximizes the expected reward. The mathematical expression of the expected reward is:

[0091]

[0092] where γ is the discount factor, used to weigh the importance of current rewards and future rewards. Through iterative optimization, the algorithm continuously adjusts the strategy π, making it gradually approach the optimal strategy π*.

[0093] It is worth noting that the iterative optimization process may be affected by various factors such as environmental uncertainty, initial state, etc., and it may take some time to converge. In order to speed up the learning process, heuristic methods or prior knowledge may need to be combined. At the same time, in order to avoid falling into local optimal solutions, the algorithm also needs to balance exploration and utilization.

[0094] Through iterative optimization, the robot can continuously try and learn, and adjust its strategy based on experience data, eventually gradually learning the optimal path planning strategy. This process embodies the core part of the algorithm and is the key to achieving autonomous path planning of the robot.

[0095] In summary, through the iterative optimization process, the robot can continuously learn and optimize its path planning strategy, achieving efficient and safe path planning. This method not only improves the robot's autonomous decision-making ability, but also provides strong support for robot collaboration.

[0096] Eight, Testing and Evaluation

[0097] After completing the iterative optimization process, the algorithm enters the testing and evaluation stage to comprehensively verify the robot's path planning performance. This stage aims to ensure that the robot can exhibit efficient and safe path planning capabilities in real-world scenarios.

[0098] First, construct a test dataset containing diverse scenarios, denoted as D = (s i ,g i ,O i ), where s i represents the initial state, g i represents the target state, and O i represents the obstacle distribution. The dataset should cover a wide range of cases to fully test the robot's path planning capabilities.

[0099] In the testing environment, apply the trained model to the robot and record key performance indicators. The main indicators include:

[0100] Path length L: The actual path length taken by the robot from the starting point to the target point. A shorter path length indicates higher efficiency.

[0101] Time consumption T: The total time required for the robot to complete path planning and reach the target point. A shorter time consumption indicates that the robot can complete tasks faster.

[0102] Collision frequency C: The number of collisions the robot encounters with obstacles or other robots during path planning. A lower collision frequency indicates higher safety.

[0103] During the testing process, the robot needs to autonomously perform path planning without additional training. By conducting multiple tests and calculating the average values of the above indicators, the robot's path planning performance can be quantitatively evaluated.

[0104] In addition, to gain a deeper understanding of the model's performance, comparative experiments under different algorithms or parameter settings are conducted. By adjusting parameters such as model structure, learning rate, and reward function, observe the impact of these changes on performance indicators. Specifically, different parameter combinations can be set, and experiments can be conducted under the same test dataset to compare indicators such as path length, time consumption, and collision frequency.

[0105] Finally, according to the results of testing and evaluation, the optimal model is selected for practical application. The selected model should have stable performance in various scenarios and good generalization ability to cope with various changes that may occur in the actual environment.

[0106] In summary, through the testing and evaluation stage, the performance of the robot's path planning can be fully verified, and the optimal model can be selected for practical application. This process not only improves the robot's path planning ability, but also provides strong support for the widespread application of robots in actual scenarios.

[0107] In summary, the algorithm of the present application realizes the autonomous path planning of the robot through the steps of state input and action selection, environment interaction and experience collection, asynchronous update, reward function calculation and iterative optimization. In the testing and evaluation stage, the performance of the robot is fully evaluated, and the optimal model is selected for practical application. This algorithm has the characteristics of high efficiency, accuracy and stability, and is suitable for robot path planning tasks in various complex scenarios, providing strong support for robot collaborative work.

[0108] Compared with the prior art, the present application has the following advantages and effects:

[0109] The algorithm of the present application realizes the autonomous path planning of the robot through the steps of state input and action selection, environment interaction and experience collection, asynchronous update, reward function calculation and iterative optimization. In the testing and evaluation stage, the performance of the robot is fully evaluated, and the optimal model is selected for practical application. This algorithm has the characteristics of high efficiency, accuracy and stability, and is suitable for robot path planning tasks in various complex scenarios, providing strong support for robot collaborative work. BRIEF DESCRIPTION OF DRAWINGS

[0110] Figure 1 The flowchart of the multi-robot path planning method based on priority game and A3C algorithm of the present application. DETAILED DESCRIPTION

[0111] The present application will be further specifically and in detail described below in combination with specific embodiments.

[0112] The specific embodiments of the multi-robot path planning method based on priority game and A3C algorithm of the present application will be described in detail below.

[0113] I. Initialization

[0114] 1. Robot state initialization

[0115] For each robot, we need to initialize its state, including its initial position x, y coordinates in the environment, initial speed v x ,vy , initial direction θ, and target position x goal , y goal . These state information will serve as the basis for path planning.

[0116] Mathematically, it can be represented as: Robot state where i represents the number of robots.

[0117] 2. Priority setting

[0118] According to the importance and urgency of the task, each robot is assigned an initial priority. The priority can be calculated according to the task type, the distance between the current position of the robot and the target position, the density of environmental obstacles, and other factors.

[0119] Specifically, the priority can be calculated using the following formula: P i = w1·d i + w2·o i + w3·t i , where P i is the priority of robot i, d i is the distance from robot i to the target position, o i is the degree of influence of environmental obstacles on robot i, t i is the importance or urgency of the task, and w1, w2, w3 are the corresponding weight coefficients.

[0120] 3. A3C model initialization

[0121] Initialize the parameters of the A3C model, including the weights and biases of the actor network and the critic network. These parameters are the basis for model learning, and through continuous trial and error and optimization, they approach the optimal solution.

[0122] II. Priority game and path planning

[0123] In each time step, the algorithm performs the following steps:

[0124] 1. Update the priority list

[0125] According to the current state of the robot and the environmental information, the priority of each robot is recalculated and updated. This ensures that the priority list can reflect the changes in the environment and the state of the robot in real time.

[0126] 2. Path planning

[0127] According to the updated priority list, the robots plan their paths from high to low. For each robot, combining its current state (position, speed, direction) and environmental information (obstacle position, other robot state), use path planning algorithms (such as A* algorithm or RRT algorithm) to calculate the optimal path from the current position to the target position.

[0128] When planning the path, the algorithm will consider the positions and priorities of other robots to avoid path conflicts and collisions. For robots with higher priority, the algorithm will give them more path selection rights to ensure that they can complete their tasks first.

[0129] 3. A3C model learning and optimization

[0130] During path planning, the A3C model continuously interacts with the environment, learning how to make optimal decisions based on the current state and environmental information. The model optimizes its behavior strategy through trial and error and reward mechanisms, enabling the robot to better adapt to complex and changing environments and task requirements.

[0131] Specifically, the model calculates the loss function according to the difference between the actual behavior of the robot and the target behavior, and updates the network parameters through the backpropagation algorithm. At the same time, the model also gives appropriate rewards or punishments according to the performance of the robot to guide its learning behavior.

[0132] As described above, the present application can be well implemented.

[0133] The embodiments of the present application are not limited by the above examples, and any changes, modifications, substitutions, combinations, simplifications made without departing from the spirit and principles of the present application shall be equivalent replacement methods and shall be included in the protection scope of the present application.

Claims

1. A multi-robot path planning algorithm based on priority game and A3C, characterized in that Comprising S1, a robot state initialization step; S2, a priority setting step; S3, an A3C model parameter initialization step; S4, a priority game step, specifically, by setting a priority mechanism, it is ensured that robots with high priority can obtain path usage rights in priority, thereby reducing the risk of collision and conflict; At each time step, the algorithm first retrieves and updates the priority list of robots; the list is based on the importance of the robot's task W i , current position P i , target position G i is dynamically adjusted; the priority calculation formula is as follows: Priority i = a * W i + b * f(P i , G i ) + g (Environment i ); Wherein, α, β, γ are weight coefficients, f(P i , G i ) represents the functional relationship between the current position of the robot and the target position, g(Environment i ) represents the evaluation function of the environment where the robot is located; Subsequently, the algorithm plans paths for the robots in turn according to the order of priority from high to low; for each robot, the algorithm will combine its current state S i and environmental information E i , calculate the optimal path Path i from the current position to the target position through a path planning algorithm; In planning the path, the algorithm will pay special attention to the conflict problem between paths; if there is a potential conflict between the path of a high-priority robot and the path of a low-priority robot, the algorithm will start an avoidance strategy; this strategy adjusts the cost function C in the path planning algorithm i , so that the low-priority robot tends to avoid the high-priority robot when selecting a path; the adjustment of the cost function is represented as: C i′ = C i + λ * Overlap(Path high , Path i ); where, Overlap(Path high , Path i ) represents the degree of overlap between the high-priority robot path and the current robot path, and λ is an avoidance coefficient.

2. The multi-robot path planning algorithm based on priority game and A3C according to claim 1, wherein, The robot state initialization step in the step S1 is specifically as follows: Set initial state parameters for each robot, including position velocity orientation and goal position G i ; these state parameters will serve as the starting point for path planning and provide baseline data for subsequent calculations; mathematically, robot state initialization can be represented as: wherein, represents the initial state of the robot i; The priority setting step in S2 is as follows: in order to avoid collision and conflict, an initial priority is set for each robot The priority is based on the importance of the task W i , the current position The target position G i The factors are calculated comprehensively; the specific calculation formula is as follows: where σ and p are weight functions, is an evaluation function of the position and target position relationship; The A3C model parameter initialization step in the S3 is specifically as follows: as a core algorithm, the A3C model needs to set the parameters of the actor network and the critic network in the initialization stage; the actor network parameters include weights θ a and biases b a , the critic network parameters include weights θ c and biases b c ; at the same time, the learning rate η also needs to be set; mathematically, the A3C model parameter initialization can be expressed as: wherein, denotes the initial parameter set of the A3C model.

3. The multi-robot path planning algorithm based on priority game and A3C according to claim 2, characterized in that, After the S4 priority game step determines the path planning order of each robot, the algorithm proceeds to the state input and action selection phase; this phase is to input the current state of the robot into the A3C model, and select the corresponding action based on the model output, so as to realize the autonomous path planning of the robot, which is specifically as follows: The algorithm first passes the current robot's state information, including position P i , velocity V i , direction D i , and goal position G i , as input data X i to the actor network of the A3C model; the actor network generates a probability distribution of actions through a parameterized function f a (θ a , X i ), where θ a is the parameter of the actor network; The action probability distribution can be expressed as: π(a|X i ; θ a ) = f a (θ a , X i ); Wherein, a represents the action that the robot can take; Subsequently, the algorithm selects the specific action to be performed by the current robot according to the output of the actor network; the ε-greedy strategy combines the ideas of exploration and utilization, selects a random action with a probability of ε for exploration, and selects the action with the maximum probability with a probability of 1-ε for utilization; mathematically, the action selection process can be expressed as: Wherein, rand() is a function of generating a random number in the range of [0, 1].

4. The multi-robot path planning algorithm based on priority game and A3C according to claim 3, characterized in that, After completing the state input and action selection step, the algorithm proceeds to the environment interaction and experience collection phase, which is the core component of the reinforcement learning process; this phase involves the action execution, environment interaction, and experience collection and storage of the robot, aiming to optimize the path planning strategy by using environment feedback data; specifically as follows: First, the robot performs the selected action a. i Perform operations, such as changing direction of movement or adjusting speed; during execution, the robot continuously receives environmental feedback, including new state P. i ′ and reward value r i Environmental feedback can be formalized as state transition probabilities and a reward function, i.e., P(P i ′|P i ,a i ) and R(P i ,a i ); Next, the algorithm stores the experience of this interaction in the form of a tuple (P i ,a i ,r i ,P i ′) to an experience replay buffer D; the experience replay buffer is a fixed-size dataset that stores historical experience data; when the buffer is full, the algorithm replaces old data using a first-in-first-out or other strategy; The A3C model updates its parameters by optimizing the objective function to maximize the expected return, so as to better guide the action selection of the robot; Mathematically, the expected return can be expressed as: Wherein, θ is the parameter of the A3C model, and γ is the discount factor, which is used to weigh the importance of recent rewards and future rewards.

5. The multi-robot path planning algorithm based on priority game and A3C according to claim 4, characterized in that, After the robot accumulates sufficient experience data through the environment interaction and experience collection phase, the algorithm enters the asynchronous update phase; this phase uses the asynchronous learning characteristics of the A3C model to update through multi-threading in parallel, so as to improve the learning efficiency and algorithm performance; specifically as follows: First, the algorithm randomly samples a batch of experience data from the experience replay buffer D in parallel on multiple threads, denoted as where k denotes the sample index in the sampled batch. Next, the algorithm evaluates the value of this batch of experiences using the critic network Q(P, a; θ c ); the critic network predicts the expected reward given the current state and the target state , providing value guidance to the actor network; the predicted value of the expected reward can be expressed as: Then, the algorithm updates the robot's policy using an actor network π(a|P; θ a ) provided by the critic network The actor network adjusts its parameters θ a by gradient descent and the like to maximize the expected reward; the objective function for policy update can be expressed as: Wherein, γ is the discount factor, which is used to weigh the importance of current rewards and future rewards; Finally, the algorithm will update the network parameters θ a and θ c asynchronously to other threads.

6. The multi-robot path planning algorithm based on priority game and A3C according to claim 5, characterized in that, After the robot completes the asynchronous update phase, it enters the reward function calculation step; during the path planning process of the robot, the reward function serves as the guide for learning behavior, which is specifically as follows: First, define the reward function R for reaching the goal goal A positive reward is given when the robot successfully reaches the goal location; the reward value can be adjusted according to the complexity and importance of the task, if: R goal = C goal ; where C goal is a fixed positive constant; Secondly, the reward function R for avoiding collision is defined collision In order to encourage the robot to avoid collision in the process of path planning, a certain reward value is given when the robot successfully avoids collision; the reward value can be set according to the degree of collision risk, and a function form inversely proportional to the collision distance can be adopted: where C collision is a constant factor of collision reward, d collision is the current distance between the robot and the obstacle or other robot; when the distance is larger, the reward value is higher; when the distance is smaller, the reward value is lower or even negative; To encourage the robot to reach the goal position as quickly as possible, a reward function R is defined that rewards reaching the goal as quickly as possible time ; this reward function can be calculated based on the time it takes for the robot to reach the goal position, assuming a function that is inversely proportional to the time of arrival: where C time is a constant factor of the time reward, t arrival is the time taken by the robot to reach the target position; the shorter the time to reach, the higher the reward value; Combining the above three aspects of the reward function, the total reward function R is obtained total : R total = R goal + R collision + R time .

7. The multi-robot path planning algorithm based on priority game and A3C according to claim 6, characterized in that, After the robot completes the reward function calculation step, in order to obtain the optimal path planning strategy, the robot needs to further try and learn through the process of iterative optimization; In the iterative optimization phase, the algorithm repeatedly executes the state input and action selection, environment interaction and experience collection, asynchronous update, and reward function calculation, forming a closed-loop learning process; specifically as follows: The robot selects and executes an action a according to current state information s, and then observes environmental feedback to collect new experience data; the experience data are then used in an asynchronous updating process to adjust the policy parameters of the robot through the collaborative work of the critic network and the actor network; at the same time, a corresponding reward value r is calculated according to the performance of the robot in executing the action, which is used to guide subsequent learning behavior t . With the progress of the iteration process, the strategy of the robot π(a|s) is gradually optimized; the optimization process of the strategy can be expressed as finding the strategy π* that can maximize the expected reward; the mathematical expression of the expected reward is: where γ is a discount factor that weighs the importance of current rewards versus future rewards; through iterative optimization, the algorithm continuously adjusts the policy π to gradually approximate the optimal policy π*.

8. The multi-robot path planning algorithm based on priority game and A3C according to claim 7, characterized in that, After completing the iterative optimization process, the algorithm enters the testing and evaluation phase to comprehensively verify the robot's path planning performance; this phase aims to ensure that the robot can exhibit efficient and safe path planning capabilities in real-world scenarios; specifically as follows: First, construct a test dataset containing diverse scenarios, denoted as D = (s i ,g i ,O i ), where s i represents the initial state, g i represents the goal state, and O i represents the obstacle distribution; The dataset should cover a wide range of situations to fully test the robot's path planning capabilities; In the testing environment, apply the trained model to the robot and record key performance indicators.

9. The multi-robot path planning algorithm based on priority game and A3C according to claim 8, characterized in that The key performance indicators include: Path length L: the actual path length traveled by the robot from the starting point to the target point; Time consumption T: the total time required for the robot to complete path planning and reach the target point; Collision times C: the number of times the robot collides with obstacles or other robots during path planning; During the testing process, the robot needs to autonomously perform path planning without additional training.

Images