A global path coordination method and system for multiple mobile robots and a storage medium

Abstract

The present application relates to the technical field of mobile robot intelligent control, in particular to a kind of global path coordination method, system and storage medium of multiple mobile robots, comprising: constructing simulation environment and kinematics model of mobile robot, design path planning model of multiple mobile robots;Reward function is designed based on the obstacle avoidance planning layer and relative position coordination layer in double-layer neural network respectively, for training the path planning model of multiple mobile robots;Based on the path planning model of multiple mobile robots that complete training, global path planning of multiple mobile robots is carried out.The present application solves the problem of dimension explosion and difficulty in convergence in multi-robot navigation of deep reinforcement learning, can effectively adapt to the demand of multiple mobile robots autonomous navigation in complex and irregular environment, improve the efficiency and accuracy of global path planning of multiple mobile robots.

Description

Technical Field

[0001] This invention relates to the field of intelligent control technology for mobile robots, and in particular to a global path coordination method, system, and storage medium for multiple mobile robots. Background Technology

[0002] Currently, mobile robots are widely used in various industries, such as factories, hospitals, homes, hotels, exhibition halls, and restaurants, mainly undertaking tasks such as logistics, handling, and delivery. In these application scenarios, multiple mobile robots need to achieve autonomous navigation, possessing the ability to accurately locate, plan paths, and complete tasks efficiently and safely.

[0003] In multi-robot operation scenarios, a global map of the surrounding environment is typically built in advance, and safe movement from the starting point to the destination is achieved according to task requirements. However, relying solely on global path planning algorithms suitable for single-robot operation scenarios, while finding the shortest or most time-saving path, cannot account for multiple robots. This can lead to difficulties in mobile robot movement or even deadlocks in narrow passages or complex environments such as intersections. Furthermore, when deep reinforcement learning models are applied to multi-robot navigation, the large number of input state parameters from multiple robots can cause problems such as dimensionality explosion and difficulty in convergence. Summary of the Invention

[0004] Therefore, the technical problem to be solved by the present invention is to overcome the problems of dimensionality explosion and difficulty in convergence encountered by the application of deep reinforcement learning models in multi-robot navigation in the prior art.

[0005] To address the aforementioned technical problems, this invention provides a global path coordination method for multiple mobile robots, comprising:

[0006] A simulation environment and a kinematic model of a mobile robot are constructed, and a path planning model for multiple mobile robots is designed. The path planning model for multiple mobile robots includes a two-layer neural network and a reinforcement learning network. The two-layer neural network includes an obstacle avoidance planning layer and a relative position coordination layer.

[0007] Reward functions were designed based on the obstacle avoidance planning layer and the relative position coordination layer in a two-layer neural network, respectively, to train the path planning model of a multi-mobile robot.

[0008] Global path planning for multiple mobile robots is performed based on a trained multi-mobile robot path planning model, including:

[0009] The observation space and motion space of each mobile robot are obtained; the observation space of each mobile robot is input into the two-layer neural network corresponding to the current mobile robot; the obstacle avoidance planning layer takes the observation space as input and outputs obstacle avoidance feature values; the relative position coordination layer obtains the relative position information of each mobile robot with other robots based on the observation space and motion space as input and outputs relative coordination feature values; the obstacle avoidance feature values ​​and the relative coordination feature values ​​are concatenated to obtain the state space of the mobile robot.

[0010] The state space of each mobile robot is input into the reinforcement learning network corresponding to the current mobile robot, and the action of each mobile robot is output.

[0011] Preferably, the observation space of the mobile robot includes the current position of the robot, the local point that the robot needs to reach, and the surrounding environment information obtained by the robot through sensors; the motion space includes linear velocity and angular velocity.

[0012] Preferably, the structure of both the obstacle avoidance planning layer and the relative position coordination layer includes: three fully connected layers, a long short-term memory layer and an encoding layer connected in sequence.

[0013] Preferably, the reward function designed based on the obstacle avoidance planning layer is:

[0014]

[0015] Among them, R OA Let V represent the reward function of the obstacle avoidance planning layer, V represent the linear velocity of the mobile robot, and W represent the angular velocity of the mobile robot.

[0016] Preferably, the reward function designed based on the relative position coordination layer is:

[0017]

[0018] Among them, R NAV Let K represent the reward function of the relative position coordination layer, and d represent the collision coefficient. curr d represents the distance between the current mobile robot and the local point that needs to be reached. step This indicates the current time step the robot is executing.

[0019] Preferably, the reinforcement learning network is a MATD3 algorithm network; the MATD3 algorithm network includes an experience replay pool for storing all information about the interaction between the mobile robot and the environment, which serves as samples for training the network; the information about the interaction between the mobile robot and the environment stored in the experience replay pool includes: the mobile robot's observation space and action space, actions, the state space at the next moment, and the reward value after performing the action.

[0020] Preferably, the MATD3 algorithm network includes:

[0021] Each mobile robot corresponds to a set of independent policy-evaluation networks, including one policy network, one target policy network, two evaluation networks, and two target evaluation networks.

[0022] Each mobile robot's policy network selects multiple actions based on the current state space of the mobile robot; the GAE algorithm is used to search for all actions, and the evaluation network calculates the value of the actions based on the state spaces of all mobile robots, outputting the action with the highest value;

[0023] After each mobile robot performs an action and interacts with the environment, it obtains the state space for the next moment and generates a reward value for the action.

[0024] Preferably, the parameter update method of the MATD3 algorithm network includes:

[0025] Select several samples from the experience replay pool to train the MATD3 algorithm network corresponding to each mobile robot;

[0026] Each mobile robot's target policy network outputs the action for the next moment based on the state space of the next moment, and feeds it into two target evaluation networks respectively; the target evaluation network calculates the value of the action for the next moment based on the state space of all mobile robots; selects the minimum value as the target action value; calculates the loss function based on the target action value, and updates the parameters of the evaluation network;

[0027] The network parameters of the target evaluation network are softly updated based on the updated evaluation network parameters.

[0028] The present invention also provides a global path coordination system for multiple mobile robots, comprising:

[0029] The model building module is used to construct the kinematic model of the simulation environment and the mobile robot, and to design the path planning model of multiple mobile robots. The path planning model of multiple mobile robots includes a two-layer neural network and a reinforcement learning network. The two-layer neural network includes an obstacle avoidance planning layer and a relative position coordination layer.

[0030] The reward function design module is used to design the reward functions for the obstacle avoidance planning layer and the relative position coordination layer in the two-layer neural network, respectively, for training the path planning model of multi-mobile robots.

[0031] The path planning module is used for global path planning of multiple mobile robots based on the trained multi-mobile robot path planning model, including:

[0032] The observation space and motion space of each mobile robot are obtained; the observation space of each mobile robot is input into the two-layer neural network corresponding to the current mobile robot; the obstacle avoidance planning layer takes the observation space as input and outputs obstacle avoidance feature values; the relative position coordination layer obtains the relative position information of each mobile robot with other robots based on the observation space and motion space as input and outputs relative coordination feature values; the obstacle avoidance feature values ​​and the relative coordination feature values ​​are concatenated to obtain the state space of the mobile robot.

[0033] The state space of each mobile robot is input into the reinforcement learning network corresponding to the current mobile robot, and the action of each mobile robot is output.

[0034] The present invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the above-described global path coordination method for multiple mobile robots.

[0035] Compared with the prior art, the above-described technical solution of the present invention has the following advantages:

[0036] This invention discloses a global path coordination method for multiple mobile robots. It extracts features from the observation space of the mobile robots by constructing a two-layer neural network to generate their state space, reducing the input dimensionality of the reinforcement learning network. Furthermore, it designs independent reward functions for the obstacle avoidance planning layer and the relative position coordination layer of the two-layer neural network to assist in the training of the multi-mobile robot path planning model and improve the convergence of the model training. This invention solves the problems of dimensionality explosion and difficulty in convergence of deep reinforcement learning in multi-robot navigation. It can effectively adapt to the needs of autonomous navigation of multiple mobile robots in complex and irregular environments, and has high safety, good orderliness, and strong real-time performance. It reduces the collision and blockage problems that may occur in the actual operation of multiple mobile robots, and improves the efficiency and accuracy of global path planning for multiple mobile robots. Attached Figure Description

[0037] To make the content of this invention easier to understand, the invention will be further described in detail below with reference to specific embodiments and accompanying drawings, wherein:

[0038] Figure 1 This is a flowchart of a global path coordination method for multiple mobile robots according to the present invention;

[0039] Figure 2 This is a diagram of a two-layer neural network structure for a single robot;

[0040] Figure 3 This is a training structure diagram of the MATD3 algorithm network. Detailed Implementation

[0041] The present invention will be further described below with reference to the accompanying drawings and specific embodiments, so that those skilled in the art can better understand and implement the present invention. However, the embodiments described are not intended to limit the present invention.

[0042] Reference Figure 1 As shown, the present invention provides a global path coordination method for multiple mobile robots, comprising:

[0043] S1. Construct a simulation environment and a kinematic model of the mobile robot, and design a path planning model for multiple mobile robots; the path planning model for multiple mobile robots includes a two-layer neural network and a reinforcement learning network; the two-layer neural network includes an obstacle avoidance planning layer and a relative position coordination layer;

[0044] S2. Design reward functions for the obstacle avoidance planning layer and the relative position coordination layer in the two-layer neural network respectively, and use them to train the path planning model of the multi-mobile robot.

[0045] S3. Based on the trained multi-mobile robot path planning model, perform global path planning for the multi-mobile robots, including:

[0046] The observation space of each mobile robot is obtained; the observation space of each mobile robot is input into the two-layer neural network corresponding to the current mobile robot; the obstacle avoidance planning layer takes the observation space as input and outputs obstacle avoidance feature values; the relative position coordination layer obtains the relative position information of each mobile robot with other robots based on the observation space as input and outputs relative coordination feature values; the obstacle avoidance feature values ​​and the relative coordination feature values ​​are concatenated to obtain the state space of the mobile robot.

[0047] The state space of each mobile robot is input into the reinforcement learning network corresponding to the current mobile robot, and the action of each mobile robot is output.

[0048] The technical solutions in the embodiments of the present invention will be clearly and completely described below.

[0049] When multiple mobile robots navigate in complex environments, each robot uses its onboard 2D laser sensor to perceive its surroundings in real time, and each robot can directly obtain its relative position information with other mobile robots. Each mobile robot needs to know its own position and the local point it needs to reach.

[0050] Construct a kinematic model of the simulation environment and the mobile robot, and design a path planning model for multiple mobile robots based on the kinematic model of the mobile robot. Collect point cloud data of the simulation or real environment using LiDAR equipment to perceive the obstacle avoidance environment of multiple robots in real time, set up a reinforcement learning framework, and define the observation space and action space of each mobile robot.

[0051] The observation space of a mobile robot includes the robot's current position, the local points the robot needs to reach, and the surrounding environment information acquired by the robot through sensors. The action space of a mobile robot includes linear velocity and angular velocity.

[0052] In indoor navigation, to avoid the problem of model non-convergence caused by the increase in the dimensionality of the network model input, the surrounding environment information obtained by each mobile robot through sensors is radar point cloud data of the current mobile robot from -90° to 90°.

[0053] The path planning model for the multi-mobile robot includes a two-layer neural network and a reinforcement learning network. The two-layer neural network includes an obstacle avoidance planning layer and a relative position coordination layer.

[0054] The obstacle avoidance planning layer and the relative position coordination layer have the same neural network architecture, both including three fully connected layers, a long short-term memory layer and an encoding layer connected in sequence, outputting corresponding feature values ​​for subsequent splicing.

[0055] The observation space of each mobile robot is input into a two-layer neural network corresponding to the current mobile robot. The obstacle avoidance planning layer takes the observation space as input and outputs obstacle avoidance feature values. The relative position coordination layer introduces a signal strength method, obtaining the relative position information of each mobile robot with other robots based on the observation space as input. That is, taking each robot as the center, it inputs the distance and direction of all other robots relative to itself according to the distance from nearest to farthest, and outputs relative coordination feature values. The obstacle avoidance feature values ​​and relative coordination feature values ​​are concatenated to obtain the state space of the mobile robot. The two-layer neural network structure of a single robot is shown below. Figure 2 As shown.

[0056] This invention designs reward functions for the obstacle avoidance planning layer and the relative position coordination layer in a two-layer neural network, respectively, for training the two-layer neural network.

[0057] The obstacle avoidance planning layer is divided into the driving process and the collision process. During the driving process, the absolute value of the linear velocity minus the angular velocity is calculated to ensure the robot's stability. When the mobile robot collides, a higher penalty of -100 is applied to distinguish the collision. The reward function of the obstacle avoidance planning layer is:

[0058]

[0059] Among them, R OA Let V represent the reward function of the obstacle avoidance planning layer, V represent the linear velocity of the mobile robot, and W represent the angular velocity of the mobile robot.

[0060] The relative position coordination layer is divided into reaching the target point and other scenarios. A higher reward of 100 is given when the mobile robot reaches the target point. Otherwise, the more steps used to solve the coordination problem between multiple mobile robots, the greater the penalty value. The reward function for the relative position coordination layer is:

[0061]

[0062] Among them, R NAV The reward function for the relative position coordination layer is denoted by K, which represents the collision coefficient used to distinguish between arrival and collision. In this embodiment, the value is 0.1. curr d represents the distance between the current mobile robot and the local point that needs to be reached. step This indicates the current time step the robot is executing.

[0063] Therefore, the reward function R of the two-layer neural network is:

[0064] R = R OA +R NAV

[0065] The state space of each mobile robot is input into the reinforcement learning network corresponding to the current mobile robot, and the action of each mobile robot is output.

[0066] This invention extracts features from the observation space and action space through a two-layer neural network and concatenates them into a state space that can adapt to the multi-agent training of the MATD3 algorithm network, reducing the dimensionality of the input. Furthermore, corresponding reward functions are designed independently for these two neural network layers to assist model training and improve the convergence of model training. Through the above two-layer neural network design, the model can effectively adapt to the needs of autonomous navigation of multiple robots in complex and irregular environments.

[0067] The reinforcement learning network used in this invention is the MATD3 algorithm network.

[0068] After each robot interacts with the environment, all information about the interaction between the mobile robots and the environment is stored in the experience replay pool, including the observation space and action space of the mobile robot, the action, the state space of the next moment, and the reward value after the action is performed, as samples for training the reinforcement learning network.

[0069] In this embodiment, the experience replay pool is a fixed-size list. A historical training data storage H is designed to store the experience sets of h buffers, forming a minibatch dataset, represented as follows. <O set A set ,R set ,S set >, where O setThis represents the observation and motion space of the mobile robot, including the raw data set consisting of the point cloud data, linear velocity, angular velocity, and pose of the three most recent frames from the 2D LiDAR of all mobile robots; S set A represents the state space of all mobile robots at the next moment; set This indicates that all mobile robots are in S set The actions performed in the state space, i.e., the actions output by the reinforcement learning network; R set This indicates that all mobile robots are performing action A. set The set of reward values ​​obtained.

[0070] Refer to the training structure diagram of the MATD3 algorithm network. Figure 3 As shown, the MATD3 algorithm network is a multi-agent reinforcement learning algorithm based on the Actor-Critic architecture, where each agent has its own independent policy-evaluation network. The MATD3 algorithm network employs a centralized training and distributed execution structure. During training, each agent's evaluation network can access the states and actions of all agents to maximize the cumulative reward for each agent. During the execution phase, the agent selects actions solely based on its own observations.

[0071] In this embodiment, the MATD3 algorithm network includes:

[0072] Each mobile robot corresponds to a set of independent policy-evaluation networks, including one policy network, one target policy network, two evaluation networks, and two target evaluation networks.

[0073] Each mobile robot's policy network selects multiple actions based on the current state space of the mobile robot; the GAE algorithm is used to search for all actions, and the evaluation network calculates the value of the actions based on the state spaces of all mobile robots, outputting the action with the highest value;

[0074] After each mobile robot performs an action and interacts with the environment, it obtains the state space for the next moment and generates a reward value for the action.

[0075] The formula for the advantage function calculated by using the GAE algorithm to search for all actions is as follows:

[0076]

[0077] in, T represents the action with the highest value for the i-th mobile robot at time t. i Let represent the time point of the i-th mobile robot, with a value ranging from 0 to T; l represents the l-th time; λ and γ both represent scaling factors. This represents the scaling factor for the i-th mobile robot at time t. and Let represent the state spaces of the i-th mobile robot at time t+1 and time t, respectively. and Let represent the value of the i-th mobile robot action at time t and time t+1, respectively.

[0078] In the above formula, T i This represents a time span of experience learned during the training and learning process. l represents the time span T. i The accumulation within the sequence aims to achieve the action at time t that maximizes the final reward during the sequence. t is time t in the mobile robot's operation, equivalent to real-time. The action at time t depends on the optimal action from previous training, i.e., from time T... i Choose from the options.

[0079] The advantage function can be used to calculate the most valuable action based on the state space of all mobile robots.

[0080] The parameter update method of the MATD3 algorithm network includes:

[0081] Select several samples from the experience replay pool to train the MATD3 algorithm network corresponding to each mobile robot;

[0082] Each mobile robot's target policy network outputs the action for the next moment based on the state space of the next moment, and feeds it into two target evaluation networks respectively; the target evaluation network calculates the value of the action for the next moment based on the state space of all mobile robots; selects the minimum value as the target action value; calculates the loss function based on the target action value, and updates the parameters of the evaluation network;

[0083] The network parameters of the target evaluation network are soft-updated based on the updated evaluation network parameters.

[0084] Finally, repeat the above steps until you learn the optimal strategy.

[0085] This invention is based on the MATD3 algorithm network and designs an experience pool replay mechanism to store historical data, thereby improving training efficiency. Finally, it determines whether to complete model training and outputs the final result by comprehensively judging the number of iterations, average reward value and loss function.

[0086] In summary, the global path coordination method for multiple mobile robots described in this invention extracts features from the observation space of the mobile robots by constructing a two-layer neural network to generate the state space of the mobile robots, thus reducing the input dimensionality of the reinforcement learning network. Furthermore, for the two-layer neural network, independent reward functions are designed for the obstacle avoidance planning layer and the relative position coordination layer, respectively, to assist in the training of the path planning model for multiple mobile robots and improve the convergence of the model training. This invention solves the problems of dimensionality explosion and difficulty in convergence of deep reinforcement learning in multi-robot navigation, effectively adapting to the needs of autonomous navigation of multiple mobile robots in complex and irregular environments. It has high safety, good orderliness, and strong real-time performance, reducing the collision and blockage problems that may occur in the actual operation of multiple mobile robots, and improving the efficiency and accuracy of global path planning for multiple mobile robots.

[0087] Based on the above-described global path coordination method for multiple mobile robots, this embodiment also provides a global path coordination system for multiple mobile robots, including:

[0088] The model building module is used to construct the kinematic model of the simulation environment and the mobile robot, and to design the path planning model of multiple mobile robots. The path planning model of multiple mobile robots includes a two-layer neural network and a reinforcement learning network. The two-layer neural network includes an obstacle avoidance planning layer and a relative position coordination layer.

[0089] The reward function design module is used to design the reward functions for the obstacle avoidance planning layer and the relative position coordination layer in the two-layer neural network, respectively, for training the path planning model of multi-mobile robots.

[0090] The path planning module is used for global path planning of multiple mobile robots based on the trained multi-mobile robot path planning model, including:

[0091] The observation space and motion space of each mobile robot are obtained; the observation space of each mobile robot is input into the two-layer neural network corresponding to the current mobile robot; the obstacle avoidance planning layer takes the observation space as input and outputs obstacle avoidance feature values; the relative position coordination layer obtains the relative position information of each mobile robot with other robots based on the observation space and motion space as input and outputs relative coordination feature values; the obstacle avoidance feature values ​​and the relative coordination feature values ​​are concatenated to obtain the state space of the mobile robot.

[0092] The state space of each mobile robot is input into the reinforcement learning network corresponding to the current mobile robot, and the action of each mobile robot is output.

[0093] This embodiment also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the global path coordination method for multiple mobile robots described above.

[0094] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0095] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0096] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0097] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0098] Obviously, the above embodiments are merely illustrative examples for clear explanation and are not intended to limit the implementation. Those skilled in the art will recognize that other variations or modifications can be made based on the above description. It is neither necessary nor possible to exhaustively list all possible implementations here. However, obvious variations or modifications derived therefrom are still within the scope of protection of this invention.

Claims

1. A global path coordination method for multiple mobile robots, characterized in that, include: A simulation environment and a kinematic model of a mobile robot are constructed, and a path planning model for multiple mobile robots is designed. The path planning model for multiple mobile robots includes a two-layer neural network and a reinforcement learning network. The two-layer neural network includes an obstacle avoidance planning layer and a relative position coordination layer. Reward functions were designed based on the obstacle avoidance planning layer and the relative position coordination layer in a two-layer neural network, respectively, to train the path planning model of a multi-mobile robot. Global path planning for multiple mobile robots is performed based on a trained multi-mobile robot path planning model, including: The observation space and motion space of each mobile robot are obtained; the observation space of each mobile robot is input into the two-layer neural network corresponding to the current mobile robot; the obstacle avoidance planning layer takes the observation space as input and outputs obstacle avoidance feature values; the relative position coordination layer obtains the relative position information of each mobile robot with other robots based on the observation space and motion space as input and outputs relative coordination feature values; the obstacle avoidance feature values ​​and the relative coordination feature values ​​are concatenated to obtain the state space of the mobile robot. The state space of each mobile robot is input into the reinforcement learning network corresponding to the current mobile robot, and the action of each mobile robot is output.

2. The global path coordination method for multiple mobile robots according to claim 1, characterized in that, The observation space of a mobile robot includes the robot's current position, the local points that the robot needs to reach, and the surrounding environment information obtained by the robot through sensors; the motion space includes linear velocity and angular velocity.

3. The global path coordination method for multiple mobile robots according to claim 1, characterized in that, The structures of the obstacle avoidance planning layer and the relative position coordination layer both include: three fully connected layers, a long short-term memory layer, and an encoding layer connected in sequence.

4. The global path coordination method for multiple mobile robots according to claim 1, characterized in that, The reward function based on the obstacle avoidance planning layer is: Among them, R OA Let V represent the reward function of the obstacle avoidance planning layer, V represent the linear velocity of the mobile robot, and W represent the angular velocity of the mobile robot.

5. The global path coordination method for multiple mobile robots according to claim 1, characterized in that, The reward function designed based on the relative position coordination layer is: Among them, R NAV Let K represent the reward function of the relative position coordination layer, and d represent the collision coefficient. curr d represents the distance between the current mobile robot and the local point that needs to be reached. step This indicates the current time step the robot is executing.

6. The global path coordination method for multiple mobile robots according to claim 1, characterized in that, The reinforcement learning network is the MATD3 algorithm network; the MATD3 algorithm network includes an experience replay pool, which is used to store all information about the interaction between the mobile robot and the environment, as samples for training the network; The experience replay pool stores information about the mobile robot's interaction with the environment, including: the mobile robot's observation space and action space, actions, the state space at the next moment, and the reward value after performing the action.

7. A global path coordination method for multiple mobile robots according to claim 6, characterized in that, The MATD3 algorithm network includes: Each mobile robot corresponds to a set of independent policy-evaluation networks, including one policy network, one target policy network, two evaluation networks, and two target evaluation networks. Each mobile robot's policy network selects multiple actions based on the current state space of the mobile robot; the GAE algorithm is used to search for all actions, and the evaluation network calculates the value of the actions based on the state spaces of all mobile robots, outputting the action with the highest value; After each mobile robot performs an action and interacts with the environment, it obtains the state space for the next moment and generates a reward value for the action.

8. A global path coordination method for multiple mobile robots according to claim 7, characterized in that, The parameter update method of the MATD3 algorithm network includes: Select several samples from the experience replay pool to train the MATD3 algorithm network corresponding to each mobile robot; Each mobile robot's target policy network outputs the action for the next moment based on the state space of the next moment, and feeds it into two target evaluation networks respectively; the target evaluation network calculates the value of the action for the next moment based on the state space of all mobile robots; selects the minimum value as the target action value; calculates the loss function based on the target action value, and updates the parameters of the evaluation network; The network parameters of the target evaluation network are softly updated based on the updated evaluation network parameters.

9. A global path coordination system for multiple mobile robots, characterized in that, include: The model building module is used to construct the kinematic model of the simulation environment and the mobile robot, and to design the path planning model of multiple mobile robots. The path planning model of multiple mobile robots includes a two-layer neural network and a reinforcement learning network. The two-layer neural network includes an obstacle avoidance planning layer and a relative position coordination layer. The reward function design module is used to design the reward functions for the obstacle avoidance planning layer and the relative position coordination layer in the two-layer neural network, respectively, for training the path planning model of multi-mobile robots. The path planning module is used for global path planning of multiple mobile robots based on the trained multi-mobile robot path planning model, including: The observation space and motion space of each mobile robot are obtained; the observation space of each mobile robot is input into the two-layer neural network corresponding to the current mobile robot; the obstacle avoidance planning layer takes the observation space as input and outputs obstacle avoidance feature values; the relative position coordination layer obtains the relative position information of each mobile robot with other robots based on the observation space and motion space as input and outputs relative coordination feature values; the obstacle avoidance feature values ​​and the relative coordination feature values ​​are concatenated to obtain the state space of the mobile robot. The state space of each mobile robot is input into the reinforcement learning network corresponding to the current mobile robot, and the action of each mobile robot is output.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the global path coordination method for multiple mobile robots as described in any one of claims 1 to 8.

Images