A multi-robot trajectory planning method

Abstract

The present invention discloses a multi-robot trajectory planning method, comprising the following steps: deep Q learning uses the state of the environment around the multi-robots to analyze the current trajectory vector, designs a reward network of deep Q learning, uses the current trajectory vector and the expected trajectory vector as inputs of the reward network, outputs as reward information, and is used to train parameters of a convolutional neural network (CNN); uses the current trajectory vector as input of the convolutional neural network (CNN), and the convolutional neural network (CNN) trained based on the reward information uses a convolutional neural network (CNN) algorithm to output corresponding action information to environmental information; and then uses a resource-based multi-robot task allocation algorithm to reasonably allocate all actions of workpieces to multiple robots, so that multiple robots can cooperate with each other without interference, realize spatial three-dimensional complex trajectory planning of multiple robots, and thus realize the high efficiency of robots collaboratively executing complex tasks.

Description

Technical Field

[0001] The present invention relates to the technical field of multi-robot collaborative control, and in particular to a multi-robot trajectory planning method. Background Art

[0002] With the continuous changes in the processing volume and working environment of complex component industries such as steel and aluminum, some tasks are difficult to be undertaken by a single robot alone and require the coordination and cooperation of multiple robots to complete. Multi-machine collaborative operation has replaced single machines and become a research hotspot for building intelligent production lines. Compared with single robot systems, multi-robot systems have strong environmental adaptability, high self-regulation ability, wide system spatial distribution, better data redundancy, and robustness. The collaborative cooperation between multiple robots can reliably complete high-precision operations and efficient processing that single robots cannot complete.

[0003] Welding is a potentially dangerous, intensive, and highly skilled job. Traditional robotic welding processes mostly use manual teaching to generate welding trajectories, which is not only time-consuming and labor-intensive, but also has low precision. In addition, it is difficult to achieve three-dimensional arbitrary complex curve welding of complex components due to the limited working space of the robot. There is an urgent need for collaborative work between multiple robots, that is, in the same workstation area, multiple robots collaborate to clamp, carry, flip, and weld the workpiece (i.e., the object to be welded) to achieve mutual cooperation and achieve punctuality, synchronization, coordination, and efficiency.

[0004] When multiple robots overlap in their workspaces, it is difficult to plan the collaborative trajectory of each robot. It is difficult to obtain the optimal solution using traditional spatial trajectory optimization methods. For complex welding tasks, the spatial three-dimensional complex trajectory planning of robot welding must not only ensure that the multi-robot system does not conflict with obstacles in the environment, but also ensure that each robot maintains a given position, especially when the robots have highly overlapping workspaces.

[0005] Existing multi-robot collaboration schemes are highly dependent on the position and speed of each robot. Traditional trajectory planning methods are difficult to adapt to complex and dynamic systems and environments because each robot has difficulty identifying the robots around it as obstacles or collaborative robots. Although machine learning has been applied to robot control and path planning, most research is limited to simulation. Genetic algorithms also have some limitations that need to be addressed. Research on trajectory planning for multi-robot reinforcement learning applied to solve one or more tasks is relatively inactive.

[0006] When multiple robots have highly overlapping workspaces, they need to cooperate with each other in the same workstation area to achieve operations such as clamping, handling, flipping and welding of workpieces. Each robot in the multi-robot system must act independently and cooperate with other robots. The multi-robot collaboration scheme is highly dependent on the position and speed of each robot. How to achieve the efficiency of robots to perform complex tasks in a coordinated manner and meet the requirements of three-dimensional complex trajectory planning is the key issue to be solved. Summary of the invention

[0007] In view of this, in order to solve the above problems in the prior art, the present invention proposes a multi-robot trajectory planning method, which integrates deep Q learning and convolutional neural network (CNN) algorithms, so that multiple robots can cooperate with each other without interference, thereby realizing spatial three-dimensional complex trajectory planning of multiple robots.

[0008] The present invention solves the above problems by the following technical means:

[0009] A multi-robot trajectory planning method comprises the following steps:

[0010] Deep Q learning uses the state of the environment around multiple robots to analyze the current trajectory vector, designs a deep Q learning reward network, takes the current trajectory vector and the expected trajectory vector as inputs of the reward network, and outputs reward information to train the parameters of the convolutional neural network CNN.

[0011] The convolutional neural network (CNN) algorithm uses the state of the environment around the multi-robot to analyze the current trajectory vector, and uses the current trajectory vector as the input of the convolutional neural network (CNN). The convolutional neural network (CNN) trained based on the reward information uses the convolutional neural network (CNN) algorithm to output corresponding action information to the environment information;

[0012] Then, a resource-based multi-robot task allocation algorithm is used to reasonably allocate all the actions of the workpiece to multiple robots, so that multiple robots can cooperate with each other without interference, thereby realizing complex three-dimensional spatial trajectory planning of multiple robots.

[0013] Furthermore, the basic structure of the convolutional neural network CNN is: input layer→convolution layer→pooling layer→repeated convolution layer, pooling layer→fully connected layer→output result.

[0014] Furthermore, the current trajectory vector will strive to be consistent with the expected trajectory vector.

[0015] Furthermore, the multi-robot trajectory planning method integrates deep Q learning and convolutional neural network (CNN) algorithms, and adopts experience representation technology. The learning experience occurring at each time step is stored in a data set through multiple events, which is called memory regeneration; the learning data samples are used to update the reconstructed memory with a certain probability each time, by reusing the experience data and reducing the correlation between samples.

[0016] Furthermore, the multi-robot trajectory planning method integrates deep Q learning and convolutional neural network (CNN) algorithms, uses empirical data based on the different roles assigned to individual robots, sets different expected values ​​for the roles of each robot before starting learning, and learns so that the compensation value always increases; if the search time of the algorithm is too long, the compensation value is reduced, and learning is performed so that the search time does not increase; the preprocessing part uses a convolutional neural network (CNN) to find outliers, and the post-processing part uses singular points to learn data; in the preprocessing part, the input image is used to search for image features, and these features are collected and learned.

[0017] Furthermore, in deep Q-learning, when the robot works in a discrete, constrained environment, it chooses one of a set of deterministic behaviors at each time interval and assumes that it is in a Markov state, whose state changes with different probabilities;

[0018] P r [s t+1 ]=s′[s t ,a t ]=P r [a t ] (1)

[0019] Where P r is the state transition probability matrix under random reward r, t is the time interval, s is the state, s' is all possible next states, s t is the state at time t, s t+1 is the state at time t+1, a t is the action at time t, r is the random reward;

[0020] In each time interval t, the robot can obtain state s from the environment and then perform action a t ; it gets a random reward r, which depends on the expected reward R st The state and behavior of the system to find the optimal strategy for the system to achieve;

[0021]

[0022] In the formula, is the expected reward of state s at time t, a iis the action at time i∈[1,n], i∈[1,n] is the time point, j∈[1,m] is the time point, r t+j is the random reward at time t+j, γ is the attenuation coefficient, and the discount factor means that the reward received within the time interval t will have a smaller impact than the current reward received; the operation value function V a By the policy function π and the policy value function V p To calculate, as shown in formula (3); starting from state s and following the strategy, the state value function of the expected reward is expressed as follows:

[0023] V a (s t )≡R s (π(s t ))+γ∑P xy [π(s t )]V p (s t ) (3)

[0024] In the formula, R s is the expected reward in state s, P xy is the conditional probability distribution matrix, γ is the attenuation coefficient, from which we can see that there is at least one optimal strategy. The goal of Q learning is to establish an optimal strategy without initial conditions. For the strategy, the Q value can be defined as follows:

[0025] Q p (s t ,a t )=R s (a t )+γ∑P xy [π(s t )]V p (s t ) (4)

[0026] In the formula, Q(s t ,a t ) is the state s t Take action t The Q value corresponds to the newly calculated Q(s t-1 ,a t-1 ), and Q(s t-1 ,a t-1 ) corresponds to the current Q(s t-1 ,a t-1 ) value and current Q(s t-1 ,a t-1 )'s next state.

[0027] Furthermore, in deep Q-learning, the Q-value is shared during learning and used for the learning machine; in order to optimize the update of the Q-value, it is necessary to define an objective function, which is defined as the error between the target value and the predicted value of the Q-value; the objective function is shown in equation (5):

[0028]

[0029] In the formula, a is the action, a' is all possible next actions, and the basic information of the loss function is to convert<s,a,r,s'> ; Therefore, first, perform a forward pass through the Q network using the state as input to obtain the action values ​​of all actions; after obtaining the environment return value of action a<r,s'> After that, the state s is used to obtain the action values ​​of all actions a again; then, the loss function is obtained through all the information obtained, which updates the weight parameters so that the Q value update of the selected action converges, that is, as close to the target value and the predicted value as possible; for the compensation function, if the distance to the current target point decreases before it decreases, the compensation will increase greatly; if the distance is getting closer, the compensation will decrease.

[0030] Furthermore, in deep Q-learning, two types of networks, a target Q-network and a Q-network, are used. The two networks have the same structure and only differ in weight parameters. In order to smooth the convergence in deep Q-learning, the target network is not updated continuously but periodically. The root mean square transfer algorithm is used as the optimizer, and the learning rate is adjusted according to the parameter gradient. In the case of a constantly changing training set, unlike the case of some training sets, it is necessary to constantly change the parameters.

[0031] Furthermore, in the multi-robot task allocation algorithm, the robots will continuously consume their resources during the execution of the task, and these resources must be refilled during operation; the robots will calculate the task performance based on their resource levels and consider all possibilities of visiting different combinations of resource stations, which enables the robots to reduce unnecessary waste of time and resources during the task.

[0032] Compared with the prior art, the beneficial effects of the present invention include at least:

[0033] The present invention integrates deep Q learning and convolutional neural network (CNN) algorithms, and adopts the convolutional neural network (CNN) algorithm to analyze the accurate position using information of its surrounding environment. Each robot acts according to the position obtained by deep Q learning analysis, and then through a resource-based robot task allocation method, all welding points of the workpiece are reasonably allocated to multiple welding robots, so that multiple robots can cooperate with each other without interference, thereby realizing spatial three-dimensional complex trajectory planning of multiple robots, and finally planning the optimal collaborative path for multiple robots, so that multiple robots can cooperate with each other without interference, and realize the high efficiency of robots collaboratively executing complex tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0035] Figure 1 It is a schematic diagram of the deep Q learning principle of the present invention;

[0036] Figure 2 It is a structural schematic diagram of a convolutional neural network CNN of the present invention;

[0037] Figure 3 It is a trajectory planning flow chart of the present invention that integrates deep Q learning and convolutional neural network CNN algorithm. DETAILED DESCRIPTION

[0038] In order to make the above-mentioned purposes, features and advantages of the present invention more obvious and easy to understand, the technical solution of the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments. It should be pointed out that the described embodiments are only part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0039] When multiple robots overlap in their workspaces, it is difficult to plan the collaborative trajectory for each robot. In traditional trajectory planning methods, robots need to search a relatively wide action area and move along a pre-designed route in a given environment. Each robot in a multi-robot system must act independently and collaborate with other robots to achieve excellent performance. In addition, the collaborative scheme of multiple robots is highly dependent on conditions such as the position and speed of each robot. However, traditional trajectory planning methods have difficulty in effectively handling various situations because each robot has difficulty identifying the robots around it as obstacles or collaborative robots.

[0040] In order to make up for these shortcomings, the present invention studies the information and strategy problems of reinforcement learning in multi-robot trajectory planning for complex tasks, and proposes a method that enables robots to quickly reach the target point through reinforcement learning. Depending on the situation, each robot can be regarded as a dynamic obstacle or a collaborative robot. That is to say, each robot in the system can perform independent actions according to a given task while cooperating with each other. After selecting an action, the relationship with the goal is evaluated, and each robot is rewarded or punished to carry out learning. At this point, reinforcement learning is a kind of deep Q-learning (Deep Q-Learning, DQN), which consumes less trajectory search time by sharing the Q parameters of each robot, and can be applied to static and dynamic environments of multiple robots.

[0041] The present invention is based on the principle of multi-robot trajectory planning based on deep Q learning, such as Figure 1 . The robot that selects action as output recognizes the environment and receives the state of the environment. When the state is changed, the state transition is transmitted to the individual as a reinforcement signal. The behavior of a single robot is selected to increase the sum of the reinforcement signal values ​​over a long period of time. The role of the action is to provide a control strategy for the control system. The ultimate goal of the multi-robot collaborative clamping / handling / flipping / welding system is to maximize the infinitely accumulated reward value during the state (multi-robot collaborative operation) process to achieve the optimal trajectory planning of the environment (multi-robot and workpiece).

[0042] When a robot operates in a discrete, constrained environment, it chooses one of a set of deterministic behaviors at each time interval and assumes that it is in a Markov state, with state changes having different probabilities.

[0043] P r [s t+1 ]=s′[s t ,a t ]=P r [a t ] (1)

[0044] Where P r is the state transition probability matrix under random reward r, t is the time interval, s is the state, s' is all possible next states, s t is the state at time t, s t+1 is the state at time t+1, a t is the action at time t, r is the random reward;

[0045] In each time interval t, the robot can obtain state s from the environment and then perform action a t It gets a random reward r, which depends on the expected reward Rst state and behavior to find the optimal strategy that the system wants to achieve.

[0046]

[0047] In the formula, is the expected reward of state s at time t, a i is the action at time i∈[1,n], i∈[1,n] is the time point, j∈[1,m] is the time point, r t+j is the random reward at time point t+j, γ is the attenuation coefficient, and the discount factor means that the reward received within the time interval t will have a smaller impact than the current reward. Operation value function V a By the policy function π and the policy value function V p To calculate, as shown in formula (3). Starting from state s and following the strategy, the state value function of the expected reward is expressed as follows.

[0048] V a (s t )≡R s (π(s t ))+γ∑P xy [π(s t )]V p (s t ) (3)

[0049] In the formula, R s is the expected reward in state s, P xy is the conditional probability distribution matrix, γ is the attenuation coefficient, γ∈[0,1], from which we can see that there is at least one optimal strategy. The goal of Q learning is to establish an optimal strategy without initial conditions. For the strategy, the Q value can be defined as follows:

[0050] Q p (s t ,a t )=R s (a t )+γ∑P xy [π(s t )]V p (s t ) (4)

[0051] In the formula, Q(s t ,a t ) is the state s t Take action t The Q value corresponds to the newly calculated Q(s t-1 ,a t-1 ), and Q(s t-1 ,a t-1 ) corresponds to the current Q(st-1 ,a t-1 ) value and current Q(s t-1 ,a t-1 )'s next state.

[0052] In multi-robot trajectory planning, existing methods are difficult to adapt to complex and dynamic systems and environments, but they can be integrated with deep Q learning and convolutional neural networks (CNN) to adopt multi-robot deep reinforcement learning. The convolutional neural network CNN proposed by Yann LeCun of New York University in 1998 can be regarded as a generalized form of the neocognitron and a variant of the multilayer perceptron (MLP). The multilayer perceptron is also called the artificial neural network (ANN). In addition to the input layer and the output layer, there can be multiple hidden layers in the middle.

[0053] The basic structure of the convolutional neural network CNN used in the present invention is: input layer (Input Layer) → convolution layer (Convolution Layer) → pooling layer (Pooling Layer) → (repeated convolution layer, pooling layer) → full connection layer (Full Connected Layer) → output result (Output Layer), such as Figure 2 As shown in the figure, the environmental information image is 2560×2000, the input layer is an integer multiple of 2, the number of convolutional layers is 16, a 3×3 filter is used, the pooling layer reduces the dimension of the convolution result, and the number of fully connected layers is 3.

[0054] The present invention integrates the trajectory planning process of deep Q learning and convolutional neural network (CNN) algorithm, such as Figure 3As shown. First, a reward network is designed, and both state information (current trajectory vector and expected trajectory vector) are used as its network input, and the output is reward information, which is used to train the parameters of the convolutional neural network CNN. Among them, the current trajectory vector will strive to be consistent with the expected trajectory vector through advanced weld tracking technology. The current trajectory vector is also used as the input of the convolutional neural network CNN. The convolutional neural network CNN trained based on the aforementioned reward output will output corresponding action information to the environmental information (multiple robots and workpieces), so that multiple robots can achieve collaborative clamping / handling / flipping / welding of three-dimensional complex welds in space. The present invention integrates deep Q learning and convolutional neural network CNN algorithms, and adopts empirical representation technology. The learning experience that occurs at each time step is stored in a data set by storing multiple events, which is also called memory regeneration. The learning data sample is used to update with a certain probability in the reconstructed memory each time. By reusing the empirical data and reducing the correlation between samples, data efficiency can be improved.

[0055] The present invention integrates deep Q learning and convolutional neural network (CNN) algorithms, uses empirical data based on the different roles assigned to individual robots, sets different expected values ​​for the roles of each robot before starting learning, and learns so that the compensation value always increases. If the search time of the algorithm is too long, the compensation value is reduced, and learning is performed so that the search time does not increase. The preprocessing part uses a convolutional neural network (CNN) to find outliers, and the post-processing part uses singular points to learn data. In the preprocessing part, the input image is used to search for image features, and these features are collected and learned. In this case, Q values ​​are learned for each robot assigned a different role, but the convolutional neural network (CNN) values ​​have the same input and different expected values. Therefore, the Q value is shared during learning and used for the learning machine. In order to optimize the update of the Q value, it is necessary to define an objective function, which is defined as the error between the target value and the Q value prediction value. The objective function is shown in equation (5).

[0056]

[0057] In the formula, a is the action, a' is all possible next actions, and the basic information of the loss function is to convert<s,a,r,s'> . So first, perform a forward pass through the Q network using the state as input to get the action values ​​for all actions. After getting the environment return value for action a<r,s'> After that, the action values ​​of all actions a are obtained again using the state s. Then, the loss function is obtained through all the information obtained, which updates the weight parameters so that the Q value of the selected action is updated and converged, that is, as close to the target value and the predicted value as possible. For the compensation function, if the distance to the current target point decreases before it decreases, the compensation will increase greatly; if the distance is getting closer, the compensation will decrease.

[0058] In deep Q-learning, two types of networks are used: target Q-network and Q-network. The two networks have the same structure and only differ in weight parameters. In order to smooth the convergence in deep Q-learning, the target network is not updated continuously but periodically. The Root Mean Square Propagation (RMSProp) algorithm is used as the optimizer and the learning rate is adjusted according to the parameter gradient. This means that in the case of a constantly changing training set, unlike in the case of some training sets, it is necessary to constantly change the parameters.

[0059] Subsequently, the project proposed a resource-based (RB) robot task allocation algorithm for the task allocation of two robots in collaborative welding. In this robot task allocation algorithm, the robot will continue to consume its resources during the execution of the task, and these resources must be refilled during operation. The robot will calculate the task performance based on its resource level and consider all possibilities of accessing different combinations of resource stations, which enables the robot to reduce unnecessary time and resource waste during the task.

[0060] Based on the above analysis, the present invention proposes a high-quality multi-robot trajectory planning method that integrates deep Q-learning and convolutional neural network (CNN) algorithms. The convolutional neural network (CNN) algorithm uses information about its surrounding environment to analyze the accurate position. Each robot acts according to the position obtained by the deep Q-learning analysis, and then adopts a resource-based multi-robot task allocation algorithm to reasonably allocate all welding points of the workpiece to two welding robots, thereby ultimately planning the optimal collaborative path for multiple robots, so that multiple robots can collaborate with each other without interference.

[0061] The above-mentioned embodiments only express several implementation methods of the present invention, and the description thereof is relatively specific and detailed, but it cannot be understood as limiting the scope of the patent of the present invention. It should be pointed out that, for ordinary technicians in this field, several variations and improvements can be made without departing from the concept of the present invention, which all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention shall be subject to the attached claims.

Claims

1. A multi-robot trajectory planning method, characterized in that: The steps include: Deep Q learning uses the state of the environment around multiple robots to analyze the current trajectory vector, designs a deep Q learning reward network, takes the current trajectory vector and the expected trajectory vector as inputs of the reward network, and outputs reward information to train the parameters of the convolutional neural network CNN. The current trajectory vector will strive to be consistent with the desired trajectory vector; The convolutional neural network (CNN) algorithm uses the state of the environment around the multi-robot to analyze the current trajectory vector, and uses the current trajectory vector as the input of the convolutional neural network (CNN). The convolutional neural network (CNN) trained based on the reward information uses the convolutional neural network (CNN) algorithm to output corresponding action information to the environment information; Then, a resource-based multi-robot task allocation algorithm is used to reasonably allocate all the actions of the workpiece to multiple robots, so that multiple robots can cooperate with each other without interference, thereby realizing complex three-dimensional spatial trajectory planning of multiple robots.

2. The multi-robot trajectory planning method according to claim 1, characterized in that: The basic structure of the convolutional neural network CNN is: input layer → convolution layer → pooling layer → repeated convolution layer, pooling layer → fully connected layer → output result.

3. The multi-robot trajectory planning method according to claim 1, characterized in that: The multi-robot trajectory planning method integrates deep Q learning and convolutional neural network (CNN) algorithms, and adopts experience representation technology. The learning experience occurring at each time step is stored in a data set through multiple events, which is called memory regeneration. The learning data samples are used to update the reconstructed memory with a certain probability each time, by reusing the experience data and reducing the correlation between samples.

4. The multi-robot trajectory planning method according to claim 1, characterized in that: The multi-robot trajectory planning method integrates deep Q learning and convolutional neural network (CNN) algorithms, uses empirical data based on the different roles assigned to individual robots, sets different expected values ​​for the roles of each robot before starting learning, and makes the compensation value always increase through learning; if the search time of the algorithm is too long, the compensation value is reduced, and learning is performed so that the search time does not increase; the preprocessing part uses convolutional neural network (CNN) to find outliers, and the post-processing part uses singular points to learn data; in the preprocessing part, the input image is used to search for image features, and these features are collected and learned.

5. The multi-robot trajectory planning method according to claim 1, characterized in that: In deep Q-learning, when a robot works in a discrete, constrained environment, it chooses one of a set of deterministic behaviors at each time interval and assumes that it is in a Markov state, with different probabilities of state changes; (1) In the formula, P r For random rewards r The state transition probability matrix under t is the time interval, s For status, s’ For all possible next states, s t for t The state of the moment, s t+1 for t +1 moment status, a t for t The action of the moment, r For random rewards; At each time interval t Inside, the robot gets the state from the environment s Then execute the action a t ; it gets a random reward r , which depends on the expected reward R st The state and behavior of the system to find the optimal strategy for the system to achieve; (2) In the formula, for t Status at the moment s The expected reward, a i for i ∈[1, n ]Moments of action, i ∈[1, n ] is the time point, j ∈[1, m ] is the time point, r t+j For time point t+j The following random rewards, γ is the decay coefficient, and the discount factor means that t The rewards received in the future will have less impact than the rewards currently received; the operation value function V a The policy function π and the policy value function V p To calculate, as shown in formula (3); from the state s When starting and following the policy, the state-value function of the expected reward is expressed as follows: (3) In the formula, R s for s The expected reward under the state, P xy is the conditional probability distribution matrix, γ is the attenuation coefficient. It can be seen that there is at least one optimal strategy. The goal of Q learning is to establish an optimal strategy without initial conditions. For the strategy, the Q value is defined as follows: (4) In the formula, Q ( s t , a t ) is the status s t Next Action a t The Q value corresponds to the newly calculated Q ( s t-1 , a t-1 ),and Q (s t-1 , a t-1 ) corresponds to the current Q ( s t-1 , a t-1 ) value and current Q ( s t-1 , a t-1 )'s next state.

6. The multi-robot trajectory planning method according to claim 5, characterized in that: In deep Q-learning, Q-values ​​are shared during learning and used in the learning machine. In order to optimize the update of Q-values, an objective function is defined as the error between the target value and the predicted value of Q-value. The objective function is shown in equation (5): (5) In the formula, a For action, a 'For all possible next actions, the basic information of the loss function is to convert < s , a , r , s' > So first, perform a Q network forward pass using the state as input to get the action values ​​for all actions; after getting the action a The environment return value of < r , s' >After that, use the status s Get all actions again a action value; then, the loss function is obtained through all the information obtained, which updates the weight parameters so that the Q value update of the selected action converges; for the compensation function, if the distance to the current target point decreases before it decreases, the compensation will increase; if the distance is getting closer, the compensation will decrease.

7. The multi-robot trajectory planning method according to claim 1, characterized in that: In deep Q-learning, two types of networks are used: target Q-network and Q-network. The two networks have the same structure and only different weight parameters. In order to smooth the convergence in deep Q-learning, the target network is not updated continuously but periodically. The root mean square transfer algorithm is used as the optimizer, and the learning rate is adjusted according to the parameter gradient. The parameters are constantly changed while the training set is constantly changing.

8. The multi-robot trajectory planning method according to claim 1, characterized in that: In the multi-robot task allocation algorithm, the robots will continuously consume their resources while performing tasks, and these resources must be refilled during operation; the robots will calculate the task performance based on their resource levels and consider all possibilities of visiting different combinations of resource stations, which enables the robots to reduce the waste of time and resources during tasks.

Images