A multi-robot navigation method for collaborative handling of large components

Abstract

This invention discloses a multi-robot navigation method for collaborative handling of large components, including the following steps: S1, obtaining the starting position s0 of the multi-robot movement s0 = (p x0 ,p y0 ), End point s g =(g x ,g y ) and obstacle information in the motion scene o S2. Design the formation of the multi-robot formation based on the shape of the object to be transported, and obtain the relative positional constraints between each robot; S3. Establish a deep neural network, the input of which is the state of the multi-robot formation, and the output is the execution action of the multi-robot formation; S4. Train the deep neural network using the PPO algorithm; S5. Input the state of the multi-robot formation into the trained Actor network to obtain the action at each step, and thus obtain the navigation path of the multi-robot formation from the starting point to the ending point. This method can obtain the navigation path and posture of the multi-robot formation based on the environment and the distance constraints between the multiple robots.

Description

Technical Field

[0001] This invention belongs to the field of mobile robot technology, specifically relating to a navigation method for multi-robot collaborative transport of large components. Background Technology

[0002] In the production and transportation of large equipment, it is often necessary to handle large components. Large component handling systems are widely used in shipbuilding, large aircraft manufacturing, concrete pipe pile production, and many other fields. Existing large component handling equipment mainly consists of hoisting equipment and lifting and translating mechanisms. Large components are hoisted by hoisting equipment and transported by lifting and translating mechanisms, resulting in relatively fixed handling paths. When the starting or ending point of transport changes, the track needs to be re-laid, leading to low efficiency and high costs. Another feasible solution is to use multi-robot collaborative handling, where multiple robots jointly support a target object, and the consistency of group movement is ensured through coordinated motion control between the robots, thus achieving the movement of the target object. Navigation and obstacle avoidance in multi-robot collaborative handling are generally achieved by setting guide paths using electromagnetic induction, lasers, or vision, but when the environment changes, the guide path needs to be re-laid, resulting in relatively low efficiency. In addition, another approach is to obtain the navigation path through traditional single-robot navigation and obstacle avoidance algorithms, and then use algorithms to calculate the poses of multiple robots. However, this method is more complex, and there is a situation where the navigation path of a single robot cannot be applied to multiple robots. That is, when multiple robots are working together to transport materials, there are constraints on the position and movement of each robot, and a path that a single robot can pass may not be passable by multiple robots. Summary of the Invention

[0003] Purpose of the invention: To address the problems existing in the prior art, the present invention provides a multi-robot navigation method for collaborative handling of large components. This method can obtain the navigation path and attitude of the multi-robot formation based on the environment and distance constraints between the multiple robots.

[0004] Technical solution: The present invention adopts the following technical solution:

[0005] A multi-robot navigation method for collaborative handling of large components includes the following steps:

[0006] S1. Obtain the starting position s0 of the multi-robot motion. x0 ,p y0 ), End point s g =(g x ,g y ) and obstacle information in the motion scene o The obstacle information s o =[(o x1 ,oy1 ),(o x1 ,o y1 ),…,(o xN ,o yN )];(o xn ,o yn ) represents the coordinates of the sampling points on the edge of the obstacle, 1≤n≤N, where N represents the total number of sampling points on the edge of the obstacle;

[0007] S2. Design the formation of the multi-robot group according to the shape of the object to be transported, and obtain the relative position constraints between each robot;

[0008] S3. Establish a deep neural network, wherein the input of the deep neural network is the state of the multi-robot formation, and the output is the execution action of the multi-robot formation;

[0009] The state S of the multi-robot formation is: S = [s g ,s r ,s o ], where s r =[(p x ,p y ),v x ,v y [arc] represents the motion parameters of a multi-robot formation, where the position parameters (p) are... x ,p y () represents the position of the multi-robot formation reference point, and the velocity parameter v x ,v y The ...

[0010] The deep neural network includes an Actor network and a Critic network. The Actor network takes the state S of the multi-robot formation as input and outputs the action act of the multi-robot formation. The Critic network takes the state S and action act of the multi-robot formation as input and outputs the evaluation value.

[0011] S4. Train the deep neural network using the PPO algorithm;

[0012] S5. Input the state of the multi-robot formation into the trained Actor network to obtain the execution action at each step. Calculate the motion parameters at the next moment based on the current motion parameters of the multi-robot formation and the execution action, and then obtain the navigation path of the multi-robot formation from the starting point to the end point.

[0013] Furthermore, in the deep neural network, the Actor network includes four hidden layers and one output layer. The number of neurons in the four hidden layers are 128, 256, 256, and 64, respectively, and the activation function is the tanh function. The output layer contains six output nodes, which represent the probabilities of different actions in the action space. The action with the highest probability value is selected as the action to be executed.

[0014] The Critic network consists of four hidden layers and one output layer. The number of neurons in the four hidden layers are 128, 256, 256, and 64, respectively, and the activation function is the tanh function. The output layer contains one output node, which represents the evaluation value.

[0015] Furthermore, the motion space includes six actions: action 1 represents the multi-robot formation moving to the left, action 2 represents the multi-robot formation moving upward, action 3 represents the multi-robot formation moving to the right, action 4 represents the multi-robot formation moving downward, action 5 represents the multi-robot formation rotating left to adjust its posture, and action 6 represents the multi-robot formation rotating right to adjust its posture.

[0016] Furthermore, step S4 specifically includes:

[0017] S41. Randomly initialize the parameters of the Actor network and the Critic network; set the iteration count m = 1;

[0018] S42. Let t = 0, and the multi-robot formation starts from the starting point s0 and initializes the motion parameters of the multi-robot formation;

[0019] S43. Based on the current state Actions are obtained from the Actor network. Calculate the state of the multi-robot formation at time t+1. And according to Calculate the current original reward This represents the state of the multi-robot formation at time t in the m-th iteration;

[0020] according to Determine whether the multi-robot formation has reached the destination or collided with an obstacle; if the multi-robot formation has not reached the destination or collided with an obstacle, let t = t + 1, and execute step S43 again until the multi-robot formation reaches the destination or collided with an obstacle.

[0021] S44. Record the trajectory of this iteration. Where Tm represents the duration of the multi-robot formation movement in the m-th iteration, i.e., reaching the destination or colliding with an obstacle;

[0022] Calculate the discount reward at each time step in this iteration. The discount reward at time step t is: γ is the discount factor;

[0023] S45. Optimize the values ​​of the Actor network using stochastic gradient descent, with the objective function being to maximize the discounted reward at each time step.

[0024] The weights of the Critic network are optimized, and the objective function of the optimization is to minimize the error between the value output by the Critic network and the discounted reward at each time step.

[0025] S46. Let m = m + 1, and jump to step S42 for the next iteration; until the change in discount reward between two iterations is less than the preset value ε.

[0026] Furthermore, in step S43, the state of the multi-robot formation at time t+1 is calculated. The specific steps are as follows:

[0027] (1) When performing an action When adjusting the direction of motion, if the time interval ΔT between time t and t+1 is greater than the adjustment time τ for the robot formation speed, the speed of the multi-robot formation reference point at time t+1 is the execution speed. The speed after that, and the position of the multi-robot formation reference point at time t+1. for:

[0028]

[0029]

[0030] Where 'a' is the preset acceleration, Let x and y be the velocities of the multi-robot formation reference point at time t in the m-th iteration, respectively;

[0031] If the time interval ΔT between time t and t+1 is less than the robot formation speed adjustment time τ, then the speeds of the multi-robot formation reference point in the x and y directions at time t+1 are... They are respectively:

[0032]

[0033]

[0034] v xmax and v ymax These are the preset maximum speeds in the x and y directions, respectively;

[0035] Location of multi-robot formation reference point at time t+1 for:

[0036]

[0037]

[0038] (2) When performing the action To adjust the attitude, the reference point position of the multi-robot formation remains unchanged. The attitude adjustment is performed by rotating around the reference point. The attitude parameters of the multi-robot formation at time t+1 are the parameters for the execution... The subsequent attitude parameter arc′.

[0039] Furthermore, when the object being transported is a rod-shaped component, the multi-robot formation consists of two robots, with a distance L between the two robots, where L is the length of the rod-shaped component.

[0040] When performing the action To adjust the attitude, the position coordinates of two robots in the multi-robot formation at time t+1. and They are respectively:

[0041]

[0042]

[0043] in The position parameters of the multi-robot formation reference point at time t;

[0044] The two robots move in the following directions: and

[0045] Furthermore, the original reward in step S43 for:

[0046] in As a reward for reaching the target point, Let be the distance between the reference point and the destination of the multi-robot formation at time t, where r1>0 is the first reward value, and ρ1 is the first distance threshold.

[0047] As a penalty for step length, c1<0 is the first penalty value; step is the number of steps taken by the multi-robot formation from the starting point to the current moment during this iteration;

[0048] Penalty for distance from obstacles, The minimum distance from the sampling point at the edge of the obstacle to the constructed object; ρ2 is the second distance threshold, and c2<0 is the second penalty value;

[0049] As a penalty for the distance from the finish line, c3<0 is the third penalty value.

[0050] Furthermore, the first distance threshold ρ1 is 0.2, the first reward value r1 is 200, the first penalty value c1 is -0.5, the second distance threshold ρ2 is 1, the second penalty value c2 is -150, and the third penalty value c3 is -2.

[0051] On the other hand, the present invention also discloses a computer storage medium storing computer instructions, which execute the above-described multi-robot navigation method when the computer instructions are run.

[0052] The present invention also discloses a computer device, including a processor and a storage medium, wherein the storage medium is the aforementioned computer-readable storage medium; the processor loads and executes instructions and data in the storage medium to implement the aforementioned multi-robot navigation method.

[0053] Beneficial effects: The multi-robot navigation method for collaborative handling of large components disclosed in this invention combines obstacle information, destination location and parameters such as relative position, distance, speed and attitude of multiple robots in the environment through reinforcement learning to obtain the path and pose of multi-robot collaborative handling, which improves efficiency and reduces cost, and makes deployment simpler. Attached Figure Description

[0054] Figure 1 This is a flowchart of the multi-robot navigation method for collaborative handling of large components disclosed in this invention;

[0055] Figure 2 This is a schematic diagram of the composition of the deep neural network in this invention;

[0056] Figure 3 A graph showing the change in discount rewards during the training process;

[0057] Figure 4 This is a schematic diagram illustrating a formation of two robots used to transport a rod-shaped component in an embodiment.

[0058] Figure 5 This is a schematic diagram of the composition of the computer device disclosed in this invention. Detailed Implementation

[0059] The present invention will be further explained below with reference to the accompanying drawings and specific embodiments.

[0060] This invention discloses a multi-robot navigation method for collaborative handling of large components, such as... Figure 1 As shown, the steps include:

[0061] Step 1: Obtain the starting position s0 of the multi-robot motion = (px0 ,p y0 ), End point s g =(g x ,g y ) and obstacle information in the motion scene o The obstacle information s o =[(o x1 ,o y1 ),(o x1 ,o y1 ),…,(o xN ,o yN )];(o xn ,o yn ) represents the coordinates of the sampling points on the edge of the obstacle, 1≤n≤N, where N represents the total number of sampling points on the edge of the obstacle;

[0062] In this invention, a lidar is installed on the robot's body to scan the environment and obtain obstacle information;

[0063] Step 2: Design the formation of the multi-robot team according to the shape of the object to be transported, and obtain the relative position constraints between each robot;

[0064] Step 3: Establish a deep neural network. The input of the deep neural network is the state of the multi-robot formation, and the output is the execution action of the multi-robot formation.

[0065] The state S of the multi-robot formation is: S = [s g ,s r ,s o ], where s r =[(px,py),v x ,v y [arc] represents the motion parameters of a multi-robot formation, where the position parameters (p) are... x ,p y () represents the position of the multi-robot formation reference point, and the velocity parameter v x ,v y The ...

[0066] like Figure 2 As shown, the deep neural network includes an Actor network and a Critic network. The input of the Actor network is the state S of the multi-robot formation, and the output is the action act of the multi-robot formation. The input of the Critic network is the state S and the action act of the multi-robot formation, and the output is the evaluation value.

[0067] In the deep neural network, the Actor network includes four hidden layers and one output layer. The number of neurons in the four hidden layers are 128, 256, 256, and 64, respectively, and the activation function is the tanh function. The output layer contains six output nodes, which represent the probabilities of different actions in the action space. The action with the highest probability value is selected as the action to be executed.

[0068] The motion space includes six actions: Action 1 represents the multi-robot formation moving to the left, Action 2 represents the multi-robot formation moving upward, Action 3 represents the multi-robot formation moving to the right, Action 4 represents the multi-robot formation moving downward, Action 5 represents the multi-robot formation rotating to the left to adjust its posture, and Action 6 represents the multi-robot formation rotating to the right to adjust its posture.

[0069] The Critic network consists of four hidden layers and one output layer. The number of neurons in the four hidden layers are 128, 256, 256, and 64, respectively, and the activation function is the tanh function. The output layer contains one output node, which represents the evaluation value.

[0070] Step 4: Train the deep neural network using the PPO algorithm, specifically including:

[0071] S41. Randomly initialize the parameters of the Actor network and the Critic network; set the iteration count m = 1;

[0072] S42. Let t = 0, and the multi-robot formation starts from the starting point s0 and initializes the motion parameters of the multi-robot formation;

[0073] S43. Based on the current state Actions are obtained from the Actor network. Calculate the state of the multi-robot formation at time t+1. And according to Calculate the current original reward This represents the state of the multi-robot formation at time t in the m-th iteration;

[0074] according to Determine whether the multi-robot formation has reached the destination or collided with an obstacle; if the multi-robot formation has not reached the destination or collided with an obstacle, let t = t + 1, and execute step S43 again until the multi-robot formation reaches the destination or collided with an obstacle.

[0075] Calculate the state of the multi-robot formation at time t+1. The specific steps are as follows:

[0076] (1) When performing an action When adjusting the direction of movement, the multi-robot formation adjusts its speed in the x and y directions according to a preset acceleration to achieve turning. For example, if the robot formation is moving along the x direction at a speed v, its speed in the y direction is 0. If the robot formation is then moved to the left, the speed in the x direction is adjusted to 0, and the speed in the y direction is adjusted to v. The speed adjustment process, i.e., the direction adjustment, is performed according to acceleration a. To simplify the calculation, the acceleration in the x and y directions is equal in this invention, both being a.

[0077] If the time interval ΔT between time t and t+1 is greater than the robot formation speed adjustment time τ, the speed of the multi-robot formation reference point at time t+1 is the execution speed. The speed after that, and the position of the multi-robot formation reference point at time t+1. for:

[0078]

[0079]

[0080] Where 'a' is the preset acceleration, Let x and y be the velocities of the multi-robot formation reference point at time t in the m-th iteration, respectively;

[0081] If the time interval ΔT between time t and t+1 is less than the robot formation speed adjustment time τ, then the speeds of the multi-robot formation reference point in the x and y directions at time t+1 are... They are respectively:

[0082]

[0083]

[0084] v xmax and v ymax These are the preset maximum speeds in the x and y directions, respectively;

[0085] Location of multi-robot formation reference point at time t+1 for:

[0086]

[0087]

[0088] (2) When performing the action To adjust the attitude, the reference point position of the multi-robot formation remains unchanged. The attitude adjustment is performed by rotating around the reference point. The attitude parameters of the multi-robot formation at time t+1 are the parameters for the execution... The subsequent attitude parameter arc′.

[0089] When the object being transported is a rod-shaped component, the multi-robot formation consists of two robots, and the distance between the two robots is L, where L is the length of the rod-shaped component.

[0090] When performing the action To adjust the attitude, the position coordinates of two robots in the multi-robot formation at time t+1. and They are respectively:

[0091]

[0092]

[0093] in The position parameters of the multi-robot formation reference point at time t;

[0094] The two robots move in the following directions: and

[0095] Original reward for:

[0096] in As a reward for reaching the target point, Let be the distance between the reference point and the destination of the multi-robot formation at time t, where r1>0 is the first reward value, and ρ1 is the first distance threshold.

[0097] As a penalty for step length, c1<0 is the first penalty value; step is the number of steps taken by the multi-robot formation from the starting point to the current moment during this iteration;

[0098] Penalty for distance from obstacles, ρ2 is the minimum distance from the sampling point on the edge of the obstacle to the transported structure at time t; ρ2 is the second distance threshold, and c2<0 is the second penalty value;

[0099] As a penalty for the distance from the finish line, c3<0 is the third penalty value.

[0100] In this embodiment, the first distance threshold ρ1 is 0.2, the first reward value r1 is 200, the first penalty value c1 is -0.5, the second distance threshold ρ2 is 1, the second penalty value c2 is -150, and the third penalty value c3 is -2.

[0101] S44. Record the trajectory of this iteration. Where Tm represents the duration of the multi-robot formation movement in the m-th iteration, i.e., reaching the destination or colliding with an obstacle;

[0102] Calculate the discount reward at each time step in this iteration. The discount reward at time step t is: γ is the discount factor;

[0103] S45. Optimize the values ​​of the Actor network using stochastic gradient descent, with the objective function being to maximize the discounted reward at each time step.

[0104] The weights of the Critic network are optimized, and the objective function of the optimization is to minimize the error between the value output by the Critic network and the discounted reward at each time step.

[0105] S46. Let m = m + 1, and jump to step S42 for the next iteration; until the change in discount reward between two iterations is less than the preset value ε.

[0106] like Figure 3 The curve represents the change in the discount reward during the training process. When the number of iterations exceeds 200, the change in the discount reward becomes very small, indicating that the algorithm has converged and the iteration has stopped.

[0107] Step 5: Input the state of the multi-robot formation into the trained Actor network to obtain the execution action for each step. Calculate the motion parameters for the next moment based on the current motion parameters of the multi-robot formation and the execution action, and thus obtain the navigation path of the multi-robot formation from the starting point to the end point.

[0108] This embodiment uses a formation of two robots to move a rod-shaped component as an example. Figure 4 As shown, there are three obstacles in the environment. The formation obtains the action for each step according to the Actor network, executes the action to obtain the position and orientation of each step, and then obtains the navigation path of the multi-robot formation from the starting point to the end point.

[0109] The present invention also discloses a computer storage medium storing computer instructions, which execute the above-described multi-robot navigation method when the computer instructions are run.

[0110] The present invention also discloses a computer device, such as Figure 5 As shown, it includes a processor 101 and a storage medium 102, wherein the storage medium 102 is the aforementioned computer-readable storage medium; the processor 101 loads and executes the instructions and data in the storage medium to implement the aforementioned multi-robot navigation method.

Claims

1. A multi-robot navigation method for collaborative handling of large components, characterized in that, Including the following steps: S1. Obtain the starting position of the multi-robot motion. Finish line and obstacle information in motion scenes The obstacle information ; Indicates the coordinates of the sampling points at the edge of the obstacle. , This indicates the total number of sampling points at the edge of the obstacle; S2. Design the formation of the multi-robot group according to the shape of the object to be transported, and obtain the relative position constraints between each robot; S3. Establish a deep neural network, wherein the input of the deep neural network is the state of the multi-robot formation, and the output is the execution action of the multi-robot formation; The state of the multi-robot formation for: ,in The motion parameters for a multi-robot formation, including position parameters. Position and velocity parameters of reference points for multi-robot formation These represent the velocities and attitude parameters of the multi-robot formation reference point in the x and y directions, respectively. The rotational radius of the multi-robot formation is centered on a reference point; the reference point for the multi-robot formation is the geometric center of the multi-robot formation. The deep neural network includes an Actor network and a Critic network, wherein the input to the Actor network is the state of the multi-robot formation. The output is the execution actions of the multi-robot formation. The input to the Critic network is the state of the multi-robot formation. and actions The output is the evaluation value. ; S4. Train the deep neural network using the PPO algorithm; S5. Input the state of the multi-robot formation into the trained Actor network to obtain the execution action for each step. Based on the current motion parameters and execution actions of the multi-robot formation... Calculate the motion parameters for the next moment, and then obtain the navigation path of the multi-robot formation from the starting point to the end point; Step S4 specifically includes: S41. Randomly initialize the parameters of the Actor network and Critic network; set the iteration number... ; S42, Order Multiple robots in formation started from the starting point Departure, initialize the motion parameters of the multi-robot formation; S43. Based on the current state The actions are obtained from the Actor network. ,calculate The state of multi-robot formation at all times and according to Calculate the current original reward ; Indicates the first In the next iteration The status of multiple robot formations at all times; according to Determine whether the multi-robot formation has reached the finish line or collided with an obstacle; if the multi-robot formation has neither reached the finish line nor collided with an obstacle, then... Repeat step S43 until the multi-robot formation reaches the finish line or collides with an obstacle; S44. Record the trajectory of this iteration. ;in Indicates the first The duration of the multi-robot formation motion in the next iteration is: That is, reaching the finish line or colliding with an obstacle; Calculate the discount reward at each time step in this iteration. The discount reward for the moment is: ; This is the discount factor; S45. Optimize the values ​​of the Actor network using stochastic gradient descent, with the objective function being to maximize the discounted reward at each time step. The weights of the Critic network are optimized, and the objective function is to minimize the output of the Critic network. The error between the value and the discount reward at each moment; S46, Order Proceed to step S42 for the next iteration; until the change in discount reward between two iterations is less than the preset value. .

2. The multi-robot navigation method according to claim 1, characterized in that, In the deep neural network, the Actor network includes four hidden layers and one output layer. The number of neurons in the four hidden layers are 128, 256, 256, and 64, respectively, and the activation function is the tanh function in all of them. The output layer contains six output nodes, each representing the probability of a different action in the action space; the action with the highest probability value is selected as the action to be executed. ; The Critic network consists of four hidden layers and one output layer. The number of neurons in the four hidden layers are 128, 256, 256, and 64, respectively, and the activation function is the tanh function in all of them. The output layer contains one output node, representing the evaluation value. .

3. The multi-robot navigation method according to claim 2, characterized in that, The motion space includes six actions: Action 1 represents the multi-robot formation moving to the left, Action 2 represents the multi-robot formation moving upward, Action 3 represents the multi-robot formation moving to the right, Action 4 represents the multi-robot formation moving downward, Action 5 represents the multi-robot formation rotating to the left to adjust its posture, and Action 6 represents the multi-robot formation rotating to the right to adjust its posture.

4. The multi-robot navigation method according to claim 1, characterized in that, Calculation in step S43 The state of multi-robot formation at all times The specific steps are as follows: (1) When performing an action When adjusting the direction of motion, if at time and The time interval between Adjustment time greater than the robot formation speed , The speed of the multi-robot formation reference point at any given time is the execution speed. The speed after that, Location of reference points for multi-robot formation at all times for: in For the preset acceleration, , The first In the next iteration The velocity of the multi-robot formation reference point in the x and y directions at any given time; If time and The time interval between Adjustment time less than the robot formation speed , The velocity of the multi-robot formation reference point in the x and y directions at any given time They are respectively: and These are the preset maximum speeds in the x and y directions, respectively; Location of reference points for multi-robot formation at all times for: (2) When performing the action To ensure that the reference point position remains unchanged during attitude adjustment in a multi-robot formation, the attitude adjustment is performed by rotating around the reference point. The attitude parameters of the multi-robot formation at any given time are for execution. Post-attitude parameters .

5. The multi-robot navigation method according to claim 4, characterized in that, When the object being transported is a rod-shaped component, the multi-robot formation consists of two robots, and the distance between the two robots is L, where L is the length of the rod-shaped component. When performing the action When adjusting posture, Position coordinates of two + robots in a multi-robot formation at any time and They are respectively: ， ； ， ； in Reference points for multi-robot formation Position parameters at any given time; The two robots move in the following directions: and .

6. The multi-robot navigation method according to claim 1, characterized in that, The original reward in step S43 for: ; in As a reward for reaching the target point, , for The distance between the reference point and the destination of the multi-robot formation at any given time. The first reward value; The first distance threshold; As a penalty for step length, , The first penalty value; This represents the number of steps taken by the multi-robot formation from the starting point to the current moment during this iteration. Penalty for distance from obstacles, , for The minimum distance from the sampling point at the edge of the obstacle to the constructed object at any given time; The second distance threshold, This is the second penalty value; As a penalty for the distance from the finish line, , This is the third penalty value.

7. The multi-robot navigation method according to claim 6, characterized in that, First distance threshold The first reward value is 0.

2. The first penalty value is 200. The second distance threshold is -0.

5. The second penalty value is 1. -150, the third penalty value It is -2.

8. A computer storage medium, characterized in that, It stores computer instructions, characterized in that the computer instructions, when executed, perform the multi-robot navigation method according to any one of claims 1 to 7.

9. A computer device, characterized in that, It includes a processor and a storage medium, wherein the storage medium is the computer-readable storage medium as described in claim 8; the processor loads and executes the instructions and data in the storage medium to implement the multi-robot navigation method as described in any one of claims 1 to 7.

Images