A data-driven multi-autonomous mobile robot dynamic collaborative transportation operation method

Abstract

The application provides a kind of data-driven based multi-autonomous mobile robot dynamic cooperative transport operation method, belongs to the field of multi-autonomous mobile robot trajectory planning and tracking control. First, the feasible global trajectory conforming to the kinematic characteristics and safety of each autonomous mobile robot is planned;Second, the speed information of each trajectory point is obtained. Then, the feasible global trajectory with speed information is processed to obtain the specific executable local reference trajectory, and the local reference trajectory is fitted to obtain the local expected trajectory. Finally, the local expected trajectory is tracked, and the control parameters are adjusted, and the multi-autonomous mobile robot dynamic cooperative transport operation is completed. The application can improve the robot trajectory tracking accuracy and stability;Through real-time local trajectory planning and adaptive parameter adjustment method based on reinforcement learning, while ensuring the multi-autonomous mobile robot dynamic cooperative transport operation, the safety and anti-interference ability of the multi-autonomous mobile robot system are improved.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of multi-autonomous mobile robot trajectory planning and tracking control, and relates to a multi-autonomous mobile robot dynamic collaborative transportation operation method with certain prior environment information. BACKGROUND

[0002] Autonomous mobile robots are widely used in various real-world scenarios, providing strong support for productivity improvement, technology iteration, and capacity expansion in related fields, such as warehouse management, factory inspection, and park transportation. During the transportation operation of multi-autonomous mobile robots, there are a large number of safety constraints and related environmental restrictions due to avoiding other autonomous mobile robots, which will seriously affect the safety and transportation operation task execution efficiency of the multi-autonomous mobile robot system. Therefore, it is of great practical significance to develop a multi-autonomous mobile robot dynamic collaborative transportation operation trajectory planning and tracking control method based on data-driven.

[0003] The paper "CL-MAPF: Multi-agent path finding for car-like robots with kinematic and spatiotemporal constraints" published in Robotics and Autonomous Systems, Volume 150, 2022, proposes a multi-robot trajectory planning method based on the combination of spatiotemporal hybrid A-star algorithm and conflict search algorithm. This method integrates Ackerman steering features into trajectory planning, significantly improving the executability of intelligent robots for trajectories. However, traditional trajectory planning algorithms simply consider the running speed of intelligent agents as a constant value and the waiting speed as zero, which severely limits the efficiency of multi-autonomous mobile robots. Therefore, a speed planning based on the principle of transition is proposed to plan the speed while tracking the trajectory. In addition, the paper "Path Planning and Trajectory Tracking for Autonomous Obstacle Avoidance in Automated Guided Vehicles at Automated Terminals" published in Axioms, Volume 13, Issue 1, 2023, proposes an autonomous obstacle avoidance path and trajectory tracking control scheme based on automated guided vehicles, significantly improving the safety of automated guided vehicles in task execution. However, the traditional model predictive control method based on artificial potential field has certain limitations in complex environments due to fixed parameters. Therefore, a parameter adaptive method based on reinforcement learning is proposed. SUMMARY

[0004] In view of the above problems existing in the prior art, in order to enable the multi-autonomous mobile robot system to be applied in different scenes and ensure the internal and external safety of the system and the stability of task execution, the present application provides a multi-autonomous mobile robot dynamic collaborative transportation operation method based on data driving.

[0005] To achieve the above object, the present application adopts the following technical scheme.

[0006] A multi-autonomous mobile robot dynamic collaborative transportation operation method based on data driving, which can realize autonomous global trajectory planning, local trajectory planning and trajectory tracking control in a structured warehouse logistics scene, is a multi-autonomous mobile robot dynamic collaborative transportation operation method based on reinforcement learning, car-like conflict search algorithm (CL-CBS) and artificial potential field method-model predictive control algorithm (APF-MPC). The multi-autonomous mobile robot dynamic collaborative transportation operation method first obtains feasible global trajectories conforming to the kinematic characteristics and safety of each autonomous mobile robot by using the CL-CBS algorithm. Secondly, the velocity information of each trajectory point is obtained by processing the feasible global trajectories by using the velocity planning based on the transition principle. Then, the local reference trajectories that can be specifically executed are obtained by processing the feasible global trajectories with velocity information by using the artificial potential field method, and the local expected trajectories are obtained by fitting the local reference trajectories by using a quintic polynomial. Finally, the local expected trajectories are tracked by using the model predictive control algorithm, and the control parameters of the model predictive control algorithm are adaptively adjusted by using reinforcement learning to realize accurate tracking of the local expected trajectories and complete the multi-autonomous mobile robot dynamic collaborative transportation operation. Specifically, the following steps are included:

[0007] Step 1: According to the real scene, a simulation environment is built, and the CL-CBS algorithm is used to find the feasible global trajectories for the multi-autonomous mobile robots. The CL-CBS algorithm decomposes the feasible global trajectory finding problem into upper and lower algorithms: the lower algorithm is responsible for planning the global reference trajectory, and the global reference trajectory is transmitted to the upper algorithm; the upper algorithm is responsible for eliminating the conflicts in the global reference trajectory, and the conflict elimination scheme is transmitted back to the lower algorithm. In addition, the lower algorithm will re-plan the global reference trajectory according to the conflict elimination scheme, and the global reference trajectory will be re-transmitted to the upper algorithm, and then the cycle between the upper algorithm and the lower algorithm will be performed in turn until there is no conflict in the global reference trajectory, i.e. the feasible global trajectories of each autonomous mobile robot are found. It should be emphasized that the upper algorithm uses a binary tree to search for conflicts, and the lower algorithm uses a penalty temporal-spatial hybrid A-star algorithm based on the low-energy principle. In addition, three evaluation indexes are introduced to accurately evaluate the performance of the CL-CBS algorithm. Specifically, the following steps are included:

[0008] Step 1.1: Construct a structured virtual map according to the real warehouse logistics scene, in which obstacles are set in clusters and randomly scattered to increase the complexity and authenticity of the map, and provide a credible environmental basis for the implementation of the dynamic collaborative transportation operation method; in addition, the structured virtual map is defined as a continuous workspace

[0009] Then, the continuous space can be obtained by step 1.1

[0010] Step 1.2: Define the graph theory basis of the CL-CBS algorithm, and let multiple autonomous mobile robots move in the continuous workspace The area occupied by the obstacle is Therefore, the free working environment of the autonomous mobile robot is The real-time state of the autonomous mobile robot is set as The position it occupies is The i-th autonomous mobile robot Rob i is responsible for reaching the target state from the initial state where X represents the horizontal coordinate of the autonomous mobile robot in the Cartesian coordinate system, Y represents the vertical coordinate of the autonomous mobile robot in the Cartesian coordinate system, and represents the yaw angle of the autonomous mobile robot in the Cartesian coordinate system.

[0011] In addition, the trajectory of the i-th autonomous mobile robot Rob i from the initial state s i to the target state g i is defined as a feasible global trajectory π i ; π i is composed of the real-time state of the autonomous mobile robot over continuous time, so it can be expressed as In addition, π i needs to satisfy the following conditions:

[0012] (1) The feasible global trajectory π i should start from the initial state π i [0] = s i and reach the target state π i [t] = g i after a finite time step; in addition, the i-th autonomous mobile robot Rob i needs to be able to maintain at the target point, t step represents the time step;

[0013] (2) When the i-th autonomous mobile robot Rob i moves along its trajectory, it should not collide with the obstacle, t represents a sampling time.

[0014] Specifically, all the starting states, target states and execution states of the autonomous mobile robots in the system should not conflict with each other during the execution of the task.

[0015] Then, the execution framework of the CL-CBS algorithm can be obtained through step 1.2.

[0016] Step 1.3: Based on the space-time hybrid A-star algorithm, a penalty term based on the pose change of the autonomous mobile robot is set to form a penalty space-time hybrid A-star algorithm, which is used as the lower algorithm of the CL-CBS algorithm. The Ackerman steering characteristics of the autonomous mobile robot are introduced into the global trajectory planning through the penalty space-time hybrid A-star algorithm, which can make the autonomous mobile robot have better executability to the global reference trajectory. Specifically, the algorithm searches and expands the trajectory nodes in the free working environment , and forms an open list through the expanded trajectory nodes. The open list forms a tuple The tuple means that the i-th autonomous mobile robot Rob i is in the state z i at the sampling time t; wherein N.f represents the total cost of the current trajectory node; N.g represents the cost from the initial state s i to the current state z i ; and N.h represents the heuristic cost. In addition, the relationship among N.f, N.g and N.h can be expressed as follows:

[0017] N.f = N.g + N.h (1)

[0018] wherein N.h can be further expressed as the cost from the current state z i to the target state g i , which is calculated as follows:

[0019]

[0020] wherein h RS represents the minimum cost of connecting the current state to the target state by using the Reed-Shepp curve, where Reeds-Shepp curve is the shortest curve connecting two points with circular arcs and straight lines, which is proposed in the paper of J. A. Reeds, "Optimal Paths for a Car that Goes Both Forwards and Backwards". In addition, in order to improve the operation accuracy of the autonomous mobile robot, a penalty N.g is imposed on N.f, which is calculated as follows:

[0021] N.g pen = w turn w dir N.g (3)

[0022] where w turn is the first penalty coefficient based on the pose change, w dir is the second penalty coefficient based on the pose change; w turn and w dir need to be manually set, and w turn ≥ 1, w dir ≥ 1; specifically, w turn is used to examine whether the autonomous mobile robot has a transition between forward and backward, w dir is used to examine whether the autonomous mobile robot has a transition between left and right directions. If there is a transition between forward and backward, w turn > 1 is adopted; if there is a transition between left and right directions, w dir > 1 is adopted; if there is no transition, w turn = w dir = 1. Then, the total cost of the current trajectory node can be further calculated as follows:

[0023] N.f = N.g pen + h (4)

[0024] Therefore, the tuple can be further represented as By continuously updating the tuple , the lower algorithm can obtain the global reference trajectory of each autonomous mobile robot, whether the global reference trajectory is feasible needs to be detected by the upper algorithm, and if there is no conflict, it is a feasible global trajectory.

[0025] Then, through step 1.3, the lower algorithm of the CL-CBS algorithm can obtain the global reference trajectory of each autonomous mobile robot, and transmit the global reference trajectory to the upper algorithm for conflict detection and resolution.

[0026] Step 1.4: The upper algorithm uses the binary tree detection principle to detect the conflict of the global reference trajectory input by the lower algorithm. Specifically, the upper algorithm will traverse the preliminary global reference trajectory based on the binary tree to detect whether the global reference trajectory is a feasible global trajectory. If there is no conflict, it means that the CL-CBS algorithm has found a feasible global trajectory for each autonomous mobile robot. If there is a conflict, the upper algorithm will impose constraints on the first conflict that occurs, that is, generate (a i ,N.π j [t],t) and (a j ,N.π i [t],t); the former requires the i-th autonomous mobile robot Rob i to be prohibited from driving into the position at the sampling time t j ; the latter requires the j-th autonomous mobile robot Rob to be prohibited from driving into the position cost at the sampling time t . This constraint is the conflict resolution scheme. Further, the upper algorithm of CL-CBS will transmit the conflict resolution scheme back to the lower algorithm, and the lower algorithm will re-plan the global reference trajectory for each autonomous mobile robot based on the resolution scheme. In addition, the upper algorithm and the lower algorithm of CL-CBS will repeat the loop solution until the trajectories of each autonomous mobile robot do not conflict, that is, the feasible global trajectory of each autonomous mobile robot is found.

[0027] Therefore, through steps 1.3-1.4, that is, through the loop solution of the lower algorithm and the upper algorithm of the CL-CBS algorithm, the feasible global trajectory of each autonomous mobile robot can be found.

[0028] Step 1.5: After obtaining the feasible trajectory of each autonomous mobile robot, in order to specifically and accurately evaluate the performance of the CL-CBS algorithm, three evaluation indexes are introduced to contribute to the convenience of data visualization and analysis, and the expressions of the evaluation indexes are as follows:

[0029]

[0030] Among them, N cost represents the total generation value, which is the first evaluation index; T makespan represents the longest running time step of CL-CBS, which is the second evaluation index; T averageflow represents the average time step of the agent to reach the target point, which is the third evaluation index. In addition, n AMR represents the number of autonomous mobile robots, N.f(i) represents the total cost of the i-th autonomous mobile robot Rob i from the initial state s i to the target state g i , t step represents the time step.

[0031] By step 1, the feasible global trajectory of the multi-autonomous mobile robot can be obtained, and step 1 is a global trajectory planning module of the multi-autonomous mobile robot.

[0032] Step 2: Based on the feasible global trajectory of each autonomous mobile robot obtained by step 1, a speed planning based on the transition principle is adopted for each autonomous mobile robot. The main purpose of the speed planning based on the transition principle is to reduce the waiting action of different autonomous mobile robots due to the intersection of the feasible global trajectory, and to keep the robot in the forward motion state as much as possible to improve the running accuracy of the autonomous mobile robot. The specific steps are as follows:

[0033] Step 2.1: According to the feasible global trajectory output by the CL-CBS algorithm, a speed planning method based on the transition principle is proposed. By analyzing the feasible global trajectory, the time nodes at which each autonomous mobile robot starts and ends the waiting action can be directly obtained, and then the time period is finely divided according to the time node at which the waiting action ends. Specifically, the time node at which each waiting action of a certain autonomous mobile robot ends is regarded as the key time node of the autonomous mobile robot, and then the feasible global trajectory of the autonomous mobile robot is divided into different time periods according to the key time node, so that the autonomous mobile robot adopts the corresponding speed in different time periods to avoid collision and waiting action.

[0034] Then, by step 2.1, the key time nodes of each autonomous mobile robot can be obtained.

[0035] Step 2.2: Through the key time nodes of each autonomous mobile robot, the first key time node t1 of the autonomous mobile robot, the time period T1 before the first key node occurs, and the time period T2 after the first key node occurs can be obtained. In addition, in order to facilitate the selection of appropriate speed by different autonomous mobile robots, the priority of the autonomous robot needs to be determined; specifically, the autonomous mobile robot with waiting action is the secondary priority, and the autonomous mobile robot without waiting action is the highest priority. The longitudinal speed v x of the autonomous mobile robot with the highest priority adopts the preset ideal speed v e , and the longitudinal speed v x of the autonomous mobile robot with the secondary priority adopts the speed v p , and the selection range of v p is calculated as follows:

[0036]

[0037] Where δ v represents the speed weighting coefficient, and 0≤δ v ≤1; Ls L represents the length of the autonomous mobile robot.

[0038] On this basis, in order to facilitate the selection of v p , different speed reference values are set in the selected range according to a speed interval of 0.5 m / s through formula (6), which can make the sub-priority autonomous mobile robot select a suitable longitudinal speed. It should be noted that the time node at which the sub-priority autonomous mobile robot ends the waiting action is different, so there is no collision risk between the sub-priority autonomous mobile robots.

[0039] Then, through step 2.2, the longitudinal speed v x of the sub-priority autonomous mobile robot and the highest-priority autonomous mobile robot in the T1 stage can be obtained.

[0040] Step 2.3: After the autonomous mobile robot rated as sub-priority passes the first key time node, the priority needs to be re-evaluated. Specifically, if the key time node continues to appear, the autonomous mobile robot is still sub-priority, and the speed is re-taken according to formula (6); if there is no longer a key node in front, the longitudinal speed of the autonomous mobile robot returns to the preset ideal longitudinal speed v e . Then, the autonomous mobile robot can pass each key time node safely in turn.

[0041] Through step 2, the feasible global trajectory with speed information of each autonomous mobile robot can be obtained.

[0042] Step 3: After obtaining the feasible global trajectory with speed information, collision avoidance processing needs to be performed, which is to prevent collision caused by deviation between the local reference trajectory output by local trajectory planning and the feasible global trajectory. First, a point mass dynamics model is established to provide a model basis for optimization solution. Second, the artificial potential field method is used to inflate the obstacles and autonomous mobile robots, and it is integrated into the objective function of the optimization solution to find a safe local reference trajectory. Then, the objective function for realizing collision avoidance function is established. Finally, to further smooth the local reference trajectory, a quintic polynomial is used to fit the local reference trajectory to obtain a local expected trajectory. The specific steps are as follows:

[0043] Step 3.1: The size information of each autonomous mobile robot and the load transfer caused by lateral and longitudinal acceleration are ignored, and a point mass dynamics model of the autonomous mobile robot is established, which can be represented as follows:

[0044]

[0045] On the basis of the point-mass dynamics model, further consider the dynamics constraints, that is, add constraint condition |a y |<ug, where u represents the control variable, that is, the front wheel steering angle w; then formula (7) can be further expressed as follows:

[0046]

[0047] where, represents the state variable, represents the first-order derivative of ξ(t) with respect to the sampling time t, f(·) represents a function that can be calculated, f(·) is composed of ξ(t) and a y v y represents the longitudinal velocity of the autonomous mobile robot in the vehicle body coordinate system, x represents the lateral velocity of the autonomous mobile robot in the vehicle body coordinate system, represents the yaw angle of the autonomous mobile robot in the Cartesian coordinate system, Y represents the lateral coordinate of the autonomous mobile robot in the Cartesian coordinate system, X represents the longitudinal coordinate of the autonomous mobile robot in the Cartesian coordinate system, and a y all represent the longitudinal acceleration of the autonomous mobile robot in the vehicle body coordinate system, represents the lateral acceleration of the autonomous mobile robot in the vehicle body coordinate system; represents the first-order derivative of Y with respect to the sampling time t, represents the first-order derivative of X with respect to the sampling time t, represents the first-order derivative of the sampling time t.

[0048] Then, through step 3.1, the point-mass dynamics model of the autonomous mobile robot can be obtained, which is the basis for optimization solution.

[0049] Step 3.2: While establishing the point-mass dynamics model, an artificial potential field function based on obstacles and the position of the autonomous mobile robot needs to be established, the purpose of the artificial potential field function is to adjust the size of the artificial potential field repulsion by calculating the distance between the obstacle and the autonomous mobile robot; Wherein, the obstacles include not only the obstacles in the virtual map, but also the remaining autonomous mobile robots except the autonomous mobile robot. By establishing the artificial potential field function, the autonomous mobile robot can avoid the area with large repulsive force in the optimization solution, that is, avoid obstacles, and the artificial potential field function is calculated as follows:

[0050]

[0051] where, w oa represents the global control weight, wdis Represents the distance weight, w oa and w dis Need to be set manually, and w oa >0,w dis >0; Г represents the repulsive threshold of the artificial potential field function, r g is the radiation radius of the artificial potential field function, Г and r g Need to be set manually, and Г≥1, r g ≥1; J obc represents the repulsive force of the artificial potential field, v represents the speed of the autonomous mobile robot; E dis represents the Euclidean distance between the autonomous mobile robot and the obstacle, which can be expressed as E dis =((XX obc ) 2 +(YY obc ) 2 ) 1 / 2 , where X obc Represents the horizontal coordinate of the obstacle in the Cartesian coordinate system, Y obc Represents the vertical coordinate of the obstacle in the Cartesian coordinate system.

[0052] It is not difficult to find that according to formula (9), when v is near zero, J obc The value of will also approach zero, which will make the autonomous mobile robot pass through obstacles. Therefore, it is necessary to constrain v in the artificial potential field function to meet the reasonable expansion of the obstacle. Therefore, v is defined as follows:

[0053]

[0054] Then, through step 3.2, the real-time repulsive force value of the autonomous mobile robot during operation can be obtained, which is also the basis for optimization solution.

[0055] Step 3.3: In order to facilitate the integration of the artificial potential field function into the optimization solution, the point mass dynamics model is directly used for solution, and the forward Euler method is used for discretization. In addition, since the control goal of the local trajectory planning is to minimize the deviation from the feasible global trajectory and achieve obstacle avoidance, the objective function J of the local trajectory planning layer can be p It is expressed as follows:

[0056]

[0057] Among them, Q local Represents the first weight matrix of the objective function, R local The second weight matrix representing the objective function, Q local and R local Manual setting is required; Jobs.t V (t) represents the value of the artificial potential field function at sampling time t, N p represents the prediction horizon, N c represents the control horizon. η (t+m|t) represents the local reference trajectory calculated by optimization starting from sampling time t for the t+m time period; η ref (t+m|t) represents the feasible global trajectory for the t+m time period starting from sampling time t; ΔU (t+m|t) represents the control increment matrix for the t+m time period starting from sampling time t; U t represents the control matrix at sampling time t, U min represents the minimum value of the control matrix, U max represents the maximum value of the control matrix, U min and U max need to be manually set. In addition, represents a column vector with N c rows, u (t-1) represents the control at sampling time t-1.

[0058] Then, by step 3.3, the optimized local reference trajectory can be obtained.

[0059] Step 3.4: After obtaining the optimized local reference trajectory, in order to make the local reference trajectory smoother and take into account the continuous change of the pose of the autonomous mobile robot, a quintic polynomial is used to fit the local reference trajectory to obtain a local expected trajectory, which can be represented as follows:

[0060]

[0061] where a n represents the fitting parameter of the local trajectory expected horizontal coordinate, n = 0, 1, 2, 3, 4, 5; b n represents the fitting parameter of the local trajectory expected yaw angle; Y local represents the local trajectory expected longitudinal coordinate, represents the yaw angle of the local expected trajectory.

[0062] By steps 2-3, the local expected trajectory of the multi-autonomous mobile robot autonomous planning can be obtained, and steps 2-3 are the local trajectory planning module of the multi-autonomous mobile robot.

[0063] Step 4: After obtaining the local expected trajectory, the local expected trajectory needs to be tracked. According to the dynamic characteristics of the autonomous mobile robot, a model predictive control algorithm based on a three-degree-of-freedom dynamic model is established, and the difference in left and right wheel steering angles caused by Ackerman steering geometry is ignored. The specific steps are as follows:

[0064] Step 4.1: Establish a three-degree-of-freedom dynamic model of the autonomous mobile robot, and the dynamic equation is as follows:

[0065]

[0066] wherein, represents the cornering stiffness of the front wheel, represents the cornering stiffness of the rear wheel; C σf represents the longitudinal stiffness of the front wheel, C σr represents the longitudinal stiffness of the rear wheel; l f represents the distance from the front axle to the center of mass, l r represents the distance from the rear axle to the center of mass, m represents the mass of the autonomous mobile robot, γ represents the yaw rate, w represents the front wheel turning angle; I z represents the moment of inertia around the z-axis, and the z-axis represents an axis perpendicular to the ∑xoy plane in the vehicle body coordinate system.

[0067] Therefore, by step 4.1, a three-degree-of-freedom dynamic model of the autonomous mobile robot can be obtained, which is the basis of the model predictive control algorithm.

[0068] Step 4.2: After obtaining the three-degree-of-freedom dynamic model, for the convenience of calculation, the nonlinear time-varying model described in formula (13) is linearized, and the linearized state space is as follows:

[0069]

[0070] wherein, represents the state quantity under the three-degree-of-freedom dynamic model; represents the first derivative of χ with respect to the sampling time t, and u represents the control quantity, i.e., the front wheel turning angle w; represents the output quantity; A, B, and C are coefficient matrices, which can be directly obtained through linearization.

[0071] Therefore, by step 4.2, a linear state space of the autonomous mobile robot can be obtained, which can greatly reduce the calculation difficulty.

[0072] Step 4.3: The linear state space shown in formula (14) is discretized by using the forward Euler method, and the discretized form is as follows:

[0073] χ(t+1)=A k χ(t)+B k u(t)(15)

[0074] wherein, A k =I6+T t A, B k =T t B; Tt representing the sampling period, needs to be set manually; A k representing the weight matrix of the state variable, B k representing the weight matrix of the control variable, I6 represents the 6-order unit matrix; χ(t+1) represents the state variable at the sampling time t+1, χ(t) represents the state variable at the sampling time t, and u(t) represents the control variable at the sampling time t.

[0075] Then, through step 4.3, the discretized linear state space of the autonomous mobile robot can be obtained, which can further reduce the operation amount.

[0076] Step 4.4: After obtaining the discretized linear state space, in order to avoid the sudden change of the control variable, the state variable needs to be expanded as ξ(t) = [χ(t) u(t-1)] T ; wherein u(t-1) represents the control variable at the sampling time t-1, and ξ(t) represents the expanded state variable at the sampling time t. Then, the new state space is as follows:

[0077]

[0078] wherein, represents the augmented parameter matrix of the expanded state variable, wherein, represents the N u row N x column zero matrix, represents the N u order unit matrix; represents the augmented parameter matrix of the control increment; represents the augmented parameter matrix of the output variable, wherein, represents the N y row N u column zero matrix; N u represents the number of control variables, N x represents the number of state variables, N y represents the number of output variables, N u , N x and N y need to be set manually; Δu(t) represents the control increment at the sampling time t, and η(t) represents the expanded output variable at the sampling time t.

[0079] Then, through step 4.4, the expanded state space of the autonomous mobile robot can be obtained, which can prevent the sudden change of the control variable on the basis of the discretized linear state space, so as to improve the stability of the local desired trajectory tracking.

[0080] Step 4.5: After obtaining the extended state space, a target function needs to be set for the model predictive control algorithm. In order to enable the autonomous mobile robot to track the local desired trajectory and improve the stability of the autonomous mobile robot, the optimization goal needs to minimize the error between the model predictive control algorithm output trajectory and the local desired trajectory; in addition, in order to avoid the control increment being too large to cause the autonomous mobile robot to lose control, the optimization goal needs to minimize the control amount as much as possible; finally, in order to avoid the control amount mutation affecting the continuity of the control amount, a soft constraint needs to be added. Therefore, the target function is as follows:

[0081]

[0082] Wherein, Q mpc is the weight matrix of the extended state quantity, R mpc is the weight matrix of the control increment matrix; p is a weight coefficient, and e is a relaxation factor; η(t+k|t) represents the execution trajectory calculated by the model predictive control algorithm in the t+k time period starting from the sampling time t; η ref.p (t+k|t) represents the local desired trajectory in the t+k time period starting from the sampling time t; ΔU(t+m|t) represents the control increment matrix in the t+k time period starting from the sampling time t; ΔU min represents the minimum value of the control increment, and ΔU max represents the maximum value of the control increment, respectively; U min represents the minimum value of the control amount, and U max represents the maximum value of the control amount, respectively; in addition, represents an N u order unit matrix, and D represents the weight matrix of the control amount constraint.

[0083] Through step 4, the autonomous mobile robot can track the local desired trajectory and output the optimal control amount, and step 4 is a trajectory tracking control module of the multiple autonomous mobile robots.

[0084] Step 5: In order to further reduce the deviation between the local trajectory planning and the feasible global trajectory and improve the tracking accuracy of the trajectory tracking control module, the weight matrices Q mpc and R mpc of formula (17) need to be adjusted in real time. The real-time parameter adjustment method based on reinforcement learning is an autonomous learning strategy, which has certain intelligence and high efficiency. The present application adopts an Actor-Critic framework, which can better compensate for the slow convergence speed of traditional reinforcement learning. The framework mainly includes Actor and Critic parts. The Actor part is composed of a target policy network and an online policy network, which is used to estimate the deterministic policy function and determine the current information (s t , a t , rt s t+1 ) and local desired trajectory, etc., wherein s t represents the current time state, a t represents the action, r t represents the reward, and s t+1 represents the next time state; it is emphasized that the action a t of the autonomous learning strategy is the weight matrix Q mpc and R mpc . The Critic part is composed of the online Q network and the target Q network, which evaluates the value of the current action through the reward signal fed back by the environment and the estimation of the next state of the autonomous mobile robot to update and adjust the policy of the Actor part; it is noted that the reward of the autonomous learning strategy is based on the tracking of the local desired trajectory, that is, the better the local desired trajectory is tracked, the higher the reward is.

[0085] Specifically, the Actor part generates an action according to the current environment, the environment feeds back the next state and the reward to the Actor part, and the action is transmitted to the Critic part for evaluation; after receiving the state, the Critic evaluates the action according to the online policy network and the target policy network, and transmits the evaluation result (gradient) to the Actor part to continuously optimize the action of the Actor, that is, to obtain the maximum reward value; at the same time, the experience pool stores information (s t ,a t ,r t ,s t+1 ) and samples to make the reinforcement learning break the data correlation. Then, through the autonomous learning strategy, the optimal action, that is, the weight matrix Q mpc and R mpc , can be obtained; in addition, the autonomous learning strategy transmits the optimal weight matrix Q mpc and R mpc to the target function of formula (17) so that the autonomous mobile robot can accurately track the local desired trajectory.

[0086] Then, through step 5, the optimal weight matrix Q mpc and R mpc can be obtained, which helps the trajectory tracking control module to accurately track the local desired trajectory.

[0087] Step 6: updating the weight matrix Q mpc and R mpcAnd the optimal control increment sequence in the control time domain can be obtained by solving the optimization problem shown in formula (17); further, the first control increment in the optimal control increment sequence is taken as the actual control increment, and the control quantity is obtained as follows:

[0088] u(t) = u(t-1) + Delta u(t-1) (18)

[0089] Wherein, u(t) = w(t), w(t) represents the front wheel steering angle at the sampling time t; u(t-1) = w(t-1), w(t-1) represents the front wheel steering angle at the sampling time t-1; Delta u(t-1) = Delta w(t-1), Delta w(t-1) represents the expected front wheel steering angle increment at the sampling time t-1.

[0090] The optimal control quantity of the autonomous mobile robot can be obtained by adjusting the weight matrix of the model predictive control in real time through steps 5-6, and steps 5-6 are the reinforcement learning module of the multiple autonomous mobile robots.

[0091] Step 7: The optimal control quantity output by step 6 is transmitted to each autonomous mobile robot, and the state information of each autonomous mobile robot is output. On this basis, the state information of each autonomous mobile robot is transmitted to the local trajectory planning module and the tracking control module, the state information of the next step is updated, and the dynamic collaborative transportation operation is carried out until the dynamic collaborative transportation operation of the multiple autonomous mobile robots is completed.

[0092] The beneficial effects of the present application are:

[0093] (1) The present application proposes a punishment type space-time hybrid A-star algorithm, which reduces the redundant steering and reversing actions of the autonomous mobile robot according to the punishment quantity, and plans an optimal feasible global trajectory according to the task information and the initial and final state, thereby significantly improving the low energy consumption and safety of the multiple autonomous mobile robots.

[0094] (2) The present application proposes a speed planning based on the transition principle, which can autonomously adopt the corresponding speed according to the key time nodes in the feasible global trajectory information. At the same time, in the local trajectory planning module, a quintic polynomial is used to smooth the trajectory discrete points, which greatly improves the transportation task execution efficiency of the multiple autonomous mobile robots and helps to complete the dynamic collaborative transportation operation.

[0095] (3) The present application provides an adaptive parameter adjustment method based on reinforcement learning, which drives each autonomous mobile robot to update the weight matrix of the model predictive control algorithm in real time according to the current environment and state, thereby significantly improving the trajectory tracking accuracy and stability of the autonomous mobile robot. BRIEF DESCRIPTION OF DRAWINGS

[0096] Figure 1is a dynamic collaborative transportation operation trajectory planning and tracking control framework of multiple autonomous mobile robots;

[0097] Figure 2 is a speed planning schematic diagram based on the principle of transition;

[0098] Figure 3 is a three-dimensional artificial potential field function diagram of a virtual warehouse logistics environment. DETAILED DESCRIPTION

[0099] In order to make the purpose, technical scheme and advantages of the present application clearer, the present application is described in detail in combination with the drawings and examples.

[0100] As Figures 1-3 shown, the present application comprises the following steps:

[0101] Step 1 comprises the following steps:

[0102] Step 1.1: Construct a structured virtual map according to the real warehouse logistics scene, and set obstacles in a cluster and randomly scattered in the map to improve the complexity and reality of the map, as Figure 1 shown, the structured virtual map can be established for multiple autonomous mobile robots according to the environment information in the prior information, which provides a reliable environment basis for the implementation of the dynamic collaborative transportation operation method; in addition, the structured virtual map is defined as a continuous workspace

[0103] Then, the continuous space

[0104] Step 1.2: Define the graph theory basis of CL-CBS algorithm, and let the multiple autonomous mobile robots work in the continuous workspace , the area occupied by the obstacle is Therefore, the free working environment of the autonomous mobile robot is The real-time state of the autonomous mobile robot is set as The position it occupies is The i-th autonomous mobile robot Rob i is responsible for reaching the target state from the initial state Where X represents the horizontal coordinate of the autonomous mobile robot in the Cartesian coordinate system, Y represents the vertical coordinate of the autonomous mobile robot in the Cartesian coordinate system, represents the yaw angle of the autonomous mobile robot in the Cartesian coordinate system.

[0105] In addition, the i-th autonomous mobile robot Rob i reaches the target state g i from the initial state s iThe trajectory of the i-th autonomous mobile robot Rob i is defined as a feasible global trajectory π i which consists of the real-time states of the autonomous mobile robot at consecutive time, so it can be expressed as π i Moreover, the following conditions need to be satisfied:

[0106] (1) The feasible global trajectory π i should start from the initial state π i [0] = s i and reach the goal state π i [t] = g i after a finite time step; in addition, the i-th autonomous mobile robot Rob i needs to be able to maintain at the goal point t step represents the time step;

[0107] (2) The i-th autonomous mobile robot Rob i should not collide with obstacles when moving along its trajectory, t represents the sampling time.

[0108] Specifically, during the execution of the task, the starting state, the goal state, and the execution state of all autonomous mobile robots in the system need to be consistent.

[0109] Then, the execution framework of the CL-CBS algorithm can be obtained through step 1.2.

[0110] Step 1.3: Based on the space-time hybrid A* algorithm, a penalty term based on the pose change of the autonomous mobile robot is set to form a penalty space-time hybrid A* algorithm, which is used as the lower algorithm of the CL-CBS algorithm. The Ackerman steering characteristics of the autonomous mobile robot are introduced into the global trajectory planning through the penalty space-time hybrid A* algorithm, which can make the autonomous mobile robot have better executability to the global reference trajectory. Specifically, this algorithm searches and expands the trajectory nodes in the free working environment , and forms an open list through the expanded trajectory nodes. The open list forms a tuple This tuple means that at the current trajectory node, the state of the i-th autonomous mobile robot Rob i at the sampling time t is z i ; where N.f represents the total cost of the current trajectory node; N.g represents the cost from the initial state s i to the current state z i ; N.h represents the heuristic cost. In addition, the relationship between N.f, N.g, and N.h is as follows:

[0111] N.f = N.g + N.h (1)

[0112] where N.h can be further expressed as the current state z i to the goal state g i , which is calculated as follows:

[0113]

[0114] where h RS represents the minimum cost of connecting the current state to the goal state by using a Reed-Shepp curve, and d represents the Euclidean distance between them; where the Reed-Shepp curve is the shortest curve connecting two points by using circular arcs and straight lines, which is proposed in the paper “Optimal Paths for a Car that Goes Both Forwards and Backwards” published by J. A. Reeds. In addition, in order to improve the running accuracy of the autonomous mobile robot, a penalty is imposed on N.g, which is calculated as follows:

[0115] N.g pen = w turn w dir N.g (3)

[0116] where w turn is the first penalty coefficient based on the pose change, w dir is the second penalty coefficient based on the pose change; w turn and w dir need to be manually set, and w turn ≥ 1, w dir ≥ 1; specifically, w turn examines whether the autonomous mobile robot has a transition between forward and backward, w dir examines whether the autonomous mobile robot has a transition between left and right directions. If there is a transition between forward and backward, w turn > 1 is adopted; if there is a transition between left and right directions, w dir > 1 is adopted; if there is no transition, w turn = w dir = 1. Then, the total cost of the current trajectory node can be further calculated as follows:

[0117] N.f = N.g pen + N.h (4)

[0118] Therefore, the tuple can be further expressed as By constantly updating the tuples The lower layer algorithm can obtain the global reference trajectory of each autonomous mobile robot, whether the global reference trajectory is feasible needs to be detected by the upper layer algorithm, if there is no conflict, it is a feasible global trajectory.

[0119] Then, through step 1.3, the lower layer algorithm of the CL-CBS algorithm can obtain the global reference trajectory of each autonomous mobile robot, and transmit the global reference trajectory to the upper layer algorithm for conflict detection and resolution.

[0120] Step 1.4: The upper layer algorithm uses the binary tree detection principle to detect the global reference trajectory input by the lower layer algorithm. Specifically, the upper layer algorithm will traverse the preliminary global reference trajectory based on the binary tree to detect whether the global reference trajectory is a feasible global trajectory. If there is no conflict, it means that the CL-CBS algorithm has found a feasible global trajectory for each autonomous mobile robot. If there is a conflict, the upper layer algorithm will impose constraints on the first conflict that occurs, that is, generate (a i ,N.π j [t],t) and (a j ,N.π i [t],t); the former requires the i-th autonomous mobile robot Rob i Inhibit driving into position at sampling time t The latter requires the j-th autonomous mobile robot Rob j Inhibit driving into position at sampling time t The constraint is the conflict resolution scheme. Further, the upper layer algorithm will transmit the conflict resolution scheme back to the lower layer algorithm, and the lower layer algorithm will plan a global reference trajectory for each autonomous mobile robot based on the resolution scheme. In addition, the upper layer algorithm and the lower layer algorithm of the CL-CBS will repeatedly loop to solve until the trajectories of each autonomous mobile robot do not conflict, that is, a feasible global trajectory for each autonomous mobile robot is found.

[0121] Then, through step 1.3-1.4, that is, through the loop solving of the lower layer algorithm and the upper layer algorithm of the CL-CBS algorithm, a feasible global trajectory for each autonomous mobile robot can be found. Specifically, as shown in Figure 1 The robot pose information and task information are provided by the prior information, and according to the virtual map, the feasible global trajectory solving of multiple autonomous mobile robots can be completed by the CL-CBS algorithm.

[0122] Step 1.5: After obtaining the feasible trajectory of each autonomous mobile robot, in order to specifically and accurately evaluate the performance of the CL-CBS algorithm, three evaluation indexes are introduced to contribute to the convenience of data visualization and analysis, and the expressions of the evaluation indexes are as follows:

[0123]

[0124] wherein, N cost represents the total cost value, which is the first evaluation index; T makespan represents the CL-CBS longest running time step, which is the second evaluation index; T averageflow represents the average time step of the agent reaching the target point, which is the third evaluation index. In addition, n AMR represents the number of autonomous mobile robots, N.f(i) represents the i-th autonomous mobile robot Rob i from the initial state s i to the target state g i , t step represents the time step.

[0125] Through step 1, the feasible global trajectory of autonomous planning of multiple autonomous mobile robots can be obtained, and step 1 is a global trajectory planning module of multiple autonomous mobile robots.

[0126] Step 2: The specific steps are as follows:

[0127] Step 2.1: According to the feasible global trajectory output by the CL-CBS algorithm, a speed planning method based on the transition principle is proposed. By analyzing the feasible global trajectory, the time nodes at which the waiting actions of each autonomous mobile robot appear and end can be directly obtained, and then the time period is finely divided according to the time nodes at which the waiting actions end. Specifically, the time nodes at which each waiting action of a certain autonomous mobile robot ends are all taken as the key time nodes of the autonomous mobile robot, and then the feasible global trajectory of the autonomous mobile robot is divided into different time periods according to the key time nodes, so that the autonomous mobile robot adopts the corresponding speed in different time periods to avoid collision and waiting action. As shown in Figure 2 , the traditional speed planning method sets the speed of the autonomous mobile robot to zero when the waiting action occurs, which greatly reduces the efficiency of dynamic collaborative transportation operation and causes excessive energy consumption due to repeated acceleration and deceleration of the autonomous mobile robot; the speed planning based on the transition principle can set different speeds for different autonomous mobile robots based on the key time nodes, effectively improving the traffic efficiency and reducing energy consumption; Figure 2 v x_1 represents the longitudinal speed of the first autonomous mobile robot, v x_2 represents the longitudinal speed of the second autonomous mobile robot.

[0128] Then, through step 2.1, the key time nodes of each autonomous mobile robot can be obtained.

[0129] Step 2.2: Through the key time nodes of each autonomous mobile robot, the first key time node t1 of the autonomous mobile robot, the time period T1 before the first key node occurs, and the time period T2 after the first key node occurs can be obtained. In addition, in order to facilitate the selection of appropriate speed by different autonomous mobile robots, the priority of the autonomous robot needs to be determined; specifically, the autonomous mobile robots with waiting actions are all sub-priority, and the autonomous mobile robots without waiting actions are all the highest priority. The longitudinal speed v x of the autonomous mobile robot with the highest priority is the preset ideal speed v e . x The longitudinal speed v p of the autonomous mobile robot with sub-priority is the speed v p . The selection range of v v is calculated as follows:

[0130]

[0131] wherein δ v represents a speed weighting coefficient, and 0≤δ s ≤1; L p represents the trajectory length of the autonomous mobile robot in the T1 stage, and L represents the length of the autonomous mobile robot.

[0132] On this basis, in order to facilitate the selection of v x , different speed reference values are sequentially set in the selection range according to a speed interval of 0.5 m / s through formula (6), which can make the autonomous mobile robot with sub-priority select an appropriate longitudinal speed. It should be noted that for the autonomous mobile robots with sub-priority that exist feasible global trajectory intersections, the time nodes at which their waiting actions end are different, so there is no collision risk between the autonomous mobile robots with sub-priority.

[0133] Therefore, through step 2.2, the longitudinal speed v x of the autonomous mobile robot with sub-priority and the autonomous mobile robot with the highest priority in the T1 stage can be obtained.

[0134] Step 2.3: After the autonomous mobile robot rated as sub-priority passes the first key time node, the priority needs to be re-evaluated. Specifically, if the key time node continues to appear, the autonomous mobile robot is still sub-priority, and the speed is re-selected according to formula (6); if there is no key node in front, the longitudinal speed of the autonomous mobile robot returns to the preset ideal longitudinal speed v e . Therefore, the autonomous mobile robot can pass each key time node in turn safely.

[0135] Through step 2, each autonomous mobile robot can obtain a feasible global trajectory with velocity information.

[0136] Step 3, the specific steps are as follows:

[0137] Step 3.1: ignoring the size information of each autonomous mobile robot and the load transfer caused by the lateral and longitudinal acceleration, a point mass dynamics model of the autonomous mobile robot is established, which can be expressed as follows:

[0138]

[0139] On the basis of the point mass dynamics model, further consider the dynamics constraint, that is, add the constraint condition |a y |<ug, where u represents the control quantity, that is, the front wheel steering angle w; then formula (7) can be further expressed as follows:

[0140]

[0141] Wherein, represents the state quantity, represents the first order derivative of ξ(t) with respect to the sampling time t, f(·) represents a function that can be calculated, f(·) is composed of ξ(t) and a y v y represents the longitudinal velocity of the autonomous mobile robot in the vehicle body coordinate system, x represents the lateral velocity of the autonomous mobile robot in the vehicle body coordinate system, represents the yaw angle of the autonomous mobile robot in the Cartesian coordinate system, Y represents the lateral coordinate of the autonomous mobile robot in the Cartesian coordinate system, X represents the longitudinal coordinate of the autonomous mobile robot in the Cartesian coordinate system, and a y all represent the longitudinal acceleration of the autonomous mobile robot in the vehicle body coordinate system, represents the lateral acceleration of the autonomous mobile robot in the vehicle body coordinate system; represents the first order derivative of Y with respect to the sampling time t, represents the first order derivative of X with respect to the sampling time t, represents the first order derivative of t with respect to the sampling time t.

[0142] Then, through step 3.1, the point mass dynamics model of the autonomous mobile robot can be obtained, which is the basis for optimization solution.

[0143] Step 3.2: While establishing the point mass dynamics model, it is necessary to establish an artificial potential field function based on the position of the obstacle and the autonomous mobile robot. The purpose of this artificial potential field function is to adjust the magnitude of the repulsive force of the artificial potential field function by calculating the distance between the obstacle and the main autonomous mobile robot. The obstacles include not only the obstacles in the virtual map but also the autonomous mobile robots other than the main autonomous mobile robot. By establishing this artificial potential field function, the autonomous mobile robot can avoid areas with large repulsive forces during the optimization solution, that is, avoid obstacles. This artificial potential field function is calculated as follows:

[0144]

[0145] Among them, w oa Represents the global control weight, w dis Represents the distance weight, w oa and w dis Need to be set manually, and w oa >0,w dis >0; Г represents the repulsive threshold of the artificial potential field function, r g is the radiation radius of the artificial potential field function, Г and r g Need to be set manually, and Г≥1, r g ≥1; J obc represents the repulsive force of the artificial potential field, v represents the speed of the autonomous mobile robot; E dis represents the Euclidean distance between the autonomous mobile robot and the obstacle, which can be expressed as E dis =((XX obc ) 2 +(YY obc ) 2 ) 1 / 2 , where X obc Represents the horizontal coordinate of the obstacle in the Cartesian coordinate system, Y obc Represents the vertical coordinate of the obstacle in the Cartesian coordinate system.

[0146] It is not difficult to find that according to formula (9), when v is near zero, J obc The value of will also approach zero, which will make the autonomous mobile robot pass through obstacles. Therefore, it is necessary to constrain v in the artificial potential field function to meet the reasonable expansion of the obstacle. Therefore, v is defined as follows:

[0147]

[0148] Specifically, if Figure 3 As shown, the repulsive threshold Г of the artificial potential field function is set to 5, and the radiation radius r g Set to 1, global control weight w oais set to 1, distance weight w dis is set to 1; on this basis, the velocity v of the autonomous mobile robot is set to 0, 0.5 m / s, 1 m / s and 2 m / s in turn, and different repulsive force values J can be obtained obc ; and the repulsive force values J of each coordinate when v is set to 0, 0.5 m / s and 1 m / s obc are equal, which verifies the feasibility of formula (9) and formula (10); in the figure, X g represents the horizontal axis of the Cartesian coordinate system, Y g represents the vertical axis of the Cartesian coordinate system, and Z represents the vertical axis of the Cartesian coordinate system.

[0149] Then, through step 3.2, the real-time repulsive force value that the autonomous mobile robot receives in the running process can be obtained, which is also the basis for optimization solving.

[0150] Step 3.3: In order to facilitate the integration of the artificial potential field function into the optimization solving, the point mass dynamics model is directly used for solving, and the forward Euler method is used for discretization processing. In addition, since the control goal of local trajectory planning is to minimize the deviation from the feasible global trajectory and to achieve obstacle avoidance, the objective function J p of the local trajectory planning layer can be represented as follows:

[0151]

[0152] Among them, Q local represents the first weight matrix of the objective function, R local represents the second weight matrix of the objective function, Q local and R local need to be manually set; J obs.t is the artificial potential field function value at sampling time t, N p represents the prediction time domain, N c represents the control time domain. η(t+m|t) represents the local reference trajectory calculated by the optimization solving in the t+m time period starting from the sampling time t; η ref (t+m|t) represents the feasible global trajectory in the t+m time period starting from the sampling time t; ΔU(t+m|t) represents the control increment matrix in the t+m time period starting from the sampling time t; U t represents the control matrix at sampling time t, U min represents the minimum value of the control matrix, U max represents the maximum value of the control matrix, U min and U max need to be manually set. In addition, represents the number of rows N ca column vector, u(t-1) represents the control variable at the sampling time t-1.

[0153] Then, by step 3.3, the optimized local reference trajectory can be obtained.

[0154] Step 3.4: After obtaining the optimized local reference trajectory, in order to make the local reference trajectory smoother and considering that the pose change of the autonomous mobile robot is continuous, a quintic polynomial is used to fit the local reference trajectory to obtain a local expected trajectory, which can be expressed as follows:

[0155]

[0156] wherein a n represents the fitting parameter of the expected horizontal coordinate of the local trajectory, n = 0, 1, 2, 3, 4, 5; b n represents the fitting parameter of the expected yaw angle of the local trajectory; Y local represents the expected longitudinal coordinate of the local trajectory, represents the expected yaw angle of the local trajectory.

[0157] By steps 2-3, the local expected trajectory of the multi-autonomous mobile robot can be obtained, and steps 2-3 are the local trajectory planning module of the multi-autonomous mobile robot. Specifically, as shown in Figure 1 based on the feasible global trajectory input by the global trajectory planning module, velocity planning based on the leap principle is performed on the autonomous mobile robot, and an artificial potential field function is established; on this basis, a quintic polynomial fitting is performed on the local reference trajectory to obtain a local expected trajectory.

[0158] Step 4, the specific steps are as follows:

[0159] Step 4.1: a three-degree-of-freedom dynamic model of the autonomous mobile robot is established, and the dynamic equation is shown in formula (13). By step 4.1, the three-degree-of-freedom dynamic model of the autonomous mobile robot is obtained, which is the basis of the model predictive control algorithm.

[0160] Step 4.2: after obtaining the three-degree-of-freedom dynamic model, in order to facilitate calculation, the nonlinear time-varying model described in formula (13) is linearized, and the linearized state space is shown in formula (14). Then, by step 4.2, the linear state space of the autonomous mobile robot is obtained, which can greatly reduce the calculation difficulty.

[0161] Step 4.3: Discretize the linear state space shown in equation (14) by using forward Euler method, and the discretized form is shown in equation (15). Then, the discretized linear state space of the autonomous mobile robot is obtained by step 4.3, which can further reduce the computation amount.

[0162] Step 4.4: After obtaining the discretized linear state space, in order to avoid the sudden change of the control amount, the state amount needs to be expanded as ξ(t) = [χ(t) u(t-1)] T ; wherein u(t-1) represents the control amount at sampling time t-1, and ξ(t) represents the expanded state amount at sampling time t. Then, the new state space is shown in equation (16). The expanded state space of the autonomous mobile robot is obtained by step 4.4, which can prevent the sudden change of the control amount on the basis of the discretized linear state space, so as to improve the stability of tracking the local desired trajectory.

[0163] Step 4.5: After obtaining the expanded state space, the objective function needs to be set for the model predictive control algorithm. In order to enable the autonomous mobile robot to track the local desired trajectory and improve the stability of the autonomous mobile robot, the error minimization between the output trajectory of the model predictive control algorithm and the local desired trajectory is taken as the optimization target; in addition, in order to avoid the control increment being too large to cause the autonomous mobile robot to lose control, the control amount being as small as possible is taken as the optimization target; finally, in order to avoid the sudden change of the control amount affecting the continuity of the control amount, the soft constraint is added. Therefore, the objective function is shown in equation (17).

[0164] By step 4, the autonomous mobile robot is controlled to track the local desired trajectory, and the optimal control amount is output. Step 4 is the trajectory tracking control module of the multiple autonomous mobile robots. Specifically, as shown in Figure 1 , the local desired trajectory information is input by the local trajectory planning module, and the information is substituted into the three-degree-of-freedom dynamic model to obtain the execution trajectory information calculated by the model predictive control algorithm; on this basis, the optimal calculation is performed by the objective function, so as to reduce the deviation from the local desired trajectory and improve the driving stability of the autonomous mobile robot.

[0165] Step 5: In order to further reduce the deviation between the local trajectory planning and the feasible global trajectory, and improve the tracking precision of the trajectory tracking control module, the weight matrix Q mpc and R mpcThe real-time parameter adjusting method based on reinforcement learning is an autonomous learning strategy, and has certain intelligence and high efficiency. The autonomous learning strategy is based on an Actor-Critic framework, which can compensate for the slow convergence speed of traditional reinforcement learning. The framework mainly includes an Actor and a Critic. The Actor includes a target policy network and an online policy network, and is used for estimating a deterministic policy function. The Actor estimates the deterministic policy function according to current information (s t ,a t ,r t ,s t+1 ) and a local expected trajectory. s t represents a current state, a t represents an action, r t represents a reward, and s t+1 represents a next state. It should be emphasized that the action a t of the autonomous learning strategy is a weight matrix Q mpc and R mpc . The Critic includes an online Q network and a target Q network. The Critic evaluates the value of the current action according to a reward signal fed back by the environment and an estimation of the next state of the autonomous mobile robot, and updates and adjusts the policy of the Actor. It should be noted that the reward of the autonomous learning strategy is based on the tracking of the local expected trajectory. The better the local expected trajectory is tracked, the higher the reward is.

[0166] Specifically, the Actor generates an action according to the current environment. The environment feeds back the next state and the reward to the Actor, and transmits the action to the Critic for evaluation. After receiving the state, the Critic evaluates the action according to the online policy network and the target policy network, and transmits the evaluation result (gradient) to the Actor, so that the action of the Actor is continuously optimized, i.e., the maximum reward value is obtained. Meanwhile, the experience pool stores information (s t ,a t ,r t ,s t+1 ), and samples the information, so that the reinforcement learning can break the data correlation. Then, the optimal action, i.e., the weight matrix Q mpc and R mpc , can be obtained through the autonomous learning strategy. In addition, the autonomous learning strategy transmits the optimal weight matrix Q mpc and R mpc to the objective function of formula (17), so that the autonomous mobile robot can accurately track the local expected trajectory.

[0167] Then, the optimal weight matrix Q mpc and R mpc, which is helpful for the trajectory tracking control module to track the local desired trajectory accurately.

[0168] Step 6: updating the weight matrix Q in the objective function as shown in formula (17) through the parameter adjustment method based on reinforcement learning mpc and R mpc , and solving the optimization problem as shown in formula (17), the optimal control increment sequence in the control time domain can be obtained; further taking the first control increment in the optimal control increment sequence as the actual control increment, the control quantity is obtained as follows:

[0169] u(t) = u(t-1) + Delta u(t-1) (18)

[0170] Wherein, u(t) = w(t), w(t) represents the front wheel steering angle at the sampling time t; u(t-1) = w(t-1), w(t-1) represents the front wheel steering angle at the sampling time t-1; Delta u(t-1) = Delta w(t-1), Delta w(t-1) represents the expected front wheel steering angle increment at the sampling time t-1.

[0171] By adjusting the weight matrix of the model predictive control in real time through steps 5-6, the optimal control quantity of the autonomous mobile robot can be obtained, and steps 5-6 are the reinforcement learning module of the multiple autonomous mobile robots. Specifically, as shown in Figure 1 , the reinforcement learning module updates the action, state and other information of the actor module according to the real-time environment and the real-time attitude of the autonomous mobile robot, and continuously optimizes the strategy through the optimizer to improve the reward value, so as to obtain the optimal weight matrix Q mpc and R mpc .

[0172] Step 7: the optimal control quantity output by step 6 is transmitted to each autonomous mobile robot, and the state information of each autonomous mobile robot is output. On this basis, the state information of each autonomous mobile robot is transmitted to the local trajectory planning module and the tracking control module, the state information of the next step is updated and the dynamic collaborative transportation operation is carried out until the dynamic collaborative transportation operation of the multiple autonomous mobile robots is completed.

[0173] The present application can ensure the stability and safety of the multiple autonomous mobile robot system on the basis of ensuring the effective completion of the collaborative transportation operation task.

[0174] The above disclosure is only a specific embodiment of the present application, but the protection scope of the present application is not limited thereto, any skilled person in the art can make equivalent replacement or change according to the technical solution and the inventive concept of the present application within the technical range disclosed by the present application, which should be covered within the protection scope of the present application.

Claims

1. A data-driven based dynamic collaborative transportation operation method for multiple autonomous mobile robots, characterized in that, The dynamic cooperative transportation operation method of the multiple autonomous mobile robots first obtains a feasible global trajectory conforming to the kinematic characteristics and safety of each autonomous mobile robot by using a CL-CBS algorithm; secondly, the velocity information of each trajectory point is obtained by processing the feasible global trajectory by using a velocity planning based on a transition principle; then, the feasible global trajectory with the velocity information is processed by using an artificial potential field method to obtain a specific executable local reference trajectory, and the local reference trajectory is fitted by using a quintic polynomial to obtain a local expected trajectory; finally, the local expected trajectory is tracked by using a model predictive control algorithm; meanwhile, the control parameters of the model predictive control algorithm are adaptively adjusted by using reinforcement learning to realize accurate tracking of the local expected trajectory and complete the dynamic cooperative transportation operation of the multiple autonomous mobile robots; the specific steps are as follows: Step 1: according to the real scene, a simulation environment is built, and a CL-CBS algorithm is used to find a feasible global trajectory for the multiple autonomous mobile robots; the feasible global trajectory finding problem is decomposed into upper and lower algorithms: the lower algorithm is used to plan a global reference trajectory, and the global reference trajectory is transmitted to the upper algorithm; the upper algorithm is used to eliminate the conflicts in the global reference trajectory, and the conflict elimination scheme is transmitted back to the lower algorithm; the lower algorithm re-plans the global reference trajectory according to the conflict elimination scheme, and the global reference trajectory is re-transmitted to the upper algorithm, and then the cycle between the upper algorithm and the lower algorithm is performed in turn until there is no conflict in the global reference trajectory, and the feasible global trajectory of each autonomous mobile robot is found; the performance of the CL-CBS algorithm is evaluated by three evaluation indexes; Step 2: the feasible global trajectory of each autonomous mobile robot is obtained by step 1, and a velocity planning based on the transition principle is used for each autonomous mobile robot to improve the running accuracy of the autonomous mobile robot; Step 3: after obtaining the feasible global trajectory with velocity information, collision avoidance processing is performed; first, a point mass dynamics model is established to provide a model basis for optimization solution; secondly, the artificial potential field method is used to inflate the obstacles and autonomous mobile robots, and they are integrated into the objective function of the optimization solution to find a safe local reference trajectory; then, a quintic polynomial is used to fit the local reference trajectory to obtain a local expected trajectory; Step 4: after obtaining the local expected trajectory, the local expected trajectory is tracked; according to the dynamic characteristics of the autonomous mobile robot, a model predictive control algorithm based on a three-degree-of-freedom dynamics model is established, and the difference between the left and right wheel steering angles caused by the Ackerman steering geometry is ignored; the final objective function is established as follows: （17）； where Q mpc is the weight matrix of the extended state variable, R mpc is the weight matrix of the control increment matrix; p is the weight coefficient, and e is the relaxation factor; represents the execution trajectory calculated by the model predictive control algorithm in the t+k time period starting from the sampling time t; represents the local desired trajectory in the t+k time period starting from the sampling time t; represents the control increment matrix in the t+k time period starting from the sampling time t; represents the prediction time domain, represents the control time domain; k represents the prediction step; and U represents the control variable; represents the minimum value of the control increment, respectively represents the maximum value of the control increment; represents the minimum value of the control variable, respectively represents the maximum value of the control variable; in addition, , represents the N u order unit matrix, and D represents the weight matrix of the control variable constraint; Step 5: Adjust the weight matrix Q of formula (17) in real time mpc and R mpc , obtain the optimal weight matrix Q mpc and R mpc ; an Actor-Critic framework is adopted, which is divided into two parts of Actor and Critic; the Actor part is composed of a target policy network and an online policy network, and is used for estimating a deterministic policy function; the Critic part is composed of an online Q network and a target Q network, and evaluates the value of the current action through the reward signal fed back by the environment and the estimation of the next state of the autonomous mobile robot, so as to update and adjust the policy of the Actor part; Step 6: updating the weight matrix Q in the objective function as shown in formula (17) by a reinforcement learning based parameter tuning method mpc and R mpc , and solving the optimization problem as shown in formula (17) to obtain the optimal control increment sequence in the control time domain; the weight matrix of the model predictive control is adjusted in real time through steps 5-6 to obtain the optimal control amount of the autonomous mobile robot; Step 7: the optimal control quantity output by step 6 is transmitted to each autonomous mobile robot, and the state information of each autonomous mobile robot is output; on this basis, the state information of each autonomous mobile robot is transmitted to the local trajectory planning module and the tracking control module, the state information of the next step is updated, and the dynamic cooperative transportation operation is performed until the dynamic cooperative transportation operation of the multiple autonomous mobile robots is completed.

2. The data-driven dynamic cooperative transportation operation method for multiple autonomous mobile robots according to claim 1, wherein, The upper algorithm in step 1 adopts a binary tree to search for conflicts, and the lower algorithm adopts a penalty temporal-spatial hybrid A-star algorithm based on the low-energy principle; step 1 specifically comprises the following steps: Step 1.1 : Construct a structured virtual map according to a real warehouse logistics scenario, in which clusters and randomly scattered obstacles are set, the structured virtual map is defined as a continuous workspace ; Step 1.2: define the graph theory basis of the CL-CBS algorithm, let the multiple autonomous mobile robots work in the continuous workspace , the area occupied by the obstacle is , so the free working environment of the autonomous mobile robot is ; the real-time state of the autonomous mobile robot is set as , the position occupied by the autonomous mobile robot is ; the i-th autonomous mobile robot Rob i is responsible for reaching the target state from the initial state ; wherein X represents the horizontal coordinate of the autonomous mobile robot in the Cartesian coordinate system, Y represents the vertical coordinate of the autonomous mobile robot in the Cartesian coordinate system, represents the yaw angle of the autonomous mobile robot in the Cartesian coordinate system; The i-th autonomous mobile robot Rob i From an initial state To a target state The trajectory is defined as a feasible global trajectory ; Composed of real-time states of the autonomous mobile robot over continuous time, it can be expressed as ; The following conditions need to be satisfied: (1) feasible global trajectory Should start from initial state , and reach target state after finite time steps ; in addition, the ith autonomous mobile robot Rob i needs to be able to remain at the target point, , t step represent time steps; (2) the ith autonomous mobile robot Rob i should not collide with obstacles while moving along its trajectory, , t denotes the sampling time; In the process of executing tasks, all starting states, target states and execution states of the autonomous mobile robots in the system need to be non-conflicting; the execution framework of the CL-CBS algorithm is obtained through step 1.2; Step 1.3: Based on the space-time hybrid A-star algorithm, a penalty term based on the pose change of the autonomous mobile robot is set to form a penalty space-time hybrid A-star algorithm as the lower algorithm of the CL-CBS algorithm, which searches and expands the trajectory nodes in the free working environment , and forms an open list through the expanded trajectory nodes; the open list forms a tuple with the information of the trajectory nodes , which means that the i-th autonomous mobile robot Rob i is in the state at the sampling time t; wherein N.f represents the total cost of the current trajectory node; N.g represents the cost from the initial state to the current state ; N.h represents the heuristic cost; in addition, the relationship among N.f, N.g and N.h is as follows: （1）； where N.hmay be further expressed as the current state to the goal state cost, which is computed as follows: （2）； wherein h RS represents the minimum cost of connecting the current state to the target state by using a Reed-Shepp curve, represents the Euclidean distance between the two; wherein the Reed-Shepp curve is the shortest curve connecting two points by using circular arcs and straight lines; In order to improve the running accuracy of the autonomous mobile robots, N.g is punished, which is calculated as follows: （3）； where w turn is a first penalty coefficient based on the pose change, w dir is a second penalty coefficient based on the pose change; w turn and w dir are manually set, and , if a transition between forward and backward occurs, w if a transition between left and right occurs, w if no transition occurs, w ; then, the total cost of the current trajectory node is further calculated as follows: （4）； Thus, the tuple may be further expressed as by constantly updating the tuple The lower layer algorithm obtains the global reference trajectory of each autonomous mobile robot, and whether the global reference trajectory is feasible needs to be detected by the upper layer algorithm. If there is no conflict, the global trajectory is feasible. Through step 1.3, the lower algorithm of the CL-CBS algorithm can obtain the global reference trajectory of each autonomous mobile robot, and the global reference trajectory is transmitted to the upper algorithm for conflict detection and resolution; Step 1.4: The upper algorithm uses the binary tree detection principle to detect the conflict of the global reference trajectory input by the lower algorithm; if there is no conflict, the CL-CBS algorithm finds a feasible global trajectory for each autonomous mobile robot; if there is a conflict, the upper algorithm imposes constraints on the first conflict; the former requires the ith autonomous mobile robot Rob i Inhibit the entry position at the sampling time t , the latter requires the jth autonomous mobile robot Rob j Inhibit the entry position at the sampling time t The constraint is the conflict resolution scheme; the upper algorithm and the lower algorithm of the CL-CBS will repeatedly perform a loop solution until the trajectories of each autonomous mobile robot do not conflict, that is, a feasible global trajectory of each autonomous mobile robot is found; Step 1.5: After obtaining the feasible trajectory of each autonomous mobile robot, three evaluation indexes are introduced to contribute to data visualization and analysis convenience, and the expressions of the evaluation indexes are as follows: （5）； wherein, represents the total cost value, which is the first evaluation index; represents the CL-CBS longest running time step, which is the second evaluation index; represents the average time step of the agent reaching the target point, which is the third evaluation index; in addition, represents the number of autonomous mobile robots, represents the i-th autonomous mobile robot Rob i from the initial state to the target state total cost, t step represents the time step. 3.The data-driven dynamic collaborative transportation operation method for multiple autonomous mobile robots according to claim 1, wherein, Step 2 obtains the feasible global trajectory with speed information of each autonomous mobile robot, and the specific steps are as follows: Step 2.1: regarding the time nodes at which each waiting action of a certain autonomous mobile robot ends as the key time nodes of the autonomous mobile robot, the feasible global trajectory of the autonomous mobile robot is divided into different time periods according to the key time nodes, and the autonomous mobile robot adopts corresponding speeds in different time periods to avoid collision and waiting action; the key time nodes of each autonomous mobile robot are obtained; Step 2.2: obtaining the first key time node t1 of the autonomous mobile robot, the time period T1 before the first key node occurs, and the time period T2 after the first key node occurs through the key time nodes of each autonomous mobile robot; determining the priority of the autonomous robot: the autonomous mobile robot with waiting action is the secondary priority, and the autonomous mobile robot without waiting action is the highest priority; the longitudinal velocity v of the autonomous mobile robot with the highest priority x takes the preset ideal velocity v e , the longitudinal velocity v of the autonomous mobile robot with the secondary priority x takes the velocity v p , and the selection range of v p is calculated as follows: （6）； wherein, represents a speed weighting coefficient, and ; represents a length of a trajectory of the autonomous mobile robot in a phase, represents a length of the autonomous mobile robot; By step 2.2 the sub-priority autonomous mobile robot and the highest-priority autonomous mobile robot are in the longitudinal velocity v of the phase x ; Step 2.3: After the autonomous mobile robot rated as the second priority passes the first critical time node, the priority needs to be re-evaluated; in particular, if the critical time node continues to appear, the autonomous mobile robot is still the second priority, and the speed is re-taken according to formula (6); if there is no critical node in front, the longitudinal speed of the autonomous mobile robot returns to the preset ideal longitudinal speed v e ; the autonomous mobile robot safely passes each critical time node in turn; Through step 2, the feasible global trajectory with speed information of each autonomous mobile robot can be obtained.

4. The data-driven based dynamic cooperative transportation operation method for multiple autonomous mobile robots according to claim 3, wherein, In step 2.2, in order to select v p By formula (6), different speed reference values are set in the selected range according to a speed interval of 0.5 m / s, and the speed reference values make the sub-priority autonomous mobile robot select a suitable longitudinal speed.

5. The data-driven based dynamic cooperative transportation operation method for multiple autonomous mobile robots according to claim 1, wherein, Step 3 specifically comprises the following steps: Step 3.1: ignoring the size information of each autonomous mobile robot and the load transfer caused by transverse and longitudinal acceleration, a point mass dynamics model of the autonomous mobile robot is established, which is represented as follows: （7）； On the basis of point mass dynamics model, considering the dynamics constraints, adding constraint conditions where u represents the control variable, i.e., the front wheel steering angle w; then formula (7) is further expressed as follows: （8）； wherein, represents a state quantity, represents a first derivative of the sampling time t, is represented by and a y consists of; represents a longitudinal velocity of the autonomous mobile robot in a body coordinate system, represents a lateral velocity of the autonomous mobile robot in the body coordinate system, represents a yaw angle of the autonomous mobile robot in a Cartesian coordinate system, Y represents a lateral coordinate of the autonomous mobile robot in the Cartesian coordinate system, and X represents a longitudinal coordinate of the autonomous mobile robot in the Cartesian coordinate system, and all represent longitudinal accelerations of the autonomous mobile robot in the body coordinate system, represents a lateral acceleration of the autonomous mobile robot in the body coordinate system; represents a first derivative of Y with respect to the sampling time t, represents a first derivative of X with respect to the sampling time t, represents a first derivative of the sampling time t; Step 3.2: while establishing the point mass dynamics model, an artificial potential field function based on the positions of obstacles and autonomous mobile robots is established, wherein the obstacles include not only the obstacles in the virtual map but also the remaining autonomous mobile robots except the body autonomous mobile robot; through the establishment of the artificial potential field function, the autonomous mobile robot can avoid the area with large repulsive force, i.e. the obstacle, in the optimization solution; Step 3.3: Obtain the optimized local reference trajectory; evaluate the objective function of the local trajectory planning layer is represented as follows: （11）； where Q local represents the first weight matrix of the objective function, R local represents the second weight matrix of the objective function, Q local and R local need to be set manually; J obs.t is the artificial potential field function value at sampling time t, represents the prediction horizon, represents the control horizon; represents the local reference trajectory in the time period of t+m starting from sampling time t, which is calculated by solving the optimization; represents the feasible global trajectory in the time period of t+m starting from sampling time t; represents the control increment matrix in the time period of t+m starting from sampling time t; represents the control amount matrix at sampling time t, represents the minimum value of the control amount matrix, represents the maximum value of the control amount matrix, and need to be set manually; in addition, , represents a column vector with rows, represents the control amount at sampling time t-1; Step 3.4: after obtaining the optimized local reference trajectory, a quintic polynomial is adopted to fit the local reference trajectory to obtain a local expected trajectory, which is as follows: （12）； where a n a fitting parameter representing the expected lateral coordinate of the local trajectory, n = 0, 1, 2, 3, 4, 5; b n a fitting parameter representing the expected yaw angle of the local trajectory; Y local a fitting parameter representing the expected longitudinal coordinate of the local trajectory, φ local a fitting parameter representing the expected yaw angle of the local trajectory.

6. The data-driven based dynamic cooperative transportation operation method for multiple autonomous mobile robots according to claim 5, wherein, In step 3.2, the artificial potential field function is as follows: （9）； wherein w oa represents a global control weight, w dis represents a distance weight, w oa and w dis need to be set manually, and , ; represents a repulsion threshold of the artificial potential field function, r g is a radiation radius of the artificial potential field function, and r g need to be set manually, and , ; represents a repulsion value of the artificial potential field, represents a velocity of the autonomous mobile robot; E dis represents a Euclidean distance between the body of the autonomous mobile robot and an obstacle, which can be represented as wherein, represents a horizontal coordinate of the obstacle in a Cartesian coordinate system, represents a vertical coordinate of the obstacle in the Cartesian coordinate system; It can be found that according to formula (9), when v is near zero, J obc will also tend to zero, which will make the autonomous mobile robot cross the obstacle, so it is necessary to constrain v in the artificial potential field function to meet the reasonable expansion of the obstacle, so v is defined as follows: （10）。 7. The data-driven based dynamic cooperative transportation operation method for multiple autonomous mobile robots according to claim 1, wherein, Step 4 specifically comprises the following steps: Step 4.1: a three-degree-of-freedom dynamics model of the autonomous mobile robot is established, and the dynamics equation is as follows: （13）； in, represents the cornering stiffness of the front wheel, Represents the cornering stiffness of the rear wheel; represents the longitudinal stiffness of the front wheel, Represents the longitudinal stiffness of the rear wheel; l f Represents the distance from the front axle to the center of mass, l r represents the distance from the rear axle to the center of mass, m represents the mass of the autonomous mobile robot, γ represents the yaw angular velocity, and w represents the front wheel angle; I z Represents the moment of inertia around the z-axis, and the z-axis represents the vertical direction in the vehicle coordinate system. the axis of a plane; Through step 4.1, the three-degree-of-freedom dynamics model of the autonomous mobile robot is obtained; Step 4.2: the three-degree-of-freedom dynamics model in formula (13) is linearized, and the linearized state space is as follows: （14）； wherein , represents the state quantity under the three-degree-of-freedom dynamic model; represents the first order derivative of the sampling time t, and u represents the control quantity, i.e. the front wheel rotation angle w; , represents the output quantity; A, B, and C are coefficient matrices, which can be directly obtained through linearization processing; Through step 4.2, the linear state space of the autonomous mobile robot is obtained; Step 4.3: the linear state space shown in formula (14) is discretized by using the forward Euler method, and the discretized form is as follows: （15）； wherein , ; represents a sampling period, which needs to be set manually; A k represents a weight matrix of the state quantity, B k represents a weight matrix of the control quantity, I6represents a 6-order unit matrix; represents the state quantity at the sampling time t+1, represents the state quantity at the sampling time t, represents the control quantity at the sampling time t; A discretized linear state space of the autonomous mobile robot is obtained through step 4.3; Step 4.4: After getting the discretized linear state space, the state dimension is extended to ; where, represents the control variable at sampling time t-1, represents the extended state variable at sampling time t; then, the new state space is as follows: （16）； wherein, , an augmented parameter matrix representing the extended state variable, wherein, represents an N u row N x column zero matrix, represents an N u order identity matrix; , an augmented parameter matrix representing the control increment; , an augmented parameter matrix representing the output variable, wherein, represents an N y row N u column zero matrix; N u represents the number of control variables, N x represents the number of state variables, N y represents the number of output variables, N u , N x and N y need to be manually set; represents the control increment at sampling time t, represents the extended output variable at sampling time t; An extended state space of the autonomous mobile robot is obtained through step 4.4; Step 4.5: After obtaining the extended state space, a target function is set for the model predictive control algorithm; the optimization goal is to minimize the error between the model predictive control algorithm output trajectory and the local desired trajectory, the optimization goal is to make the control quantity as small as possible, and a soft constraint is added; finally, the target function as shown in formula (17) is obtained; The autonomous mobile robot is controlled to track the local desired trajectory through step 4, and the optimal control quantity is output. 8.The data-driven dynamic collaborative transportation operation method for multiple autonomous mobile robots according to claim 1, wherein, The step 5 is specifically: The actor part generates an action according to the current environment, the environment feeds back the next state and reward to the actor part, and the action is transmitted to the critic part for evaluation; the critic part evaluates the action according to the online policy network and the target policy network after receiving the state, and transmits the evaluation result to the actor part, so that the action of the actor part is continuously optimized to obtain the maximum reward value; at the same time, the experience recycling pool stores information and samples; By the above process, i.e., forming a data-driven autonomous learning strategy, the optimal action, i.e., the weight matrix Q mpc and R mpc ; moreover, the autonomous learning strategy will pass the optimal weight matrix Q mpc and R mpc to the objective function of formula (17), so that the autonomous mobile robot can accurately track the local desired trajectory.

9. The data-driven based dynamic cooperative transportation operation method for multiple autonomous mobile robots according to claim 1, wherein, In step 6, the first control increment in the optimal control increment sequence is obtained as the control increment As the actual control increment, the control increment is obtained as follows: （18）； wherein, , represents a front wheel steering angle at a sampling time t-1; , represents a front wheel steering angle at a sampling time t-1; , represents a front wheel steering angle increment desired at a sampling time t-1.

Images