Dynamic collaborative transportation operation method for multiple autonomous mobile robots based on data driving

Abstract

The invention provides a multi-autonomous mobile robot dynamic collaborative transportation operation method based on data driving, and belongs to the field of trajectory planning and tracking control of multi-autonomous mobile robots. The method comprises the following steps: firstly, planning a feasible global trajectory conforming to kinematics characteristics and safety of each autonomous mobile robot; secondly, obtaining speed information of each track point; then, processing the feasible global trajectory with speed information to obtain a local reference trajectory which can be specifically executed, and fitting the local reference trajectory to obtain a local expected trajectory; and finally, the local expected trajectory is tracked, control parameters are adjusted, and the dynamic collaborative transportation operation of the multiple autonomous mobile robots is completed. The robot trajectory tracking precision and stability can be improved; through real-time local trajectory planning and a self-adaptive parameter adjustment method based on reinforcement learning, the safety and the anti-interference capability of a multi-autonomous mobile robot system are improved while the dynamic collaborative transportation operation of multiple autonomous mobile robots is ensured.

Description

Technical Field

[0001] The invention belongs to the field of trajectory planning and tracking control of multiple autonomous mobile robots, and relates to a dynamic cooperative transportation operation method of multiple autonomous mobile robots with certain prior environmental information. Background Art

[0002] Autonomous mobile robots are widely used in various real-world scenarios, providing strong support for productivity improvement, technology iteration, and capacity expansion in related fields, such as warehouse management, factory inspection, and campus transportation. In the process of transport operations of multiple autonomous mobile robots, there are a large number of safety constraints and related environmental restrictions caused by avoiding other autonomous mobile robots, which will seriously affect the safety of the multi-autonomous mobile robot system and the efficiency of transport task execution. Therefore, it is of great practical significance to develop a trajectory planning and tracking control method for dynamic collaborative transport operations of multiple autonomous mobile robots based on data-driven.

[0003] The paper "CL-MAPF: Multi-agent path finding for car-like robots with kinematic and spatiotemporal constraints" published in "Robotics and Autonomous Systems" Vol. 150, 2022, proposed a multi-intelligent robot trajectory planning method based on the combination of spatiotemporal hybrid A-star algorithm and conflict search algorithm. This method integrates the Ackerman steering feature into trajectory planning, significantly improving the executableness of the intelligent robot for the trajectory; however, the traditional trajectory planning algorithm simply considers the running speed of the intelligent agent as a constant value and the waiting speed as zero, which seriously limits the efficiency of multi-autonomous mobile robots. Therefore, speed planning based on the principle of jump is proposed to plan the speed while tracking the trajectory. In addition, the paper "Path Planning and Trajectory Tracking for AutonomousObstacle Avoidance in Automated Guided Vehicles at Automated Terminals" published in "Axioms" Vol. 13, No. 1, 2023 proposed an autonomous obstacle avoidance path and trajectory tracking control scheme based on an automated guided vehicle, which significantly improved the task execution safety of the automated guided vehicle. However, the traditional model predictive control method based on artificial potential field method has certain limitations in complex environments due to its fixed parameters. Therefore, a parameter adaptation method based on reinforcement learning is proposed. Summary of the invention

[0004] In view of the above-mentioned problems existing in the prior art, in order to enable the multi-autonomous mobile robot system to be applied in different scenarios, and on the basis of ensuring the internal and external security of the system and the stability of task execution, the present invention proposes a data-driven dynamic collaborative transportation operation method of multiple autonomous mobile robots.

[0005] To achieve the above objectives, the present invention adopts the following technical solutions.

[0006] A data-driven dynamic collaborative transportation operation method for multiple autonomous mobile robots is disclosed. The dynamic collaborative transportation operation method for multiple autonomous mobile robots can realize autonomous global trajectory planning, local trajectory planning and trajectory tracking control in a structured warehousing and logistics scenario. It is a dynamic collaborative transportation operation method for multiple autonomous mobile robots based on reinforcement learning, car-like conflict-based search algorithm (CL-CBS) and artificial potential field method-model predictive control algorithm (APF-MPC). The dynamic collaborative transportation operation method for multiple autonomous mobile robots first uses the CL-CBS algorithm to obtain a feasible global trajectory that meets the kinematic characteristics and safety of each autonomous mobile robot. Secondly, the feasible global trajectory is processed by speed planning based on the transition principle to obtain the speed information of each trajectory point. Then, the feasible global trajectory with speed information is processed by the artificial potential field method to obtain a specific executable local reference trajectory, and the local reference trajectory is fitted with a fifth-order polynomial to obtain a local desired trajectory. Finally, the local desired trajectory is tracked by the model predictive control algorithm; at the same time, the control parameters of the model predictive control algorithm are adaptively adjusted by reinforcement learning to achieve accurate tracking of the local desired trajectory and complete the dynamic collaborative transportation operation of multiple autonomous mobile robots. The specific steps include:

[0007] Step 1: Build a simulation environment according to the real scene, and use the car-based conflict search algorithm (CL-CBS) to find feasible global trajectories for multiple autonomous mobile robots. The CL-CBS algorithm decomposes the problem of finding a feasible global trajectory into two algorithms: the lower algorithm is responsible for planning the global reference trajectory and transmitting the global reference trajectory to the upper algorithm; the upper algorithm is responsible for resolving conflicts in the global reference trajectory and transmitting the conflict resolution solution back to the lower algorithm. In addition, the lower algorithm will replan the global reference trajectory according to the conflict resolution solution, and retransmit the global reference trajectory to the upper algorithm, and then cycle between the upper and lower algorithms in turn until there is no conflict in the global reference trajectory, that is, the feasible global trajectory of each autonomous mobile robot is found. It should be emphasized that the upper algorithm uses a binary tree for conflict search, and the lower algorithm uses a penalty-based spatiotemporal hybrid A-star algorithm based on the principle of low energy consumption. In addition, three evaluation indicators are introduced to accurately evaluate the performance of the CL-CBS algorithm. Specifically, it includes the following steps:

[0008] Step 1.1: Construct a structured virtual map based on the real warehouse logistics scenario, in which clustered and randomly scattered obstacles are set to improve the map complexity and authenticity, and provide a credible environment foundation for the implementation of dynamic collaborative transportation operation methods; in addition, the structured virtual map is defined as a continuous workspace

[0009] Then, through step 1.1, we can get the continuous space

[0010] Step 1.2: Define the graph theory basis of the CL-CBS algorithm and let multiple autonomous mobile robots move in a continuous workspace. The area occupied by the obstacle is Therefore, the free working environment of the autonomous mobile robot is Set the real-time status of the autonomous mobile robot to The position it occupies is The i-th autonomous mobile robot Rob i Responsible for starting from the initial state Reach the target state Wherein, X represents the horizontal coordinate of the autonomous mobile robot in the Cartesian coordinate system, and Y represents the vertical coordinate of the autonomous mobile robot in the Cartesian coordinate system. Represents the yaw angle of the autonomous mobile robot in the Cartesian coordinate system.

[0011] In addition, the i-th autonomous mobile robot Rob i From the initial state s i Arriving at the target state g i The trajectory of i ; π i It consists of the real-time state of the autonomous mobile robot in continuous time, so it can be expressed as In addition, π i The following conditions must be met:

[0012] (1) Feasible global trajectory π i Should start from the initial state π i [0] = s i , and reaches the target state π after a finite time step i [t] = g i ; In addition, the i-th autonomous mobile robot Rob i Need to be able to stay at the target point, t step represents the time step;

[0013] (2) The i-th autonomous mobile robot Rob i While moving along its trajectory, it should not collide with obstacles. t represents the sampling time.

[0014] Specifically, when the autonomous mobile robot is performing a task, it is required that the starting states, target states and execution states of all autonomous mobile robots within the system do not conflict.

[0015] Then, through step 1.2, we can get the execution framework of the CL-CBS algorithm.

[0016] Step 1.3: Based on the space-time hybrid A-star algorithm, a penalty term based on the position change of the autonomous mobile robot is set to form a penalty space-time hybrid A-star algorithm, which is used as the lower layer algorithm of the CL-CBS algorithm. The Ackerman steering characteristics of the autonomous mobile robot are introduced into the global trajectory planning through the penalty space-time hybrid A-star algorithm, which can make the autonomous mobile robot have better executableness for the global reference trajectory. Specifically, the algorithm is in a free working environment. The trajectory nodes are searched and expanded, and an open list is formed through the expanded trajectory nodes. The open list forms a tuple of the information of the trajectory nodes. This tuple means that at the current trajectory node, the i-th autonomous mobile robot Rob i The state at sampling time t is z i ; Among them, Nf represents the total cost of the current trajectory node; Ng represents the total cost from the initial state s i To the current state z i The cost of Nf, Ng and Nh is the heuristic cost. In addition, the relationship between Nf, Ng and Nh can be expressed as follows:

[0017] Nf=N.g+Nh (1)

[0018] Among them, Nh can be further expressed as the current state z i To the target state g i The cost is calculated as follows:

[0019]

[0020] Among them, h RS Represents the minimum cost of connecting the current state to the target state using the Reed-Shepp curve. represents the Euclidean distance between the two points; the Reed-Shepp curve is the shortest curve connecting two points using an arc and a straight line, which was proposed in the paper "Optimal Paths for a Car that Goes Both Forwardsand Backwards" published by JAReeds. In addition, in order to improve the operating accuracy of the autonomous mobile robot, a penalty is imposed on Ng, which is calculated as follows:

[0021] Ng pen =w turn w dir Ng (3)

[0022] Among them, w turn is the first penalty coefficient based on posture change, w dir is the second penalty coefficient based on posture change; w turn and w dir All need to be set manually, and w turn ≥1,w dir ≥1; specifically, w turn Investigate whether the autonomous mobile robot switches between forward and backward, w dir Investigate whether the autonomous mobile robot switches between left and right directions. If the robot switches between forward and backward, use w turn >1; if a switch between left and right directions occurs, w is used dir >1; if no conversion occurs, then w turn =w dir = 1. Then, the total cost of the current trajectory node can be further calculated as follows:

[0023] Nf=Ng pen +Nh (4)

[0024] Therefore, the tuple It can be further expressed as By continuously updating the tuple The lower-level algorithm can obtain the global reference trajectory of each autonomous mobile robot. Whether the global reference trajectory is feasible requires conflict detection based on the upper-level algorithm. If there is no conflict, it is a feasible global trajectory.

[0025] Then, through step 1.3, the lower-level algorithm of the CL-CBS algorithm can obtain the global reference trajectory of each autonomous mobile robot and transmit the global reference trajectory to the upper-level algorithm for conflict detection and resolution.

[0026] Step 1.4: The upper-level algorithm uses the binary tree detection principle to perform conflict detection on the global reference trajectory input by the lower-level algorithm. Specifically, the upper-level algorithm will traverse the preliminary global reference trajectory based on the binary tree to detect whether the global reference trajectory is a feasible global trajectory. If there is no conflict, it means that the CL-CBS algorithm has found a feasible global trajectory for each autonomous mobile robot. If there is a conflict, the upper-level algorithm will impose constraints on the conflict that occurs first, that is, (a i ,N.π j [t],t) and (a j ,N.π i [t],t); the former requires the i-th autonomous mobile robot Rob i No entry position at sampling time t The latter requires the jth autonomous mobile robot Rob j No entry position at sampling time t This constraint is the conflict resolution solution. Furthermore, the upper-level algorithm transmits the conflict resolution solution back to the lower-level algorithm, and the lower-level algorithm re-plans the global reference trajectory for each autonomous mobile robot based on the resolution solution. In addition, the upper-level and lower-level algorithms of CL-CBS will repeatedly solve the problem in a cycle until the trajectories of each autonomous mobile robot do not conflict, that is, a feasible global trajectory for each autonomous mobile robot is found.

[0027] Then, through steps 1.3 to 1.4, that is, through the cyclic solution of the lower algorithm and the upper algorithm of the CL-CBS algorithm, the feasible global trajectory of each autonomous mobile robot can be found.

[0028] Step 1.5: After obtaining the feasible trajectories of each autonomous mobile robot, in order to specifically and accurately evaluate the performance of the CL-CBS algorithm, three evaluation indicators are introduced to contribute to data visualization and analysis convenience. The expressions of the evaluation indicators are as follows:

[0029]

[0030] Among them, N cost Represents the total generation value, which is the first evaluation indicator; T makespan represents the longest running time step of CL-CBS, which is the second evaluation index; T averageflow represents the average time step for the agent to reach the target point, which is the third evaluation indicator. In addition, n AMR represents the number of autonomous mobile robots, Nf(i) represents the i-th autonomous mobile robot Rob i From the initial state s i To the target state g i The total cost, t step Represents the time step.

[0031] Through step 1, a feasible global trajectory of the multiple autonomous mobile robots can be obtained. Step 1 is a global trajectory planning module of the multiple autonomous mobile robots.

[0032] Step 2: Obtain the feasible global trajectory of each autonomous mobile robot through step 1, and adopt a speed planning based on the jump principle for each autonomous mobile robot. The main purpose of the speed planning based on the jump principle is to reduce the situation where different autonomous mobile robots have to wait for action due to the intersection of feasible global trajectories, and try to keep the robot in a forward motion state to improve the operation accuracy of the autonomous mobile robot. The specific steps are as follows:

[0033] Step 2.1: According to the feasible global trajectory output by the CL-CBS algorithm, a speed planning method based on the transition principle is proposed. By analyzing the feasible global trajectory, the time nodes of the appearance and end of the waiting action of each autonomous mobile robot can be directly obtained, and then the time period is finely divided according to the time node of the end of the waiting action. Specifically, the time node of the end of each waiting action of a certain autonomous mobile robot is used as the key time node of the autonomous mobile robot, and then the feasible global trajectory of the autonomous mobile robot is divided into different time periods according to the key time nodes, so that the autonomous mobile robot adopts the corresponding speed in different time periods to avoid collision and waiting action.

[0034] Then, through step 2.1, the key time nodes of each autonomous mobile robot can be obtained.

[0035] Step 2.2: Through the key time nodes of each autonomous mobile robot, we can get the first key time node t1 of the autonomous mobile robot, the time period T1 before the first key node, and the time period T2 after the first key node. In addition, in order to facilitate different autonomous mobile robots to choose the appropriate speed, it is necessary to determine the priority of the autonomous robots; specifically, autonomous mobile robots with waiting actions are all secondary priorities, and autonomous mobile robots without waiting actions are all the highest priorities. The longitudinal speed v of the highest priority autonomous mobile robot x Take the preset ideal speed v e , the longitudinal speed v of the next priority autonomous mobile robot x Take speed v p , v p The selection range is calculated as follows:

[0036]

[0037] Among them, δ v represents the velocity weighting coefficient, and 0≤δ v ≤1;Ls represents the trajectory length of the autonomous mobile robot in stage T1, and L represents the length of the autonomous mobile robot.

[0038] On this basis, in order to facilitate v p The selection of different speed reference values ​​is sequentially set in the selection range according to the speed interval of 0.5 m / s by formula (6), and the speed reference value enables the second-priority autonomous mobile robot to select a suitable longitudinal speed. It should be noted that for the second-priority autonomous mobile robots with feasible global trajectory intersections, the time nodes for waiting for the end of the action are different, so there is no collision risk between the second-priority autonomous mobile robots.

[0039] Then, through step 2.2, the longitudinal speed v of the second priority autonomous mobile robot and the highest priority autonomous mobile robot in stage T1 can be obtained: x .

[0040] Step 2.3: After the autonomous mobile robot that is rated as a low priority passes the first critical time node, the priority needs to be re-evaluated. Specifically, if the critical time node continues to appear, the autonomous mobile robot is still a low priority and re-adopts the speed according to formula (6); if there is no more critical node ahead, the longitudinal speed of the autonomous mobile robot is restored to the preset ideal longitudinal speed v e Then, the autonomous mobile robot can safely pass through each key time node in sequence.

[0041] Through step 2, a feasible global trajectory of each autonomous mobile robot with speed information can be obtained.

[0042] Step 3: After obtaining a feasible global trajectory with speed information, anti-collision processing is required to prevent collisions caused by deviations between the local reference trajectory output by local trajectory planning and the feasible global trajectory. First, a point mass dynamics model is established to provide a model basis for the optimization solution. Secondly, the artificial potential field method is used to expand obstacles and autonomous mobile robots, and they are integrated into the objective function of the optimization solution to find a safe local reference trajectory. Then, an objective function is established to achieve collision avoidance. Finally, in order to further smooth the local reference trajectory, a quintic polynomial is used to fit the local reference trajectory to obtain the local expected trajectory. The specific steps are as follows:

[0043] Step 3.1: Ignoring the size information of each autonomous mobile robot and the load transfer caused by the lateral and longitudinal accelerations, a point mass dynamics model of the autonomous mobile robot is established. The point mass dynamics model can be expressed as follows:

[0044]

[0045] Based on the point-mass dynamics model, further consider the dynamic constraints, that is, add the constraint condition |a y | < ug, where u represents the control quantity, that is, the front wheel steering angle w; then formula (7) can be further expressed as follows:

[0046]

[0047] Among them, represents the state quantity, represents the first derivative of ξ(t) with respect to the sampling time t, f(·) represents a function that can be obtained through calculation, and f(·) is composed of ξ(t) and a y v y represents the longitudinal velocity of the autonomous mobile robot in the vehicle body coordinate system, v x represents the lateral velocity of the autonomous mobile robot in the vehicle body coordinate system, represents the yaw angle of the autonomous mobile robot in the Cartesian coordinate system, Y represents the abscissa of the autonomous mobile robot in the Cartesian coordinate system, and X represents the ordinate of the autonomous mobile robot in the Cartesian coordinate system, and a y both represent the longitudinal acceleration of the autonomous mobile robot in the vehicle body coordinate system, represents the lateral acceleration of the autonomous mobile robot in the vehicle body coordinate system; represents the first derivative of Y with respect to the sampling time t, represents the first derivative of X with respect to the sampling time t, represents the first derivative of with respect to the sampling time t.

[0048] Then, through step 3.1, the point-mass dynamics model of the autonomous mobile robot can be obtained, which is the basis for performing the optimization solution.

[0049] Step 3.2: While establishing the point-mass dynamics model, it is necessary to establish an artificial potential field function based on the positions of the obstacles and the autonomous mobile robot. The purpose of the artificial potential field function is to adjust the magnitude of the repulsive force of the artificial potential field function by calculating the distance between the obstacle and the autonomous mobile robot itself; among them, the obstacles include not only the obstacles in the virtual map but also the other autonomous mobile robots except the autonomous mobile robot itself. By establishing the artificial potential field function, the autonomous mobile robot can avoid the area with a large repulsive force in the optimization solution, that is, avoid the obstacles. The artificial potential field function is calculated as follows:

[0050]

[0051] where w oa represents the global regulation weight, wdis Represents the distance weight, w oa and w dis Need to be set manually, and w oa >0,w dis >0; Г represents the repulsive force threshold of the artificial potential field function, r g is the radiation radius of the artificial potential field function, Г and r g Need to be set manually, and Г≥1, r g ≥1; J obc represents the repulsive force value of the artificial potential field, v represents the speed of the autonomous mobile robot; E dis represents the Euclidean distance between the autonomous mobile robot and the obstacle, which can be expressed as E dis =((XX obc ) 2 +(YY obc ) 2 ) 1 / 2 , where X obc Represents the horizontal coordinate of the obstacle in the Cartesian coordinate system, Y obc Represents the vertical coordinate of the obstacle in the Cartesian coordinate system.

[0052] It is not difficult to find that according to formula (9), when v is near zero, J obc The value of will also approach zero, which will make the autonomous mobile robot cross obstacles. Therefore, it is necessary to constrain v in the artificial potential field function to meet the reasonable expansion of obstacles. Therefore, v is defined as follows:

[0053]

[0054] Then, through step 3.2, the real-time repulsive force value of the autonomous mobile robot during operation can be obtained, which is also the basis for the optimization solution.

[0055] Step 3.3: In order to facilitate the integration of the artificial potential field function into the optimization solution, the point mass dynamics model is directly used for solution, and the forward Euler method is used for discretization. In addition, since the control goal of local trajectory planning is to minimize the deviation from the feasible global trajectory and achieve obstacle avoidance, the objective function J of the local trajectory planning layer can be p It is expressed as follows:

[0056]

[0057] Among them, Q local Represents the first weight matrix of the objective function, R local The second weight matrix representing the objective function, Q local and R local Manual setting is required; Jobs.t is the value of the artificial potential field function at sampling time t, N p represents the prediction time domain, N c represents the control time domain. η(t+m|t) represents the local reference trajectory in the time period t+m calculated by the optimization solution starting from the sampling time t; η ref (t+m|t) represents the feasible global trajectory within the time period t+m starting from the sampling time t; ΔU(t+m|t) represents the control increment matrix within the time period t+m starting from the sampling time t; U t Represents the control quantity matrix at sampling time t, U min Represents the minimum value of the control matrix, U max Represents the maximum value of the control matrix, U min and U max Manual setup is required. In addition, Represents the number of rows as N c The column vector of , u(t-1) represents the control amount at sampling time t-1.

[0058] Then, through step 3.3, the optimized local reference trajectory can be obtained.

[0059] Step 3.4: After obtaining the optimized local reference trajectory, in order to make the local reference trajectory smoother and considering that the posture change of the autonomous mobile robot is continuous, a quintic polynomial is used to fit the local reference trajectory to obtain the local desired trajectory, which can be expressed as follows:

[0060]

[0061] Among them, a n b represents the fitting parameters of the expected horizontal coordinates of the local trajectory, n = 0, 1, 2, 3, 4, 5; n represents the fitting parameter of the desired yaw angle of the local trajectory; Y local represents the expected ordinate of the local trajectory, Represents the yaw angle of the local desired trajectory.

[0062] Through steps 2 and 3, the local desired trajectory of the multi-autonomous mobile robots can be obtained. Steps 2 and 3 are the local trajectory planning modules of the multi-autonomous mobile robots.

[0063] Step 4: After obtaining the local desired trajectory, it is necessary to track the local desired trajectory. According to the dynamic characteristics of the autonomous mobile robot, a model predictive control algorithm based on a three-degree-of-freedom dynamic model is established, and the difference in the steering angles of the left and right wheels caused by the Ackerman steering geometry is ignored. The specific steps are as follows:

[0064] Step 4.1: Establish a three-degree-of-freedom dynamic model of the autonomous mobile robot. The dynamic equation is as follows:

[0065]

[0066] in, represents the cornering stiffness of the front wheel, Represents the cornering stiffness of the rear wheel; C σf Represents the longitudinal stiffness of the front wheel, C σr Represents the longitudinal stiffness of the rear wheel; l f Represents the distance from the front axle to the center of mass, l r represents the distance from the rear axle to the center of mass, m represents the mass of the autonomous mobile robot, γ represents the yaw angular velocity, and w represents the front wheel turning angle; I z Represents the moment of inertia about the z-axis, and the z-axis represents the axis perpendicular to the ∑xoy plane in the vehicle coordinate system.

[0067] Then, through step 4.1, the three-degree-of-freedom dynamics model of the autonomous mobile robot can be obtained, and the three-degree-of-freedom dynamics model is the basis of the model predictive control algorithm.

[0068] Step 4.2: After obtaining the three-freedom dynamics model, for the convenience of calculation, the nonlinear time-varying model described in formula (13) is linearized. The state space after linear transformation is as follows:

[0069]

[0070] in, Represents the state quantity under the three-degree-of-freedom dynamic model; represents the first-order derivative of χ with respect to the sampling time t, and u represents the control variable, i.e., the front wheel steering angle w; represents the output quantity; A, B, C are coefficient matrices, which can be directly obtained through linearization processing.

[0071] Then, through step 4.2, the linear state space of the autonomous mobile robot can be obtained, which can greatly reduce the difficulty of calculation.

[0072] Step 4.3: Use the forward Euler method to discretize the linear state space shown in formula (14). The discretization form is as follows:

[0073] χ(t+1)=A k χ(t)+B k u(t)(15)

[0074] Among them, A k =I6+T t A, B k =T t B; Tt Represents the sampling period, which needs to be set manually; A k Represents the weight matrix of the state quantity, B k represents the weight matrix of the control quantity, I6 represents the unit matrix of order 6; χ(t+1) represents the state quantity at sampling time t+1, χ(t) represents the state quantity at sampling time t, and u(t) represents the control quantity at sampling time t.

[0075] Then, through step 4.3, the discretized linear state space of the autonomous mobile robot can be obtained, and the discretized linear state space can further reduce the amount of calculation.

[0076] Step 4.4: After obtaining the discretized linear state space, in order to avoid sudden changes in the control quantity, it is necessary to expand the state quantity to ξ(t) = [χ(t)u(t-1)] T ; where u(t-1) represents the control quantity at sampling time t-1, and ξ(t) represents the expanded state quantity at sampling time t. Then, the new state space is as follows:

[0077]

[0078] in, Represents the augmented parameter matrix of the expanded state quantity, where Represents N u Row N x A column zero matrix, Represents N u The identity matrix of order; The augmented parameter matrix representing the control increment; Represents the augmented parameter matrix of the output, where Represents N y Row N u Column zero matrix; N u Represents the number of controlled quantities, N x Represents the number of state quantities, N y Represents the number of outputs, N u 、N x and N y Manual setting is required; Δu(t) represents the control increment at sampling time t, and η(t) represents the expanded dimension output at sampling time t.

[0079] Then, through step 4.4, the extended-dimensional state space of the autonomous mobile robot can be obtained. This extended-dimensional state space can prevent the control quantity from changing suddenly on the basis of the discretized linear state space, so as to improve the stability of tracking the local desired trajectory.

[0080] Step 4.5: After obtaining the expanded state space, it is necessary to set the objective function for the model predictive control algorithm. In order to enable the autonomous mobile robot to track the local desired trajectory and improve the stability of the autonomous mobile robot, it is necessary to minimize the error between the output trajectory of the model predictive control algorithm and the local desired trajectory as the optimization goal; in addition, in order to avoid the control increment being too large and causing the autonomous mobile robot to lose control, it is necessary to minimize the control amount as much as possible as the optimization goal; finally, in order to avoid the sudden change of the control amount and affect the continuity of the control amount, it is necessary to add soft constraints. Therefore, the objective function is as follows:

[0081]

[0082] Among them, Q mpc is the weight matrix of the expanded state quantity, R mpc is the weight matrix of the control increment matrix; ρ is the weight coefficient, ε is the relaxation factor; η(t+k|t) represents the execution trajectory in the t+k time period calculated by the model predictive control algorithm starting from the sampling time t; η ref.p (t+k|t) represents the local desired trajectory within the time period t+k starting from the sampling time t; ΔU(t+m|t) represents the control increment matrix within the time period t+k starting from the sampling time t; ΔU min Represents the minimum value of the control increment, ΔU max Respectively represent the maximum value of the control increment; U min Represents the minimum value of the control amount, U max Respectively represent the maximum value of the control amount; in addition, Represents N u The unit matrix of order, D represents the weight matrix of the control quantity constraint.

[0083] Through step 4, the autonomous mobile robot can be controlled to track the local desired trajectory and output the optimal control amount. Step 4 is the trajectory tracking control module of multiple autonomous mobile robots.

[0084] Step 5: In order to further reduce the deviation between the local trajectory planning and the feasible global trajectory and improve the tracking accuracy of the trajectory tracking control module, it is necessary to adjust the weight matrix Q of formula (17) in real time mpc and R mpc The real-time parameter adjustment method based on reinforcement learning is an autonomous learning strategy with certain intelligence and efficiency. The present invention adopts an Actor-Critic framework, which can better make up for the disadvantage of slow convergence speed of traditional reinforcement learning. The framework is mainly divided into two parts: Actor and Critic. The Actor part consists of a target policy network and an online policy network. This part is used to estimate the deterministic policy function. According to the current information (s t ,a t ,rt ,s t+1 ) and local expected trajectory information, where s t Represents the current state, a t Represents action, r t Rewards, s t+1 represents the state at the next moment; it should be emphasized that the action a of the autonomous learning strategy t is the weight matrix Q mpc and R mpc The Critic part consists of an online Q network and a target Q network. This part evaluates the value of the current action through the reward signal fed back by the environment and the estimation of the next state of the autonomous mobile robot to update and adjust the strategy of the Actor part. It should be noted that the reward of this autonomous learning strategy is based on the tracking of the local expected trajectory, that is, the better the tracking of the local expected trajectory, the higher the reward.

[0085] Specifically, the Actor generates an action based on the current environment, and the environment feeds back the next state and reward to the Actor, and transmits the action to the Critic for evaluation. After receiving the state, the Critic evaluates the action based on the online policy network and the target policy network, and transmits the evaluation result (gradient) to the Actor, so that the Actor's action is continuously optimized, that is, the maximum reward value is obtained. At the same time, the experience recycling pool stores information (s t ,a t ,r t ,s t+1 ), and sampling is performed so that reinforcement learning can break the data correlation. Then, through this autonomous learning strategy, the optimal action can be obtained, that is, the weight matrix Q mpc and R mpc ; In addition, the autonomous learning strategy will optimize the weight matrix Q mpc and R mpc The objective function passed to formula (17) enables the autonomous mobile robot to accurately track the local desired trajectory.

[0086] Then, through step 5, we can get the optimal weight matrix Q mpc and R mpc , which helps the trajectory tracking control module to accurately track the local desired trajectory.

[0087] Step 6: Update the weight matrix Q in the objective function as shown in formula (17) through the parameter adjustment method based on reinforcement learning mpc and R mpc, and solve the optimization problem shown in formula (17), we can get the optimal control increment sequence in the control time domain; further take the first control increment Δu in the optimal control increment sequence as the actual control increment, and get the control quantity as follows:

[0088] u(t)=u(t-1)+Δu(t-1)(18)

[0089] Among them, u(t)=w(t), w(t) represents the front wheel steering angle at sampling time t; u(t-1)=w(t-1), w(t-1) represents the front wheel steering angle at sampling time t-1; Δu(t-1)=Δw(t-1), Δw(t-1) represents the expected front wheel steering angle increment at sampling time t-1.

[0090] By adjusting the weight matrix of the model predictive control in real time through steps 5 to 6, the optimal control amount of the autonomous mobile robot can be obtained. Steps 5 to 6 are the reinforcement learning modules of multiple autonomous mobile robots.

[0091] Step 7: The optimal control quantity outputted in step 6 is passed to each autonomous mobile robot, and the state information of each autonomous mobile robot is outputted. On this basis, the state information of each autonomous mobile robot is passed to the local trajectory planning module and the tracking control module, the state information of the next step is updated, and the dynamic cooperative transportation operation is performed until the dynamic cooperative transportation operation of multiple autonomous mobile robots is completed.

[0092] The beneficial effects of the present invention are:

[0093] (1) The present invention proposes a penalty-based spatiotemporal hybrid A-star algorithm, which reduces the redundant turning and backward actions of autonomous mobile robots according to the penalty amount, and plans the optimal feasible global trajectory according to the task information and the initial and final states, thereby significantly improving the low energy consumption and safety of multiple autonomous mobile robots.

[0094] (2) The present invention proposes a speed planning based on the transition principle, which can autonomously adopt the corresponding speed according to the key time nodes in the feasible global trajectory information. At the same time, in the local trajectory planning module, a fifth-order polynomial is used to smooth the discrete points of the trajectory, which greatly improves the efficiency of the transportation task execution of multiple autonomous mobile robots and helps to complete dynamic collaborative transportation operations.

[0095] (3) The present invention provides an adaptive parameter adjustment method based on reinforcement learning, which drives each autonomous mobile robot to update the weight matrix of the model predictive control algorithm in real time according to the current environment and state through reinforcement learning, thereby significantly improving the trajectory tracking accuracy and stability of the autonomous mobile robot. BRIEF DESCRIPTION OF THE DRAWINGS

[0096] Figure 1It is a framework diagram of trajectory planning and tracking control for dynamic cooperative transportation operations of multiple autonomous mobile robots;

[0097] Figure 2 This is a schematic diagram of speed planning based on the principle of transition;

[0098] Figure 3 It is a three-dimensional artificial potential field function diagram of the virtual warehousing and logistics environment. DETAILED DESCRIPTION

[0099] In order to make the objectives, technical solutions and advantages of the present invention more clear, the present invention is described in detail with reference to the accompanying drawings and embodiments.

[0100] like Figure 1-Figure 3 As shown, the present invention comprises the following steps:

[0101] Step 1: It includes the following steps:

[0102] Step 1.1: Build a structured virtual map based on the actual warehouse logistics scenario, and set clustered and randomly scattered obstacles in it to improve the map complexity and authenticity, such as Figure 1 As shown, a structured virtual map can be established for multiple autonomous mobile robots based on the environmental information in the prior information, and the virtual map provides a credible environmental basis for the implementation of the dynamic collaborative transportation operation method; in addition, the structured virtual map is defined as a continuous workspace

[0103] Then, through step 1.1, we can get the continuous space

[0104] Step 1.2: Define the graph theory basis of the CL-CBS algorithm and let multiple autonomous mobile robots move in a continuous workspace. The area occupied by the obstacle is Therefore, the free working environment of the autonomous mobile robot is Set the real-time status of the autonomous mobile robot to The position it occupies is The i-th autonomous mobile robot Rob i Responsible for starting from the initial state Reach the target state Wherein, X represents the horizontal coordinate of the autonomous mobile robot in the Cartesian coordinate system, and Y represents the vertical coordinate of the autonomous mobile robot in the Cartesian coordinate system. Represents the yaw angle of the autonomous mobile robot in the Cartesian coordinate system.

[0105] In addition, the i-th autonomous mobile robot Rob i From the initial state s i Arriving at the target state g iThe trajectory of i ; π i It consists of the real-time state of the autonomous mobile robot in continuous time, so it can be expressed as In addition, π i The following conditions must be met:

[0106] (1) Feasible global trajectory π i Should start from the initial state π i [0] = s i , and reaches the target state π after a finite time step i [t] = g i ; In addition, the i-th autonomous mobile robot Rob i Need to be able to stay at the target point t step represents the time step;

[0107] (2) The i-th autonomous mobile robot Rob i While moving along its trajectory, it should not collide with obstacles. t represents the sampling time.

[0108] Specifically, when the autonomous mobile robot is performing a task, it is required that the starting states, target states and execution states of all autonomous mobile robots within the system do not conflict.

[0109] Then, through step 1.2, we can get the execution framework of the CL-CBS algorithm.

[0110] Step 1.3: Based on the space-time hybrid A-star algorithm, a penalty term based on the position change of the autonomous mobile robot is set to form a penalty space-time hybrid A-star algorithm, which is used as the lower layer algorithm of the CL-CBS algorithm. The Ackerman steering characteristics of the autonomous mobile robot are introduced into the global trajectory planning through the penalty space-time hybrid A-star algorithm, which can make the autonomous mobile robot have better executableness for the global reference trajectory. Specifically, the algorithm is in a free working environment. The trajectory nodes are searched and expanded, and an open list is formed through the expanded trajectory nodes. The open list forms a tuple of the trajectory node information. This tuple means that at the current trajectory node, the i-th autonomous mobile robot Rob i The state at sampling time t is z i ; Among them, Nf represents the total cost of the current trajectory node; Ng represents the total cost from the initial state s i To the current state z i The cost of Nf, Ng and Nh is as follows:

[0111] Nf=N.g+Nh (1)

[0112] Among them, Nh can be further expressed as the current state z i To the target state g i The cost is calculated as follows:

[0113]

[0114] Among them, h RS Represents the minimum cost of connecting the current state to the target state using the Reed-Shepp curve. represents the Euclidean distance between the two points; the Reed-Shepp curve is the shortest curve connecting two points using an arc and a straight line, which was proposed in the paper "Optimal Paths for a Car that Goes Both Forwardsand Backwards" published by JAReeds. In addition, in order to improve the operating accuracy of the autonomous mobile robot, a penalty is imposed on Ng, which is calculated as follows:

[0115] Ng pen =w turn w dir Ng (3)

[0116] Among them, w turn is the first penalty coefficient based on posture change, w dir is the second penalty coefficient based on posture change; w turn and w dir All need to be set manually, and w turn ≥1,w dir ≥1; specifically, w turn Investigate whether the autonomous mobile robot switches between forward and backward, w dir Investigate whether the autonomous mobile robot switches between left and right directions. If the robot switches between forward and backward, use w turn >1; if a switch between left and right directions occurs, w is used dir >1; if no conversion occurs, then w turn =w dir = 1. Then, the total cost of the current trajectory node can be further calculated as follows:

[0117] Nf=Ng pen +Nh (4)

[0118] Therefore, the tuple It can be further expressed as By continuously updating the tuple The lower-level algorithm can obtain the global reference trajectory of each autonomous mobile robot. Whether the global reference trajectory is feasible requires conflict detection based on the upper-level algorithm. If there is no conflict, it is a feasible global trajectory.

[0119] Then, through step 1.3, the lower-level algorithm of the CL-CBS algorithm can obtain the global reference trajectory of each autonomous mobile robot and transmit the global reference trajectory to the upper-level algorithm for conflict detection and resolution.

[0120] Step 1.4: The upper-level algorithm uses the binary tree detection principle to perform conflict detection on the global reference trajectory input by the lower-level algorithm. Specifically, the upper-level algorithm will traverse the preliminary global reference trajectory based on the binary tree to detect whether the global reference trajectory is a feasible global trajectory. If there is no conflict, it means that the CL-CBS algorithm has found a feasible global trajectory for each autonomous mobile robot. If there is a conflict, the upper-level algorithm will impose constraints on the conflict that occurs first, that is, (a i ,N.π j [t],t) and (a j ,N.π i [t],t); the former requires the i-th autonomous mobile robot Rob i No entry position at sampling time t The latter requires the jth autonomous mobile robot Rob j No entry position at sampling time t This constraint is the conflict resolution solution. Furthermore, the upper-level algorithm transmits the conflict resolution solution back to the lower-level algorithm, and the lower-level algorithm re-plans the global reference trajectory for each autonomous mobile robot based on the resolution solution. In addition, the upper-level and lower-level algorithms of CL-CBS will repeatedly solve the problem in a cycle until the trajectories of each autonomous mobile robot do not conflict, that is, a feasible global trajectory for each autonomous mobile robot is found.

[0121] Then, through steps 1.3 to 1.4, that is, through the cyclic solution of the lower and upper algorithms of the CL-CBS algorithm, the feasible global trajectory of each autonomous mobile robot can be found. Specifically, Figure 1 As shown, the robot posture information and task information are provided by the prior information, and according to the virtual map, the feasible global trajectory solution of multiple autonomous mobile robots can be completed through the CL-CBS algorithm.

[0122] Step 1.5: After obtaining the feasible trajectories of each autonomous mobile robot, in order to specifically and accurately evaluate the performance of the CL-CBS algorithm, three evaluation indicators are introduced to contribute to data visualization and analysis convenience. The expressions of the evaluation indicators are as follows:

[0123]

[0124] Among them, N cost Represents the total generation value, which is the first evaluation indicator; T makespan represents the longest running time step of CL-CBS, which is the second evaluation index; T averageflow represents the average time step for the agent to reach the target point, which is the third evaluation indicator. In addition, n AMR represents the number of autonomous mobile robots, Nf(i) represents the i-th autonomous mobile robot Rob i From the initial state s i To the target state g i The total cost, t step Represents the time step.

[0125] Through step 1, a feasible global trajectory of the multiple autonomous mobile robots can be obtained. Step 1 is a global trajectory planning module of the multiple autonomous mobile robots.

[0126] Step 2: The specific steps are as follows:

[0127] Step 2.1: According to the feasible global trajectory output by the CL-CBS algorithm, a speed planning method based on the transition principle is proposed. By analyzing the feasible global trajectory, the time nodes of the appearance and end of the waiting action of each autonomous mobile robot can be directly obtained, and then the time period is finely divided according to the time nodes of the end of the waiting action. Specifically, the time nodes of the end of each waiting action of a certain autonomous mobile robot are used as the key time nodes of the autonomous mobile robot, and then the feasible global trajectory of the autonomous mobile robot is divided into different time periods according to the key time nodes, so that the autonomous mobile robot adopts the corresponding speed in different time periods to avoid collisions and waiting actions. Figure 2 As shown in the figure, the traditional speed planning method sets the speed of the autonomous mobile robot to zero when waiting for the action to occur, which greatly reduces the efficiency of dynamic cooperative transportation operations and causes excessive energy consumption due to repeated acceleration and deceleration of the autonomous mobile robot; the speed planning based on the jump principle can set different speeds for different autonomous mobile robots based on key time nodes, effectively improving the traffic efficiency and reducing energy consumption; Figure 2 v in (b) x_1 represents the longitudinal velocity of the first autonomous mobile robot, v x_2 Represents the longitudinal velocity of the second autonomous mobile robot.

[0128] Then, through step 2.1, the key time nodes of each autonomous mobile robot can be obtained.

[0129] Step 2.2: Through the key time nodes of each autonomous mobile robot, we can get the first key time node t1 of the autonomous mobile robot, the time period T1 before the first key node, and the time period T2 after the first key node. In addition, in order to facilitate different autonomous mobile robots to choose the appropriate speed, it is necessary to determine the priority of the autonomous robots; specifically, autonomous mobile robots with waiting actions are all secondary priorities, and autonomous mobile robots without waiting actions are all the highest priorities. The longitudinal speed v of the highest priority autonomous mobile robot x Take the preset ideal speed v e , the longitudinal speed v of the next priority autonomous mobile robot x Take speed v p , v p The selection range is calculated as follows:

[0130]

[0131] Among them, δ v represents the velocity weighting coefficient, and 0≤δ v ≤1;L s represents the trajectory length of the autonomous mobile robot in stage T1, and L represents the length of the autonomous mobile robot.

[0132] On this basis, in order to facilitate v p The selection of different speed reference values ​​is sequentially set in the selection range according to the speed interval of 0.5 m / s by formula (6), and the speed reference value enables the second-priority autonomous mobile robot to select a suitable longitudinal speed. It should be noted that for the second-priority autonomous mobile robots with feasible global trajectory intersections, the time nodes for waiting for the end of the action are different, so there is no collision risk between the second-priority autonomous mobile robots.

[0133] Then, through step 2.2, the longitudinal speed v of the second priority autonomous mobile robot and the highest priority autonomous mobile robot in stage T1 can be obtained: x .

[0134] Step 2.3: After the autonomous mobile robot that is rated as a low priority passes the first critical time node, the priority needs to be re-evaluated. Specifically, if the critical time node continues to appear, the autonomous mobile robot is still a low priority and re-adopts the speed according to formula (6); if there is no more critical node ahead, the longitudinal speed of the autonomous mobile robot is restored to the preset ideal longitudinal speed v e Then, the autonomous mobile robot can safely pass through each key time node in sequence.

[0135] Through step 2, a feasible global trajectory with speed information for each autonomous mobile robot can be obtained.

[0136] Step 3 is as follows:

[0137] Step 3.1: Ignoring the size information of each autonomous mobile robot and the load transfer caused by lateral and longitudinal accelerations, establish a point-mass dynamics model for the autonomous mobile robot. The point-mass dynamics model can be expressed as follows:

[0138]

[0139] Based on the point-mass dynamics model, further consider the dynamic constraints, that is, add the constraint condition |a y | ≤ μg, where u represents the control quantity, that is, the front-wheel steering angle ω; then formula (7) can be further expressed as follows:

[0140]

[0141] Where, represents the state quantity, represents the first derivative of ξ(t) with respect to the sampling time t, f(·) represents a function that can be obtained through calculation, and f(·) is composed of ξ(t) and a y v y represents the longitudinal speed of the autonomous mobile robot in the vehicle body coordinate system, v x represents the lateral speed of the autonomous mobile robot in the vehicle body coordinate system, represents the yaw angle of the autonomous mobile robot in the Cartesian coordinate system, Y represents the abscissa of the autonomous mobile robot in the Cartesian coordinate system, and X represents the ordinate of the autonomous mobile robot in the Cartesian coordinate system. and a y both represent the longitudinal acceleration of the autonomous mobile robot in the vehicle body coordinate system, represents the lateral acceleration of the autonomous mobile robot in the vehicle body coordinate system; represents the first derivative of Y with respect to the sampling time t, represents the first derivative of X with respect to the sampling time t, represents the first derivative of

[0142] Then, through step 3.1, the point-mass dynamics model of the autonomous mobile robot can be obtained, which is the basis for performing optimization solution.

[0143] Step 3.2: When establishing the point mass dynamics model, it is necessary to establish an artificial potential field function based on the position of the obstacle and the autonomous mobile robot. The purpose of the artificial potential field function is to adjust the magnitude of the repulsive force of the artificial potential field function by calculating the distance between the obstacle and the main autonomous mobile robot; wherein, the obstacles include not only the obstacles in the virtual map, but also the remaining autonomous mobile robots except the main autonomous mobile robot. By establishing the artificial potential field function, the autonomous mobile robot can avoid areas with large repulsive force, that is, avoid obstacles, in the optimization solution. The artificial potential field function is calculated as follows:

[0144]

[0145] Among them, w oa represents the global control weight, w dis Represents the distance weight, w oa and w dis Need to be set manually, and w oa >0,w dis >0; Г represents the repulsive force threshold of the artificial potential field function, r g is the radiation radius of the artificial potential field function, Г and r g Need to be set manually, and Г≥1, r g ≥1; J obc represents the repulsive force value of the artificial potential field, v represents the speed of the autonomous mobile robot; E dis represents the Euclidean distance between the autonomous mobile robot and the obstacle, which can be expressed as E dis =((XX obc ) 2 +(YY obc ) 2 ) 1 / 2 , where X obc Represents the horizontal coordinate of the obstacle in the Cartesian coordinate system, Y obc Represents the vertical coordinate of the obstacle in the Cartesian coordinate system.

[0146] It is not difficult to find that according to formula (9), when v is near zero, J obc The value of will also approach zero, which will make the autonomous mobile robot cross obstacles. Therefore, it is necessary to constrain v in the artificial potential field function to meet the reasonable expansion of obstacles. Therefore, v is defined as follows:

[0147]

[0148] Specifically, if Figure 3 As shown, the repulsive threshold Г of the artificial potential field function is set to 5, and the radiation radius r g Set to 1, global control weight w oaSet to 1, distance weight w dis Set to 1; on this basis, set the speed v of the autonomous mobile robot to 0, 0.5m / s, 1m / s, 2m / s in turn, and different repulsive force values ​​J can be obtained obc ; and the repulsive force value J of each coordinate when v is set to 0, 0.5m / s and 1m / s obc are equal, which verifies the feasibility of formula (9) and formula (10); in the figure, X g Represents the horizontal axis of the Cartesian coordinate system, Y g represents the longitudinal axis of the Cartesian coordinate system, and Z represents the vertical axis of the Cartesian coordinate system.

[0149] Then, through step 3.2, the real-time repulsive force value of the autonomous mobile robot during operation can be obtained, which is also the basis for the optimization solution.

[0150] Step 3.3: In order to facilitate the integration of the artificial potential field function into the optimization solution, the point mass dynamics model is directly used for solution, and the forward Euler method is used for discretization. In addition, since the control goal of local trajectory planning is to minimize the deviation from the feasible global trajectory and achieve obstacle avoidance, the objective function J of the local trajectory planning layer can be p It is expressed as follows:

[0151]

[0152] Among them, Q local Represents the first weight matrix of the objective function, R local The second weight matrix representing the objective function, Q local and R local Manual setting is required; J obs.t is the value of the artificial potential field function at sampling time t, N p represents the prediction time domain, N c represents the control time domain. η(t+m|t) represents the local reference trajectory in the time period t+m calculated by the optimization solution starting from the sampling time t; η ref (t+m|t) represents the feasible global trajectory within the time period t+m starting from the sampling time t; ΔU(t+m|t) represents the control increment matrix within the time period t+m starting from the sampling time t; U t Represents the control quantity matrix at sampling time t, U min Represents the minimum value of the control matrix, U max Represents the maximum value of the control matrix, U min and U max Manual setup is required. In addition, Represents the number of rows as N cThe column vector of , u(t-1) represents the control amount at sampling time t-1.

[0153] Then, through step 3.3, the optimized local reference trajectory can be obtained.

[0154] Step 3.4: After obtaining the optimized local reference trajectory, in order to make the local reference trajectory smoother and considering that the posture change of the autonomous mobile robot is continuous, a quintic polynomial is used to fit the local reference trajectory to obtain the local desired trajectory, which can be expressed as follows:

[0155]

[0156] Among them, a n b represents the fitting parameters of the expected horizontal coordinates of the local trajectory, n = 0, 1, 2, 3, 4, 5; n represents the fitting parameter of the desired yaw angle of the local trajectory; Y local represents the expected ordinate of the local trajectory, Represents the yaw angle of the local desired trajectory.

[0157] Through steps 2 and 3, the local desired trajectory of the multi-autonomous mobile robot can be obtained. Steps 2 and 3 are the local trajectory planning modules of the multi-autonomous mobile robot. Specifically, Figure 1 As shown, based on the feasible global trajectory input by the global trajectory planning module, the velocity planning of the autonomous mobile robot is based on the jump principle, and an artificial potential field function is established; on this basis, the local reference trajectory is fitted with a fifth-order polynomial to obtain the local desired trajectory.

[0158] Step 4: The specific steps are as follows:

[0159] Step 4.1: Establish a three-degree-of-freedom dynamic model of the autonomous mobile robot, and the dynamic equation is shown in formula (13). The three-degree-of-freedom dynamic model of the autonomous mobile robot is obtained through step 4.1, and the three-degree-of-freedom dynamic model is the basis of the model predictive control algorithm.

[0160] Step 4.2: After obtaining the three-freedom dynamics model, for the convenience of calculation, the nonlinear time-varying model described in formula (13) is linearized, and the state space after linear transformation is shown in formula (14). Then, the linear state space of the autonomous mobile robot is obtained through step 4.2, which can greatly reduce the difficulty of calculation.

[0161] Step 4.3: Use the forward Euler method to discretize the linear state space shown in formula (14), and the discretization form is shown in formula (15). Then, the discretized linear state space of the autonomous mobile robot is obtained through step 4.3, and the discretized linear state space can further reduce the amount of calculation.

[0162] Step 4.4: After obtaining the discretized linear state space, in order to avoid sudden changes in the control quantity, it is necessary to expand the state quantity to ξ(t) = [χ(t)u(t-1)] T ; Where u(t-1) represents the control quantity at sampling time t-1, and ξ(t) represents the expanded-dimensional state quantity at sampling time t. Then, the new state space is shown in formula (16). The expanded-dimensional state space of the autonomous mobile robot is obtained through step 4.4. This expanded-dimensional state space can prevent the control quantity from changing suddenly on the basis of the discretized linear state space, so as to improve the stability of tracking the local desired trajectory.

[0163] Step 4.5: After obtaining the expanded state space, it is necessary to set the objective function for the model predictive control algorithm. In order to enable the autonomous mobile robot to track the local desired trajectory and improve the stability of the autonomous mobile robot, it is necessary to minimize the error between the trajectory output by the model predictive control algorithm and the local desired trajectory as the optimization goal; in addition, in order to avoid the control increment being too large and causing the autonomous mobile robot to lose control, it is necessary to minimize the control amount as much as possible as the optimization goal; finally, in order to avoid the sudden change of the control amount and affect the continuity of the control amount, it is necessary to add soft constraints. Therefore, the objective function is shown in formula (17).

[0164] Through step 4, the autonomous mobile robot is controlled to track the local desired trajectory and output the optimal control amount. Step 4 is the trajectory tracking control module of multiple autonomous mobile robots. Specifically, Figure 1 As shown, the local trajectory planning module inputs the local desired trajectory information, and substitutes the information into the three-degree-of-freedom dynamics model to obtain the execution trajectory information calculated by the model predictive control algorithm; on this basis, the objective function is used for optimization calculation to reduce the deviation from the local desired trajectory and improve the driving stability of the autonomous mobile robot.

[0165] Step 5: In order to further reduce the deviation between the local trajectory planning and the feasible global trajectory and improve the tracking accuracy of the trajectory tracking control module, it is necessary to adjust the weight matrix Q of formula (17) in real time mpc and R mpcThe real-time parameter adjustment method based on reinforcement learning is an autonomous learning strategy with certain intelligence and efficiency. The present invention adopts an Actor-Critic framework, which can better make up for the disadvantage of slow convergence speed of traditional reinforcement learning. The framework is mainly divided into two parts: Actor and Critic. The Actor part consists of a target policy network and an online policy network. This part is used to estimate the deterministic policy function. According to the current information (s t ,a t ,r t ,s t+1 ) and local expected trajectory information, where s t Represents the current state, a t Represents action, r t Rewards, s t+1 represents the state at the next moment; it should be emphasized that the action a of the autonomous learning strategy t is the weight matrix Q mpc and R mpc The Critic part consists of an online Q network and a target Q network. This part evaluates the value of the current action through the reward signal fed back by the environment and the estimation of the next state of the autonomous mobile robot to update and adjust the strategy of the Actor part. It should be noted that the reward of this autonomous learning strategy is based on the tracking of the local expected trajectory, that is, the better the tracking of the local expected trajectory, the higher the reward.

[0166] Specifically, the Actor generates an action based on the current environment, and the environment feeds back the next state and reward to the Actor, and transmits the action to the Critic for evaluation. After receiving the state, the Critic evaluates the action based on the online policy network and the target policy network, and transmits the evaluation result (gradient) to the Actor, so that the Actor's action is continuously optimized, that is, the maximum reward value is obtained. At the same time, the experience recycling pool stores information (s t ,a t ,r t ,s t+1 ), and sampling is performed so that reinforcement learning can break the data correlation. Then, through this autonomous learning strategy, the optimal action can be obtained, that is, the weight matrix Q mpc and R mpc ; In addition, the autonomous learning strategy will optimize the weight matrix Q mpc and R mpc The objective function passed to formula (17) enables the autonomous mobile robot to accurately track the local desired trajectory.

[0167] Then, through step 5, we can get the optimal weight matrix Q mpc and R mpc, which helps the trajectory tracking control module to accurately track the local desired trajectory.

[0168] Step 6: Update the weight matrix Q in the objective function as shown in formula (17) through the parameter adjustment method based on reinforcement learning mpc and R mpc , and solve the optimization problem shown in formula (17), we can get the optimal control increment sequence in the control time domain; further take the first control increment Δu in the optimal control increment sequence as the actual control increment, and get the control quantity as follows:

[0169] u(t)=u(t-1)+Δu(t-1)(18)

[0170] Among them, u(t)=w(t), w(t) represents the front wheel steering angle at sampling time t; u(t-1)=w(t-1), w(t-1) represents the front wheel steering angle at sampling time t-1; Δu(t-1)=Δw(t-1), Δw(t-1) represents the expected front wheel steering angle increment at sampling time t-1.

[0171] By adjusting the weight matrix of the model predictive control in real time through steps 5 and 6, the optimal control amount of the autonomous mobile robot can be obtained. Steps 5 and 6 are the reinforcement learning modules of multiple autonomous mobile robots. Specifically, Figure 1 As shown in the figure, the reinforcement learning module updates the action, state and other information of the Actor module according to the real-time environment and the real-time posture of the autonomous mobile robot, and continuously optimizes the strategy through the optimizer to improve the reward value to obtain the optimal weight matrix Q mpc and R mpc .

[0172] Step 7: The optimal control quantity outputted in step 6 is passed to each autonomous mobile robot, and the state information of each autonomous mobile robot is outputted. On this basis, the state information of each autonomous mobile robot is passed to the local trajectory planning module and the tracking control module, the state information of the next step is updated, and the dynamic cooperative transportation operation is performed until the dynamic cooperative transportation operation of multiple autonomous mobile robots is completed.

[0173] The present invention can ensure the stability and safety of the multi-autonomous mobile robot system on the basis of ensuring the effective completion of the collaborative transportation operation task.

[0174] What is disclosed above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical scheme and inventive concept of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. A data-driven multi-autonomous mobile robot dynamic collaborative transportation operation method, characterized in that: The method for dynamic collaborative transportation of multiple autonomous mobile robots first uses the CL-CBS algorithm to obtain a feasible global trajectory that meets the kinematic characteristics and safety of each autonomous mobile robot; secondly, the feasible global trajectory is processed by speed planning based on the transition principle to obtain the speed information of each trajectory point; then, the feasible global trajectory with speed information is processed by the artificial potential field method to obtain a specific executable local reference trajectory, and the local reference trajectory is fitted with a fifth-order polynomial to obtain a local desired trajectory; finally, the model predictive control algorithm is used to track the local desired trajectory; at the same time, reinforcement learning is used to adaptively adjust the control parameters of the model predictive control algorithm to achieve accurate tracking of the local desired trajectory and complete the dynamic collaborative transportation operation of multiple autonomous mobile robots.

2. The data-driven multi-autonomous mobile robot dynamic cooperative transportation operation method according to claim 1 is characterized in that: The multi-autonomous mobile robot dynamic cooperative transportation operation method comprises the following steps: Step 1: Build a simulation environment according to the real scene, and use the car-based conflict search algorithm CL-CBS to find feasible global trajectories for multiple autonomous mobile robots; decompose the problem of finding a feasible global trajectory into two algorithms: the lower algorithm is used to plan the global reference trajectory and transmit the global reference trajectory to the upper algorithm; the upper algorithm is used to resolve the conflict in the global reference trajectory and transmit the conflict resolution solution back to the lower algorithm; the lower algorithm replans the global reference trajectory according to the conflict resolution solution, and retransmits the global reference trajectory to the upper algorithm, and then loops between the upper and lower algorithms in turn until there is no conflict in the global reference trajectory, and the feasible global trajectory of each autonomous mobile robot is found; the performance of the CL-CBS algorithm is evaluated by three evaluation indicators; Step 2: Obtain the feasible global trajectory of each autonomous mobile robot through step 1, and adopt a speed planning based on the jump principle for each autonomous mobile robot to improve the operation accuracy of the autonomous mobile robot; Step 3: After obtaining a feasible global trajectory with speed information, perform anti-collision processing; first, establish a point mass dynamics model to provide a model basis for the optimization solution; second, use the artificial potential field method to expand the obstacles and the autonomous mobile robot, and integrate them into the objective function of the optimization solution to find a safe local reference trajectory; then, establish an objective function to achieve collision avoidance; finally, use a quintic polynomial to fit the local reference trajectory to obtain the local desired trajectory; Step 4: After obtaining the local desired trajectory, track the local desired trajectory; according to the dynamic characteristics of the autonomous mobile robot, establish a model predictive control algorithm based on the three-degree-of-freedom dynamic model, and ignore the difference in left and right wheel steering angles caused by Ackerman steering geometry; establish the final objective function as follows: Among them, Q mpc is the weight matrix of the expanded state quantity, R mpc is the weight matrix of the control increment matrix; ρ is the weight coefficient, ε is the relaxation factor; η(t+k|t) represents the execution trajectory in the t+k time period calculated by the model predictive control algorithm starting from the sampling time t; η ref.p (t+k|t) represents the local desired trajectory within the time period t+k starting from the sampling time t; ΔU(t+m|t) represents the control increment matrix within the time period t+k starting from the sampling time t; ΔU min Represents the minimum value of the control increment, ΔU max Respectively represent the maximum value of the control increment; U min Represents the minimum value of the control amount, U max Represent the maximum value of the control amount respectively; in addition, Represents N u The unit matrix of order, D represents the weight matrix of the control quantity constraint; Step 5: Real-time adjustment of the weight matrix Q of formula (17) mpc and R mpc , get the optimal weight matrix Q mpc and R mpc ; The Actor-Critic framework is adopted, which is divided into two parts: Actor and Critic. The Actor part consists of a target policy network and an online policy network, which is used to estimate the deterministic policy function. The Critic part consists of an online Q network and a target Q network. Through the reward signal fed back by the environment and the estimation of the next state of the autonomous mobile robot, the value of the current action is evaluated to update and adjust the strategy of the Actor part. Step 6: Update the weight matrix Q in the objective function as shown in formula (17) through the parameter adjustment method based on reinforcement learning mpc and R mpc , and solve the optimization problem shown in formula (17) to obtain the optimal control increment sequence in the control time domain; adjust the weight matrix of the model predictive control in real time through steps 5 to 6 to obtain the optimal control amount of the autonomous mobile robot; Step 7: The optimal control quantity output from step 6 is passed to each autonomous mobile robot respectively, and the status information of each autonomous mobile robot is output; on this basis, the status information of each autonomous mobile robot is passed to the local trajectory planning module and the tracking control module, the status information of the next step is updated and dynamic collaborative transportation operations are performed until the dynamic collaborative transportation operation of multiple autonomous mobile robots is completed.

3. The data-driven multi-autonomous mobile robot dynamic cooperative transportation operation method according to claim 2 is characterized in that: The upper layer algorithm in step 1 uses a binary tree for conflict search, and the lower layer algorithm uses a penalty-type spatiotemporal hybrid A-star algorithm based on the principle of low energy consumption; step 1 specifically includes the following steps: Step 1.1: Construct a structured virtual map based on the real warehouse logistics scenario, in which clustered and randomly scattered obstacles are set. The structured virtual map is defined as a continuous workspace. Step 1.2: Define the graph theory basis of the CL-CBS algorithm and let multiple autonomous mobile robots move in a continuous workspace. The area occupied by the obstacle is Therefore, the free working environment of the autonomous mobile robot is Set the real-time status of the autonomous mobile robot to The position it occupies is The i-th autonomous mobile robot Rob i Responsible for starting from the initial state Reach the target state Wherein, X represents the horizontal coordinate of the autonomous mobile robot in the Cartesian coordinate system, and Y represents the vertical coordinate of the autonomous mobile robot in the Cartesian coordinate system. It represents the yaw angle of the autonomous mobile robot in the Cartesian coordinate system; The i-th autonomous mobile robot Rob i From the initial state s i Arriving at the target state g i The trajectory of i ; π i It consists of the real-time state of the autonomous mobile robot in continuous time, so it can be expressed as π i The following conditions must be met: (1) Feasible global trajectory π i Should start from the initial state π i [0] = s i , and reaches the target state π after a finite time step i [t] = g i ; In addition, the i-th autonomous mobile robot Rob i Need to be able to stay at the target point, t step represents the time step; (2) The i-th autonomous mobile robot Rob i While moving along its trajectory, it should not collide with obstacles. t represents the sampling time; When the autonomous mobile robot is executing a task, it is required that the initial states, target states and execution states of all autonomous mobile robots in the system do not conflict with each other. The execution framework of the CL-CBS algorithm is obtained through step 1.

2. Step 1.3: Based on the spatiotemporal hybrid A-star algorithm, a penalty term based on the position change of the autonomous mobile robot is set to form a penalty-type spatiotemporal hybrid A-star algorithm as the lower layer algorithm of the CL-CBS algorithm. This algorithm works in a free working environment. The trajectory nodes are searched and expanded, and an open list is formed through the expanded trajectory nodes; the open list forms a tuple with the information of the trajectory nodes This tuple means that at the current trajectory node, the i-th autonomous mobile robot Rob i The state at sampling time t is z i ; Among them, Nf represents the total cost of the current trajectory node; Ng represents the total cost from the initial state s i To the current state z i The cost of Nf, Ng and Nh is as follows: Nf=N.g+Nh(1) Among them, Nh can be further expressed as the current state z i To the target state g i The cost is calculated as follows: Among them, h RS Represents the minimum cost of connecting the current state to the target state using the Reed-Shepp curve. represents the Euclidean distance between the two points; the Reed-Shepp curve is the shortest curve connecting two points using an arc and a straight line; In order to improve the operation accuracy of the autonomous mobile robot, a penalty is imposed on Ng, which is calculated as follows: N.g pen =w turn w dir N.g(3) Among them, w turn is the first penalty coefficient based on posture change, w dir is the second penalty coefficient based on posture change; w turn and w dir All need to be set manually, and w turn ≥1,w dir ≥1; if a transition between forward and backward occurs, w is used turn >1; if a switch between left and right directions occurs, w is used dir >1; if no conversion occurs, then w turn =w dir =1; then, the total cost of the current trajectory node is further calculated as follows: N.f=N.g pen +N.h(4) Therefore, the tuple It can be further expressed as By continuously updating the tuple The lower-level algorithm obtains the global reference trajectory of each autonomous mobile robot. Whether the global reference trajectory is feasible needs to be detected by conflict according to the upper-level algorithm. If there is no conflict, it is a feasible global trajectory. Through step 1.3, the lower-level algorithm of the CL-CBS algorithm can obtain the global reference trajectory of each autonomous mobile robot and transmit the global reference trajectory to the upper-level algorithm for conflict detection and resolution; Step 1.4: The upper-level algorithm uses the binary tree detection principle to perform conflict detection on the global reference trajectory input by the lower-level algorithm; if there is no conflict, the CL-CBS algorithm finds a feasible global trajectory for each autonomous mobile robot; if there is a conflict, the upper-level algorithm imposes constraints on the first conflict, that is, (a i ,N.π j [t],t) and (a j ,N.π i [t],t); the former requires the i-th autonomous mobile robot Rob i No entry position at sampling time t The latter requires the jth autonomous mobile robot Rob j No entry position at sampling time t This constraint is the conflict resolution solution; the upper and lower algorithms of CL-CBS will repeatedly solve the problem in a cycle until the trajectories of each autonomous mobile robot do not conflict, that is, the feasible global trajectory of each autonomous mobile robot is found; Step 1.5: After obtaining the feasible trajectories of each autonomous mobile robot, three evaluation indicators are introduced to contribute to data visualization and analysis convenience. The expressions of the evaluation indicators are as follows: Among them, N cost Represents the total generation value, which is the first evaluation indicator; T makespan represents the longest running time step of CL-CBS, which is the second evaluation index; T averageflow represents the average time step for the agent to reach the target point, which is the third evaluation indicator; in addition, n AMR represents the number of autonomous mobile robots, Nf(i) represents the i-th autonomous mobile robot Rob i From the initial state s i To the target state g i The total cost, t step Represents the time step.

4. The data-driven multi-autonomous mobile robot dynamic cooperative transportation operation method according to claim 2 is characterized in that: The step 2 described above obtains a feasible global trajectory of each autonomous mobile robot with speed information, and the specific steps are as follows: Step 2.1: The time nodes at which each waiting action of a certain autonomous mobile robot ends are taken as the key time nodes of the autonomous mobile robot, and the feasible global trajectory of the autonomous mobile robot is divided into different time periods according to the key time nodes, so that the autonomous mobile robot adopts corresponding speeds in different time periods to avoid collision and waiting actions; the key time nodes of each autonomous mobile robot are obtained; Step 2.2: Through the key time nodes of each autonomous mobile robot, the first key time node t1 of the autonomous mobile robot, the time period T1 before the first key node, and the time period T2 after the first key node are obtained; the priority of the autonomous robot is determined: the autonomous mobile robots with waiting actions are all secondary priorities, and the autonomous mobile robots without waiting actions are all the highest priorities; the longitudinal speed v of the highest priority autonomous mobile robot x Take the preset ideal speed v e , the longitudinal speed v of the next priority autonomous mobile robot x Take speed v p , v p The selection range is calculated as follows: Among them, δ v represents the velocity weighting coefficient, and 0≤δ v ≤1;L s represents the trajectory length of the autonomous mobile robot in stage T1, and L represents the length of the autonomous mobile robot; The longitudinal speed v of the second priority autonomous mobile robot and the highest priority autonomous mobile robot in stage T1 is obtained through step 2.2 x ; Step 2.3: After the autonomous mobile robot rated as the second priority passes the first critical time node, the priority needs to be re-evaluated; specifically, if the critical time node continues to appear, the autonomous mobile robot is still the second priority and re-adopts the speed according to formula (6); if there is no more critical node ahead, the longitudinal speed of the autonomous mobile robot is restored to the preset ideal longitudinal speed v e ; The autonomous mobile robot passes each key time node safely in sequence; Through step 2, a feasible global trajectory of each autonomous mobile robot with speed information can be obtained.

5. The data-driven multi-autonomous mobile robot dynamic cooperative transportation operation method according to claim 3 is characterized in that: In step 2.2, in order to select v p , different speed reference values ​​are set in sequence within the selected range at a speed interval of 0.5 m / s through formula (6), and the speed reference value enables the second-priority autonomous mobile robot to select a suitable longitudinal speed.

6. The data-driven multi-autonomous mobile robot dynamic cooperative transportation operation method according to claim 2 is characterized in that: The step 3 specifically comprises the following steps: Step 3.1: Ignoring the size information of each autonomous mobile robot and the load transfer caused by the lateral and longitudinal accelerations, a point mass dynamics model of the autonomous mobile robot is established. The point mass dynamics model is expressed as follows: Considering dynamic constraints on the basis of the point mass dynamics model and adding the constraint condition |a y | < ug, where u represents the control quantity, i.e., the front wheel steering angle w; then Equation (7) is further expressed as follows: in, Represents the state quantity, represents the first-order derivative of ξ(t) with respect to sampling time t, and f(·) is composed of ξ(t) and a y Composition; v y represents the longitudinal velocity of the autonomous mobile robot in the vehicle coordinate system, v x Represents the lateral velocity of the autonomous mobile robot in the vehicle coordinate system, represents the yaw angle of the autonomous mobile robot in the Cartesian coordinate system, Y represents the abscissa of the autonomous mobile robot in the Cartesian coordinate system, and X represents the longitudinal position of the autonomous mobile robot in the Cartesian coordinate system. and a y Both represent the longitudinal acceleration of the autonomous mobile robot in the vehicle coordinate system. Represents the lateral acceleration of the autonomous mobile robot in the vehicle coordinate system; represents the first-order derivative of Y with respect to sampling time t, represents the first-order derivative of X with respect to sampling time t, represent The first derivative with respect to sampling time t; Step 3.2: While establishing the point mass dynamics model, an artificial potential field function based on the position of obstacles and autonomous mobile robots is established. The obstacles include not only obstacles in the virtual map but also other autonomous mobile robots except the main autonomous mobile robot. By establishing the artificial potential field function, the autonomous mobile robot can avoid areas with larger repulsive force, that is, avoid obstacles, in the optimization solution. Step 3.3: Obtain the optimized local reference trajectory; transform the objective function J of the local trajectory planning layer into p It is expressed as follows: Among them, Q local Represents the first weight matrix of the objective function, R local The second weight matrix representing the objective function, Q local and R local Manual setting is required; J obs.t is the value of the artificial potential field function at sampling time t, N p represents the prediction time domain, N c represents the control time domain; η(t+m|t) represents the local reference trajectory in the t+m time period calculated by the optimization solution starting from the sampling time t; η ref (t+m|t) represents the feasible global trajectory within the time period t+m starting from the sampling time t; ΔU(t+m|t) represents the control increment matrix within the time period t+m starting from the sampling time t; U t Represents the control quantity matrix at sampling time t, U min Represents the minimum value of the control matrix, U max Represents the maximum value of the control matrix, U min and U max Manual setup is required; in addition, Represents the number of rows as N c The column vector of , u(t-1) represents the control amount at sampling time t-1; Step 3.4: After obtaining the optimized local reference trajectory, a fifth-order polynomial is used to fit the local reference trajectory to obtain a local desired trajectory, where the local desired trajectory is: Among them, a n b represents the fitting parameters of the expected horizontal coordinates of the local trajectory, n = 0, 1, 2, 3, 4, 5; n represents the fitting parameter of the desired yaw angle of the local trajectory; Y local represents the expected ordinate of the local trajectory, Represents the yaw angle of the local desired trajectory.

7. The data-driven multi-autonomous mobile robot dynamic cooperative transportation operation method according to claim 6 is characterized in that: In step 3.2, the artificial potential field function is as follows: Among them, w oa represents the global control weight, w dis Represents the distance weight, w oa and w dis Need to be set manually, and w oa >0,w dis >0; Г represents the repulsive force threshold of the artificial potential field function, r g is the radiation radius of the artificial potential field function, Г and r g Need to be set manually, and Г≥1, r g ≥1; J obc represents the repulsive force value of the artificial potential field, v represents the speed of the autonomous mobile robot; E dis represents the Euclidean distance between the autonomous mobile robot and the obstacle, which can be expressed as E dis =((XX obc ) 2 +(YY obc ) 2 ) 1 / 2 , where X obc Represents the horizontal coordinate of the obstacle in the Cartesian coordinate system, Y obc Represents the ordinate of the obstacle in the Cartesian coordinate system; It is not difficult to find that according to formula (9), when v is near zero, J obc The value of will also approach zero, which will make the autonomous mobile robot cross obstacles. Therefore, it is necessary to constrain v in the artificial potential field function to meet the reasonable expansion of obstacles. Therefore, v is defined as follows:

8. The data-driven multi-autonomous mobile robot dynamic cooperative transportation operation method according to claim 2 is characterized in that: The specific steps of step 4 are as follows: Step 4.1: Establish a three-degree-of-freedom dynamic model of the autonomous mobile robot. The dynamic equation is as follows: in, represents the cornering stiffness of the front wheel, Represents the cornering stiffness of the rear wheel; C σf Represents the longitudinal stiffness of the front wheel, C σr Represents the longitudinal stiffness of the rear wheel; l f Represents the distance from the front axle to the center of mass, l r represents the distance from the rear axle to the center of mass, m represents the mass of the autonomous mobile robot, γ represents the yaw angular velocity, and w represents the front wheel turning angle; I z represents the moment of inertia about the z-axis, where the z-axis represents the axis perpendicular to the Σxoy plane in the vehicle coordinate system; The three-degree-of-freedom dynamic model of the autonomous mobile robot is obtained through step 4.1; Step 4.2: Linearize the nonlinear time-varying model described in formula (13). The state space after linear transformation is as follows: in, Represents the state quantity under the three-degree-of-freedom dynamic model; represents the first-order derivative of χ with respect to the sampling time t, and u represents the control variable, i.e., the front wheel steering angle w; represents the output; A, B, C are coefficient matrices, which can be directly obtained through linearization processing; The linear state space of the autonomous mobile robot is obtained through step 4.2; Step 4.3: Use the forward Euler method to discretize the linear state space shown in formula (14). The discretization form is as follows: x(t+1)=A k x(t)+B k u(t)(15) Among them, A k =I6+T t A, B k =T t B; T t Represents the sampling period, which needs to be set manually; A k Represents the weight matrix of the state quantity, B k represents the weight matrix of the control quantity, I6 represents the unit matrix of order 6; χ(t+1) represents the state quantity at sampling time t+1, χ(t) represents the state quantity at sampling time t, and u(t) represents the control quantity at sampling time t; Obtain the discretized linear state space of the autonomous mobile robot through step 4.3; Step 4.4: After obtaining the discretized linear state space, expand the state quantity to ξ(t) = [χ(t)u(t-1)] T ; Among them, u(t-1) represents the control quantity at sampling time t-1, and ξ(t) represents the expanded state quantity at sampling time t; then, the new state space is as follows: in, Represents the augmented parameter matrix of the expanded state quantity, where Represents N u Row N x A column zero matrix, Represents N u The identity matrix of order; The augmented parameter matrix representing the control increment; Represents the augmented parameter matrix of the output, where 0 Ny×Nu Represents N y Row N u Column zero matrix; N u Represents the number of controlled quantities, N x Represents the number of state quantities, N y Represents the number of outputs, N u 、N x and N y Manual setting is required; Δu(t) represents the control increment at sampling time t, and η(t) represents the expanded dimension output at sampling time t; The extended-dimensional state space of the autonomous mobile robot is obtained through step 4.4; Step 4.5: After obtaining the expanded state space, set the objective function for the model predictive control algorithm; minimize the error between the output trajectory of the model predictive control algorithm and the local desired trajectory as the optimization goal, minimize the control amount as much as possible as the optimization goal, and add soft constraints; finally, the objective function shown in formula (17) is obtained; Through step 4, the autonomous mobile robot is controlled to track the local desired trajectory and output the optimal control amount.

9. The data-driven multi-autonomous mobile robot dynamic cooperative transportation operation method according to claim 2 is characterized in that: The step 5 is specifically as follows: The Actor generates an action based on the current environment. The environment feeds back the next state and reward to the Actor, and transmits the action to the Critic for evaluation. After receiving the state, the Critic evaluates the action based on the online policy network and the target policy network, and transmits the evaluation result to the Actor, so that the Actor's action is continuously optimized to obtain the maximum reward value. At the same time, the experience recycling pool stores information (s t ,a t ,r t ,s t+1 ), and perform sampling; the optimal action is obtained through the autonomous learning strategy, that is, the weight matrix Q mpc and R mpc ; In addition, the autonomous learning strategy will optimize the weight matrix Q mpc and R mpc The objective function passed to formula (17) enables the autonomous mobile robot to accurately track the local desired trajectory.

10. The data-driven multi-autonomous mobile robot dynamic cooperative transportation operation method according to claim 2, characterized in that: In step 6, the first control increment Δu in the optimal control increment sequence is used as the actual control increment, and the control amount is obtained as follows: u(t)=u(t-1)+Δu(t-1)(18) Among them, u(t)=w(t), w(t) represents the front wheel steering angle at sampling time t; u(t-1)=w(t-1), w(t-1) represents the front wheel steering angle at sampling time t-1; Δu(t-1)=Δw(t-1), Δw(t-1) represents the expected front wheel steering angle increment at sampling time t-1.

Images