A Gait and Trajectory Planning Method for Multi-Legged Robots Based on a Hierarchical Mutual Coordination Strategy

Abstract

The present invention discloses a gait and trajectory planning method for a multi-legged robot based on a hierarchical mutual cooperation strategy, which solves the problem that there are still challenges in the overall algorithm control of existing legged robots. The key technical solution is to divide the control of the robot into a lower-level controller responsible for local gait planning and an upper-level controller for global trajectory planning. For the lower-level controller, a non-linear optimization method is used. According to the local terrain and body information of the robot's current position, a non-linear optimization problem is established to search for the trajectory parameters of the tip of each leg. Through the inverse solution of the obtained tip trajectory of each leg, the control signal of the robot's motor is obtained. For the upper-level controller, a supervised learning method is used to learn the capabilities of the lower-level controller from the motion information samples, and a model-based reinforcement learning algorithm is used for global trajectory planning. The gait and trajectory planning method for a multi-legged robot based on a hierarchical mutual cooperation strategy of the present invention has universality, simple ideas, and is more convenient and effective.

Description

Technical Field

[0001] The present invention relates to multi-legged robot motion control, and in particular to a multi-legged robot gait and trajectory planning method based on a hierarchical mutual cooperation strategy. Background Art

[0002] With the progress of the times and the development of science and technology, robotics technology plays an increasingly important role in a wide range of fields such as industrial production, transportation, exploration of unknown environments, unmanned operations in dangerous areas, and military and national defense. Robotics technology has now become a strong driving force for humans to greatly improve productivity.

[0003] According to the different modes of movement, robots can be divided into wheeled, footed, tracked and hybrid. Wheeled robots rely on the friction between wheels and contact surfaces for movement. They are mainly suitable for flat roads and have high-speed mobility, but are powerless against more complex terrains. Tracked robots can adapt well to softer terrains, such as land, but have the disadvantage of being powerless against terrains with large height differences. Footed robots can adapt to almost all kinds of complex terrains. Their disadvantages are low movement speeds and easy rollovers due to the high center of gravity. Most of the world's terrain is complex. For complex terrains, footed robots have obvious advantages, so the research on footed robots has broad development prospects. This is also the most worthwhile aspect of footed robots - they can walk on unstructured and uneven terrains. They are much more flexible than wheeled robots, which are difficult to move in areas with gaps or significant changes in height.

[0004] The main research contents of legged robots are mainly divided into two aspects: gait control and path planning. Gait control requires the robot to maintain balance while walking on complex terrain, and path planning requires the robot to avoid obstacles in complex environments and find an optimal path to reach the final target point. Therefore, how to design the gait controller and trajectory planner of the legged robot is crucial in the research of environmental exploration of the legged robot. In a complex terrain environment, how to effectively integrate the gait controller and trajectory planner of the legged robot is a challenge for the legged robot. The quality of the overall control algorithm will directly affect the efficiency and effect of the legged robot in performing tasks, and is extremely important for the stability and safety of the legged robot. Summary of the invention

[0005] The purpose of the present invention is to provide a multi-legged robot gait and trajectory planning method based on a hierarchical cooperative strategy, which is universal, simple in concept, and more convenient and effective.

[0006] The above technical objectives of the present invention are achieved through the following technical solutions:

[0007] A multi-legged robot gait and trajectory planning method based on a hierarchical cooperative strategy includes the following steps:

[0008] S1. Collecting map information of obstacles and terrains required for the multi-legged robot to complete the task, including overall terrain information used for global path planning and local terrain information used for local gait and trajectory planning;

[0009] S2, collecting information about the movement of the multi-legged robot in the environment, including local terrain information, the robot's own state information, and statistical information on the success rate of reaching the target point;

[0010] S3, establishing a neural network in the upper controller that is required to derive a state transition model based on local terrain information, using the collected information data of movement in the environment, and training the neural network through supervised learning until convergence;

[0011] S4, using the state transition model obtained after training and the overall terrain information required for global path planning, using the value iteration method to obtain the global path planning result;

[0012] S5. For each local goal in the global planning, a nonlinear optimization method is used to optimize the gait and trajectory to obtain the toe trajectory of the multi-legged robot;

[0013] S6. Perform kinematic inverse analysis based on the toe trajectory of the multi-legged robot, and output the motor control signal of the multi-legged robot required for gait and trajectory planning.

[0014] Preferably, the information of the multi-legged robot moving in the environment is collected as follows:

[0015] Select a preset gait of the multi-legged robot;

[0016] Define the multi-legged robot's motion by combining its preset gait and walking direction;

[0017] The multi-legged robot is controlled to walk along the preset action through the underlying controller, the success rate of walking to the target point is recorded, and the walking ability data of the multi-legged robot is collected.

[0018] Preferably, in step S3, local terrain information and self-state information are used as inputs of the neural network, and the probability of the multi-legged robot executing the corresponding action and transferring to the next state is output.

[0019] As a preference, the global path planning is obtained by using the value iteration method as follows:

[0020] The trained state transfer model converts local terrain information and body posture information to generate a state transfer matrix;

[0021] Use the state transfer matrix to perform value iteration algorithm operations;

[0022] Value iteration completes global path planning.

[0023] Preferably, step S5 includes:

[0024] Establish a kinematic model of a multi-legged robot;

[0025] The phase time of the robot's gait is added to the optimized variable, and the toe trajectory and force are represented by cubic function fitting;

[0026] Limit the contact between toes and the ground;

[0027] The nonlinear optimization problem of multi-legged robot gait and trajectory planning is established and solved.

[0028] In summary, the present invention has the following beneficial effects:

[0029] By basing both path planning and trajectory planning on terrain, the method can be applied to various multi-legged robots. Compared with conventional trajectory planning, the global path planning is combined with local gait and trajectory planning in a hierarchical control manner, which can solve longer continuous gait and trajectory planning of multi-legged robots. In addition, the combination of global path and local trajectory planning can reasonably avoid some terrains that are inaccessible by only considering local terrain through path planning when the global path is established, thus providing an effective method for gait and trajectory planning of multi-legged robots. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 The basic flow chart of this method is as follows;

[0031] Figure 2 Schematic diagrams of the two most common gaits of the hexapod robot in the example;

[0032] Figure 3 Schematic diagram of the effective terrain area for different actions of the hexapod robot in the example.

[0033] Figure 4 This is the network structure diagram of the state transition model in the example.

[0034] Figure 5 This is a schematic diagram of the upper-level controller structure in the example.

[0035] Figure 6 This is a schematic diagram of the underlying controller structure in the example.

[0036] Figure 7 This is a schematic diagram of the change of toe displacement and force with time and phase when the legged robot moves in the example.

[0037] Figure 8Construct a schematic for the nonlinear optimization problem of gait and trajectory planning for a multi-legged robot. DETAILED DESCRIPTION

[0038] The present invention is further described in detail below in conjunction with the accompanying drawings.

[0039] Based on the advantages of multi-legged robots and the difficulties and pain points of multi-legged robot motion planning at this stage, gait planning is the key part to ensure that legged robots can maintain stability during movement, and is an important prerequisite for legged robots to move and explore in complex environments. Trajectory planning is the "navigation" that guides the direction of robot movement, and its importance in robot movement is self-evident. An excellent legged robot motion control system must be able to adapt to the structure of the legged robot it controls, be able to design gaits for it, so that it can maintain balance when moving in complex environments, and at the same time provide correct navigation for the target location the robot is going to.

[0040] According to one or more embodiments, a multi-legged robot gait and trajectory planning method based on a hierarchical cooperative strategy is disclosed, such as Figure 1 As shown, the following steps are included:

[0041] S1. Collect map information of obstacles and terrains required for the multi-legged robot to complete the task, including overall terrain information used for global path planning and local terrain information used for local gait and trajectory planning.

[0042] S2. Collect the motion information of the multi-legged robot in the environment, including local terrain information, the robot's own state information, and statistical information on the success rate of reaching the target point; the bottom-level controller controls the robot's motion, and the collected motion information data represents the robot's ability to cross the terrain under the control of the bottom-level controller. The collected motion information data is used as a sample to generate samples for the training of the upper-level controller.

[0043] For a multi-legged robot, its movement generally follows a certain gait, that is, the timing combination of the specific legs and feet hanging in the air and landing in each step of its walking. The steps for collecting robot movement information data are:

[0044] Select a preset gait of the multi-legged robot;

[0045] Define the multi-legged robot's motion by combining its preset gait and walking direction;

[0046] The multi-legged robot is controlled to walk along the preset action through the underlying controller, the success rate of walking to the target point is recorded, and the walking ability data of the multi-legged robot is collected.

[0047] S3. Establish a neural network in the upper-level controller that is required to derive a state transition model based on local terrain information, use the collected information data on movement in the environment, and train the neural network through supervised learning until convergence.

[0048] Taking local terrain information and its own state information as input, the output is the probability that the multi-legged robot will perform the corresponding action and transfer to the next state.

[0049] S4. Using the state transition model obtained after training and the overall terrain information required for global path planning, the value iteration method is used to obtain the global path planning result.

[0050] The trained state transfer model converts local terrain information and body posture information to generate a state transfer matrix;

[0051] Use the state transition matrix to perform value iteration algorithm operations.

[0052] S5. For each local goal in the global planning, use the nonlinear optimization method to optimize the gait and trajectory to obtain the toe trajectory of the multi-legged robot.

[0053] Establish a kinematic model of a multi-legged robot;

[0054] The phase time of the robot's gait is added to the optimized variable, and the toe trajectory and force are represented by cubic function fitting;

[0055] Limit the contact between toes and the ground;

[0056] The nonlinear optimization problem of multi-legged robot gait and trajectory planning is established and solved.

[0057] S6. Perform kinematic inverse analysis based on the toe trajectory of the multi-legged robot, and output the motor control signal of the multi-legged robot required for gait and trajectory planning.

[0058] For the sake of clarity, let's take a six-legged robot as an example:

[0059] S1. Collect the map information of obstacles and terrains required for the multi-legged robot to complete the planning task. The map information includes the overall terrain information used for global path planning and the local terrain information used for small-segment gait and trajectory planning. The form and characteristics of the required information here will be further reflected and explained in the subsequent steps.

[0060] S2. Collect a large amount of information about the movement of the multi-legged robot in the environment, including terrain information and the robot's own state information, and count the success or failure information, and use this as a sample, so as to generate a large number of samples for the training of the state transition model required by the upper-level controller. The purpose of collecting data is to use data to represent the robot's ability to cross the terrain under the control of the bottom-level controller. For a multi-legged robot, its movement generally follows a certain gait, that is, the specific combination of the timing of the legs and feet hanging in the air and landing in each step of its walking.

[0061] S21. Select the preset gait of the multi-legged robot:

[0062] For this embodiment, Figure 2 The two most common gaits of this hexapod robot are shown, namely the 3-3 gait and the 2-4 gait. The 3-3 gait divides the six legs into two groups, A and B, with each group consisting of three non-adjacent legs. It can be clearly seen in the phase diagram that when using the 3-3 gait, the legs in group A leave the ground first, at which time the legs in group B are the supporting legs and in contact with the ground. When the legs in group A land on the ground and in contact with the ground, there is a short period of time when both legs in group A and group B are in contact with the ground. Then the legs in group B leave the ground, and the legs in group A serve as the supporting legs and in contact with the ground until the legs in group B land on the ground and in contact with the ground. At this time, there is another short period of time when both legs in group A and group B are in contact with the ground. This cycle constitutes the 3-3 gait. Similarly, in Figure 2 The phase diagram on the right shows the 2-4 gait.

[0063] S22. The preset gait and walking direction of the multi-legged robot are combined to define the action:

[0064] In order to adapt to the value iteration module used in the subsequent path planning, the local map is divided into 9 areas in the form of a nine-square grid. Therefore, the robot has 8 moving directions, namely, to the lower left, lower, lower right, left, right, upper left, upper, and upper right. Because the robot can use two different gait types, 3-3 gait and 2-4 gait, the two are combined to form 16 actions. That is, using 2 different gaits to go to the surrounding 8 grids respectively.

[0065] S23. Data collection of multi-legged robot walking ability:

[0066] Taking into account the size and step length of the hexapod robot, and combined with the limit on the number of gait cycles specified by the underlying controller, the size of the local terrain environment is set to 3 meters * 3 meters. Then, the initial position of the robot is at the center of the terrain, that is, the vertical projection of the center of the robot on the horizontal plane is at (1.5m, 1.5m), the position of the robot's toes is the same as the default initial value in the xy plane, and the height is the same as the ground in the z direction, which means that it is in contact with the ground. For terrain information, the continuous terrain information is discretized and represented by discrete and evenly distributed points on the xy plane. Therefore, a point is selected every 10 centimeters, so that a 30*30 terrain information matrix is ​​obtained, which can represent the terrain conditions.

[0067] Construct various terrains in such local terrain, use the underlying controller to control the robot to walk along one of the 16 preset actions, and record the success rate of walking to the target point. This step can be performed in a virtual environment or a simulation environment to prevent the robot from damaging its own mechanical structure in the trial and error experiment. In addition, it saves experimental time and reduces the mechanical wear of the robot itself. The terrain information and body information in the sample are normalized using the MinMax method, and the success rate is stored in the form of a vector.

[0068] S3. Establish a neural network required in the upper-level controller that can derive a state transition model based on local terrain information, and use the data collected by S2 to train the network through supervised learning until convergence.

[0069] S31. Input and network structure of state transition model:

[0070] like Figure 3 As shown, action a 1 ,a 2 That is, use the 3-3 gait type and the 2-4 gait type to go to the lower left corner grid. At this time, Figure 3 The terrain information in the lower left corner is what the state transition model needs. The underlying solver does not use the terrain information in other places. Similarly, for action a 9 ,a 10 That is, use two types of gait to go to the right grid. Each network classifies the input terrain and toe position information, and determines the transfer probability of this action under the current terrain according to the category of the input information.

[0071] After the model and network selection are determined, a suitable neural network structure can be constructed. The network structure diagram used in this embodiment is shown in FIG. Figure 4 shown.

[0072] S32. Train the state transfer model neural network.

[0073] According to different actions in the sample, the terrain information of different areas is selected as the first part of the input in S31, and the initial toe position in the sample is used as the second part of the input. Figure 4 The network structure shown builds a neural network, uses cross entropy as the loss function and Adam optimizer, and trains the sample data. In this way, the upper-level controller obtains a transition probability model, which includes the capabilities of the underlying controller and the current terrain and robot body information, which can be used for the next step of value iteration based on reinforcement learning to plan the trajectory of the global map for the hexapod robot.

[0074] S4: In the upper controller, the state transition model trained in S3 and the overall terrain map required for global path planning are used to obtain the global path planning result using the value iteration method. This result can provide local goals for the lower controller. Figure 5 shown.

[0075] S41. Determine the sources of the inputs used by the upper-level controller and value iteration:

[0076] In S3, the trained state transition model neural network can convert local terrain information and body posture into a state transition matrix, which is then input into the value iteration algorithm. Another important input used in value iteration is the reward function. The global map is divided into grids and covered with 1m*1m grids. The action distribution in the grid world is relatively simple, and it is easy to establish a value iteration model. Mark the robot's target point on the map and establish a reward function R. The design of the reward function is relatively simple. Only when you walk near the target point will you get a positive reward, and when you walk into other grids, you will receive a negative reward as the cost of survival.

[0077] S42, value iteration completes global path planning:

[0078] The basis of value iteration is the Markov decision process MDP, which is a framework that can solve most reinforcement learning problems. MDP is described by a tuple (S, A, P, R, γ). The goal of reinforcement learning is to find the optimal strategy π to maximize the expected cumulative return. The so-called strategy is the mapping from state to action π: s→a.

[0079] According to the Bellman optimization equation, we have:

[0080]

[0081]

[0082] Solving it using the Gauss-Seidel iterative algorithm yields:

[0083]

[0084] When the greedy strategy is selected, the last time becomes:

[0085]

[0086] Where k is the iteration round, In return, is the state transfer matrix, which has been described in S41. v is the state value function, which is the expectation of the cumulative return of the current state in reinforcement learning and is used to describe the "goodness" of the current state.

[0087] In the value iteration algorithm, the above formula is iterated repeatedly until convergence. At this time, the optimal strategy can be obtained based on the state value function:

[0088]

[0089] In this way, the global path planning is completed.

[0090] S5. For each local goal in the global planning, a nonlinear optimization method is used to optimize the gait and trajectory. Specifically, the phase time of the robot gait is added to the optimized variable, and a cubic function is used to fit the displacement and force between the feet of the legged robot during movement. The parameters of the fitting function are added to the optimized variable. In addition, the contact between the toes and the ground, the ground height, that is, the terrain is provided by the local terrain information collected in the first step, and the robot kinematic model is added to this nonlinear optimization problem. The legged robot gait optimization framework optimizer is used to solve the nonlinear optimization problem to obtain the toe trajectory of the multi-legged robot, such as Figure 6 shown.

[0091] S51. Modeling of the hexapod robot:

[0092] To simplify calculation, the hexapod robot is modeled as a massive body and six toe points. When the six toe points touch the ground, the ground generates a support force fi on them, thus keeping the body stable.

[0093] S52. Expression of toe trajectory and force:

[0094] When the legged robot moves, the displacement and force of each leg tip can be expressed as follows: Figure 7 As shown in the figure, in each motion phase ΔT i,j In the figure, the displacement p of the toe of the i-th leg i and the ground support force f i There are only two cases, either a constant value or a series of cubic functions that are continuously differentiable at the intersection point:

[0095] x(t)=a0 +a 1 t+a 2 t 2 +a 3 t 3

[0096] The calculation of cubic function parameters can be done using the Hermitian method.

[0097] S53. Limitation of toe-ground contact.

[0098] like Figure 7 , for p that needs to be represented by several consecutive cubic functions i and f i , in each p i and the ground support force f i Three consecutive cubic functions are used to represent it, so we have The number of cycles is limited according to the different gaits, that is, the number of steps per leg n s,i .

[0099] Each leg goes through two phases every time it takes a step, namely in the air and touching the ground. Therefore, the time set when the toe of the i-th leg touches the ground for the s-th time is:

[0100]

[0101] Therefore, the set of times when the toe of the i-th leg touches the ground is:

[0102]

[0103] Then, we can define some constraints for solving the gait planning of the hexapod robot. First, the toes in contact with the ground will not slide: In addition, the toes in the air are not supported: The toes in contact with the ground can only be supported but not pulled. Therefore, the ground support force limit is defined as:

[0104]

[0105] Here n(x,y) represents the normal vector of the terrain slope at (x,y). For horizontal ground, n(x,y) = [0,0,1] T , this restriction becomes f z (t)≥0.

[0106] S54. Other restrictions:

[0107] Other limitations include phase time limitations, ground height limitations, robot kinematic model limitations, etc.

[0108] Phase time ΔT i,j are optimization variables, and the optimizer can adjust their lengths, but they must add up to the total time T of the gait planning task:

[0109]

[0110] The horizontal plane is the xy plane, the projection of the robot center on the horizontal plane is the origin, and the vertical upward direction is the z axis (height) to establish a coordinate system. The toe is in contact with the ground if and only if the height of the toe is equal to the height of the ground. Therefore, the toe position on the xy plane needs to be equal to the value given by the terrain function:

[0111]

[0112] The robot dynamics constraints are expressed as:

[0113]

[0114] S55. Establishment and solution of nonlinear optimization problem of multi-legged robot gait and trajectory planning:

[0115] After the above analysis and discussion, the nonlinear optimization problem of the gait planning task of the bottom-level controller of the hexapod robot is finally established as follows: Figure 8 The form shown.

[0116] The solution cost c is the minimized objective variable of the optimizer, which is related to the number of nodes in each toe trajectory and the number of solution iterations. After the nonlinear optimization problem is established, the corresponding nonlinear solver or robot optimization solver is used to solve the corresponding toe trajectory. In this embodiment, the Trajectory Optimization for Walking Robot (TOWR) framework is used to help solve the gait planning problem of the hexapod robot. Add the robot model and constraints to TOWR, and TOWR will use IPOpt (interior point optimization solver) to solve the toe trajectory that meets the given constraints and robot model.

[0117] S6, performing kinematic inverse solution on the robot toe trajectory obtained in step 5, and finally outputting the multi-legged robot motor control signal required for gait and trajectory planning.

[0118] The research idea of ​​the present invention is simple, convenient, effective and universal. The movement of the multi-legged robot is always carried out on the basis of gait, and the path planning and trajectory planning of the present invention are based on the terrain, so it is suitable for various multi-legged robots. Compared with conventional trajectory planning, it combines global path planning with local gait and trajectory planning through hierarchical control, so it can solve longer continuous gait and trajectory planning of multi-legged robots. In addition, the combination of global path and local trajectory planning, for some terrains that are inaccessible only by considering local terrain, the method proposed by the present invention can reasonably avoid through path planning when the global path is established, providing relevant scientific and technological personnel with an effective method for gait and trajectory planning of multi-legged robots.

[0119] This specific embodiment is merely an explanation of the present invention and is not a limitation of the present invention. After reading this specification, those skilled in the art may make non-creative modifications to the present embodiment as needed. However, as long as they are within the scope of the claims of the present invention, they are protected by the patent law.

Claims

1. A gait and trajectory planning method for a multi-legged robot based on a hierarchical mutual cooperation strategy, characterized in that, it includes the following steps: S1. Collect the map information of obstacles and terrain required for the multi-legged robot to complete tasks, including the overall terrain information for global path planning and the local terrain information for local gait and trajectory planning; S2. Collect the information of the multi-legged robot moving in the environment, including local terrain information, the robot's own state information, and the success rate information of reaching the target point; S3. Establish a neural network in the upper controller that needs to obtain a state transition model based on local terrain information, and use the information data of the movement in the collected environment to train the neural network through supervised learning until convergence; S4. Utilize the trained state transition model and the overall terrain information required for global path planning, and use the value iteration method to obtain the global path planning result; S5. For each local target in the global plan, use the nonlinear optimization method to optimize the gait and trajectory to obtain the toe trajectory of the multi-legged robot; S6. Perform kinematic inverse solution according to the toe trajectory of the multi-legged robot, and output the motor control signal of the multi-legged robot required for gait and trajectory planning.

2. The gait and trajectory planning method for a multi-legged robot based on a hierarchical mutual cooperation strategy according to claim 1, characterized in that, the information of the multi-legged robot moving in the environment collected is specifically: select the preset gait of the multi-legged robot; combine the preset gait of the multi-legged robot with the walking direction to define the actions of the multi-legged robot; control the multi-legged robot to walk along the preset actions through the lower controller, record the success rate of walking to the target point, and collect the walking ability data of the multi-legged robot.

3. The gait and trajectory planning method for a multi-legged robot based on a hierarchical mutual cooperation strategy according to claim 1, characterized in that: In step S3, the local terrain information and the self-state information are used as the input of the neural network, and the probability of the multi-legged robot performing the corresponding action and transferring to the next state is output.

4. The gait and trajectory planning method for a multi-legged robot based on a hierarchical mutual cooperation strategy according to claim 1, characterized in that, the specific process of using the value iteration method to obtain the global path planning is: the trained state transition model converts the local terrain information and the body posture information into a state transition matrix; use the state transition matrix to perform the operation of the value iteration algorithm; the value iteration completes the global path planning.

5. The gait and trajectory planning method for a multi-legged robot based on a hierarchical mutual cooperation strategy according to claim 1, characterized in that, step S5 includes: establish a kinematic model of the multi-legged robot; add the phase time of the robot gait to the variables to be optimized, and use a cubic function fitting to represent the toe trajectory and the force; limit the contact between the toe and the ground; establish and solve the nonlinear optimization problem of the multi-legged robot gait and trajectory planning.

Images