Quadruped robot motion planning method based on hierarchical reinforcement learning

Abstract

The invention discloses a quadruped robot motion planning method based on hierarchical reinforcement learning. According to the method, an upper layer behavior decision controller based on deep reinforcement learning and a lower layer motion execution controller based on model prediction control are constructed; for a deep reinforcement learning network at an upper layer, a state updating network, an action execution network, a reward function and the like of the quadruped robot are designed based on an SAC algorithm, and corresponding behavior control parameters are output in combination with height map information of the environment around the robot; and for a model prediction controller in the lower layer execution controller, a corresponding control instruction is obtained by solving a control parameter quadratic programming equation set input based on the upper layer. A hierarchical control framework combines the advantages of the deep reinforcement learning and model prediction control, so that the quadruped robot can make a safer and more reliable behavior mode according to the current fuselage state and the terrain environment, a fuselage posture is kept stable, risks are effectively avoided, and effective motion track planning under the complex terrain is achieved.

Description

technical field [0001] The invention relates to the field of intelligent legged robots, in particular to a quadruped robot motion planning method based on layered reinforcement learning. Background technique [0002] Compared with other forms of mobile robots, quadruped robots have independent footholds when moving, and can change their gait according to the characteristics of terrain and tasks. It is more suitable for operations in complex environments such as uneven, rugged terrain, and stairs. Better environmental adaptability. However, motion planning for quadruped robots is quite challenging. It must consider a large number of time-varying factors in order to find the optimal control decision from a large action space. Deep reinforcement learning is a new control method. In deep reinforcement learning, the neural network representing the action strategy updates the network parameters through continuous interaction with the environment, and learns how to obtain the ma...

AI Technical Summary

Problems solved by technology

However, the quadruped robot control strategy based on deep reinforcement learning cannot effectively balance the ability to explore complex environments and the stability of the fuselage. At the same time, the cooperation between the upper-level control decision-making scheme and the underlying control actuators cannot be directly and effectively coordinated.

Images