Quadruped Gait Control Using Position-Increment Reinforcement Learning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for controlling quadruped robot motions face challenges such as sudden changes in control commands, difficulty in designing reward functions, and limited environmental adaptability, which can lead to motor damage and inefficient gait performance.
Innovation Solution
A method and system that utilize reinforcement learning combined with position increment to control quadruped robot motions, constraining foot sole position changes within each time step to avoid sudden control command changes, and using a preset reward function to guide stable and smooth gait performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If reinforcement learning directly outputs motor position commands, then the quadruped robot can adapt to complex environments, but the motor position changes suddenly causing physical damage to the motor
Solution Approach 1:
The patent introduces an intermediary mechanism (constraint reward function) between the reinforcement learning output and the motor execution. This intermediary constrains the motor position changes to be within a safe range, preventing sudden large movements that would damage the motor while still allowing the robot to adapt to complex environments through reinforcement learning.
Solution Approach 2:
The patent changes the parameter representation from direct motor position to motor position increment. By outputting the change amount rather than the absolute position, the reinforcement learning system can control the magnitude of movements, ensuring they remain within safe limits while maintaining environmental adaptability.
2Reliability
If constraint reward function of motor torque or velocity is introduced, then motor damage is prevented, but the difficulty of designing reward function and adjusting parameters increases greatly
Solution Approach 1:
The patent simplifies the reward function design by changing from constraining motor torque or velocity to constraining motor position increment. This parameter transformation makes the constraint more direct and easier to design, as it directly limits the movement amount without requiring complex torque or velocity models.
Data Source
AI summary
A method for controlling motions of a quadruped robot based on reinforcement learning and position increment, including: acquiring motion environment information, quadruped robot attitude information, and foot sole position information; based on the acquired information, generating foot sole positions of the quadruped robot during motions within all preset time steps, and calculating a change of the foot sole positions in all the time steps; taking a maximum moving distance within a single time step as a constraint, and accumulating the time steps at the same time to obtain a foot sole position trajectory; and controlling the quadruped robot to perform corresponding actions based on the foot sole position track combined with a preset reward function, so as to keep motion balance of the quadruped robot.