Legged Robot Gait Timing Using Reinforcement Learning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Selecting an appropriate gait for a legged robot is challenging, as different gaits can result in instability or undesirable movement, especially in complex multi-terrain environments.
Innovation Solution
A method using a neural network trained with reinforcement learning to generate gait timing parameters based on the robot's state and target trajectory, allowing for adaptive control of the robot's movement.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional heuristic-based gait selection is used, then the system is simple to implement, but the robot exhibits instability and undesirable movement in complex multi-terrain environments
Solution Approach 1:
The patent replaces traditional heuristic-based control algorithms with a reinforcement learning-based neural network controller. The neural network learns optimal gait timing parameters through reinforcement learning from simulated experiences, substituting manual heuristic design with automated machine learning-based control that adapts to complex terrains while maintaining system reliability.
Solution Approach 2:
The patent changes the control parameters from fixed heuristic rules to dynamically learned timing parameters. The neural network outputs gait timing parameters (such as swing phase duration, stance phase duration, and foot placement timing) that are continuously adjusted based on terrain conditions, robot state, and target trajectory, enabling adaptive control that maintains stability across varied environments.
2Adaptability or versatility
If reinforcement learning-based neural network control is implemented, then the robot achieves stable and efficient movement in complex environments, but the training and computation requirements increase
Solution Approach 1:
The patent performs gait policy learning in advance through extensive reinforcement learning training in simulated environments before deployment. The neural network is pre-trained on diverse terrain conditions, robot configurations, and gait patterns, storing learned knowledge in its weights and biases. This preliminary training allows the controller to rapidly adapt to new terrains during actual operation without requiring real-time learning, thus reducing operational computation time while maintaining high adaptability.
Data Source
AI summary
Techniques for determining robotic step timing and sequencing using reinforcement learning are provided. In one aspect, a method includes receiving a target trajectory for a robot and receiving a state of the robot. The method further includes generating, using a neural network, a set of gait timing parameters for the robot based, at least in part, on the state of the robot and the target trajectory and controlling movement of the robot based on the set of gait timing parameters.