Bipedal Robot Push-Recovery Control Using a Trained Neural Network
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing exoskeletons face challenges in achieving dynamic stability, particularly in responding to strong and unexpected disturbances, which can lead to instability and potential harm to the operator or damage to the robot.
Innovation Solution
A method for training a neural network to stabilize a bipedal robot by simulating sequences of pushes on a virtual twin of the robot, using reinforcement learning to maximize a reward representative of recovery from these pushes, and implementing the trained neural network to provide commands to the robot's actuators.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional planning approaches and feedback control are used to maintain stable trajectory, then the exoskeleton can walk smoothly on flat ground, but it cannot effectively recover from strong unexpected disturbances
Solution Approach 1:
The patent replaces traditional mechanical control systems (feedback controllers, ZMP trajectory generation, MPC methods) with a neural network-based system. The neural network is trained through reinforcement learning to directly map sensor inputs to actuator commands, substituting complex mechanical control algorithms with a trained artificial intelligence model that can generalize to unseen disturbance scenarios
Solution Approach 2:
The patent applies preliminary action by training the neural network in advance through simulation before deploying it on the real exoskeleton. The reinforcement learning process pre-trains the network to handle a wide variety of push scenarios, so when deployed, the system already has pre-learned recovery strategies ready to execute without needing complex real-time computation
2Reliability
If reinforcement learning is used to train the neural network in simulation, then the system can learn effective recovery strategies, but there may be a gap between simulation performance and real-world performance
Solution Approach 1:
The patent creates a virtual twin (digital replica) of the exoskeleton in simulation that copies the physical system's dynamics and sensor characteristics. The neural network is trained on this copied virtual system, and the same network architecture and training approach are then transferred to the real system, leveraging the copy to avoid the need for extensive real-world trial-and-error training
Solution Approach 2:
The patent employs parameter changes by gradually adjusting simulation parameters to match real-world conditions and by using domain randomization techniques where simulation parameters are varied during training. This allows the neural network to learn robust policies that are invariant to parameter variations, improving simulation-to-reality transfer
3Speed
If a purely reactive controller is implemented for emergency push recovery, then the system can respond quickly to disturbances, but it may lack the ability to guarantee predictable and smooth behavior
Solution Approach 1:
The patent implements feedback by continuously monitoring the exoskeleton's state through sensors (IMU, joint encoders, force sensors) and feeding this information back to the neural network in real-time. The network processes the current state and generates appropriate recovery commands, creating a closed-loop control system that adapts to actual system behavior while maintaining predictable responses
Data Source
AI summary
A method for training a neural network for stabilizing a bipedal robot (1) presenting a plurality of degrees of freedom actuated by actuators is proposed. The method comprises the implementation by the data processing means (11) of a server (10) of steps of:(a) In a simulation, applying a sequence of pushes on a virtual twin of the robot (1).(b) Performing a reinforcement learning algorithm on said neural network, wherein the neural network provides commands to said actuators of the virtual twin of the robot (1), so as to maximise a reward representative of a recovery of said virtual twin of the robot (1) from each push.