Wheel-Legged Robot Control via Policy Iteration Equilibrium Learning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Controlling the motion of underactuated robots, particularly wheel-legged robots, is challenging due to the complexity of their mechanical structure, making it difficult to obtain an accurate dynamic model and perform parameter identification, which affects the controller's effectiveness.
Innovation Solution
A method using adaptive dynamic programming and policy iteration to build a controller that adapts to the robot's dynamic characteristics, allowing it to maintain equilibrium and follow a target trajectory, even with unknown or changing parameters, by collecting motion and control data during the robot's movement and updating a linear equilibrium parameter matrix.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional dynamic model-based control is used for underactuated robots, then control accuracy can be improved, but the complexity of obtaining accurate dynamic models and performing parameter identification increases significantly
Solution Approach 1:
The patent replaces traditional mechanics-based dynamic modeling with a data-driven reinforcement learning approach. Instead of deriving complex dynamic models through mechanical analysis and parameter identification, the system uses neural networks trained via reinforcement learning to directly map robot states to control actions, thereby achieving accurate control without the complexity of traditional dynamic model acquisition
Solution Approach 2:
The system enables the robot to automatically learn and adapt its control policy through self-reinforcement learning. The robot collects its own state data during operation, trains its own neural network controller, and continuously improves its control performance without requiring external dynamic model identification or manual parameter tuning
2Reliability
If accurate dynamic model parameters are obtained through identification, then controller effectiveness is improved, but the time and data requirements for training increase
Solution Approach 1:
The patent performs preliminary exploration actions during the reinforcement learning training phase to efficiently gather diverse state-space coverage. By designing smart exploration strategies that prioritize informative states, the system reduces the total training time required to achieve reliable control performance compared to random or exhaustive data collection methods
Solution Approach 2:
The system implements continuous learning and adaptation during robot operation. Instead of requiring separate offline training phases, the reinforcement learning controller continuously updates its policy based on ongoing operational data, allowing the robot to improve its effectiveness progressively without stopping for extended training periods
Data Source
AI summary
In a method for controlling a robot, movement of the robot is controlled according to a first controller. Motion state data and control data of the robot are obtained while the movement of the robot is controlled according to the first controller. A linear equilibrium parameter matrix of the first controller is updated according to a policy iteration algorithm that uses the obtained motion state data and the control data. A second controller of the robot is built based on the updated linear equilibrium parameter matrix.