Reinforcement Learning for Servo Velocity Feedforward Coefficient Optimization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In servo control devices using velocity feedforward control, highly ordered coefficients for velocity feedforward control lead to position errors due to nonlinear characteristics like machine friction and lost motion, making coefficient adjustment difficult.
Innovation Solution
A machine learning device performs reinforcement learning to adjust the coefficients of the velocity feedforward control, using state information and reward values based on position errors to optimize the coefficients, thereby improving followability with the position command without requiring troublesome manual adjustments.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If the coefficients of velocity feedforward are set to be highly ordered to improve followability with respect to position command, then position error is reduced, but adjustment of the coefficients becomes difficult
Solution Approach 1:
The system performs self-learning through reinforcement learning, where the machine learning device automatically adjusts the velocity feedforward coefficients by itself without requiring manual intervention. The device acquires position error information, calculates reward values, and updates coefficients through automated iterative learning processes.
Solution Approach 2:
The system implements feedback mechanisms by acquiring position error information from the servo control device and using this feedback to calculate reward values. These reward values guide the adjustment of velocity feedforward coefficients in subsequent learning iterations, creating a closed-loop self-optimizing system.
2Ease of operation
If reinforcement learning is implemented to automatically adjust velocity feedforward coefficients, then manual adjustment complexity is reduced, but system complexity increases
Solution Approach 1:
The system divides the complex control problem into separate functional modules: a servo control device for position control, a machine learning device for coefficient optimization, and distinct processing units for acquiring position error information and calculating reward values. This segmentation makes the overall system more manageable and understandable.
Solution Approach 2:
The machine learning device acts as an intermediary between the servo control device and the velocity feedforward coefficient adjustment. It receives position error information from the servo control device and translates this into reward-based feedback for coefficient optimization, simplifying the interaction between control elements.
Data Source
AI summary
A machine learning device performs machine learning with respect to a servo control device including a velocity feedforward calculation unit. The machine learning device comprises: a state information acquisition unit configured to acquire from the servo control device, state information including at least position error, and combination of coefficients of a transfer function of the velocity feedforward calculation unit; an action information output unit configured to output action information including adjustment information of the combination of coefficients included in the state information, to the servo control device; a reward output unit configured to output a reward value in reinforcement learning based on the position error included in the state information; and a value function updating unit configured to update an action value function on the basis of the reward value output by the reward output unit, the state information, and the action information.