Machine Controller Using Trajectory Prediction to Limit Error Drift
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing control systems for machines trained with batch data are susceptible to prediction errors when operating in conditions poorly covered by the training data, leading to inadequate performance and potential escalation of errors over time.
Innovation Solution
A method that considers entire action and state trajectories, using a control agent trained to minimize deviations from training data, combined with a performance evaluator to select optimized action trajectories, reducing prediction errors and adapting to changing goals without extensive retraining.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If batch training data is used to train the control agent, then training efficiency is improved, but control reliability deteriorates in operating conditions poorly covered by training data
Solution Approach 1:
The method performs preliminary validation of the control agent by simulating action trajectories on training data before actual deployment. This preliminary action identifies potential failure modes in advance, allowing the system to prepare mitigation strategies or select alternative control approaches for poorly covered operating conditions, thereby improving reliability without sacrificing batch training efficiency
Solution Approach 2:
The system implements feedback by using the performance evaluator to assess control agent decisions against training data patterns. When the evaluator detects that current operating conditions fall into poorly covered regions of the state space, it provides feedback signals that trigger trajectory simulation and validation procedures, creating a closed-loop system that adapts to data coverage limitations and maintains reliability
2Device complexity
If the control agent is trained to predict only the next control action, then device complexity is reduced, but manufacturing precision deteriorates due to accumulation of prediction errors
Solution Approach 1:
Instead of reacting to errors after they accumulate, the system performs preliminary simulation of complete action trajectories using the control agent before execution. By validating the entire trajectory against training data patterns in advance, the system identifies and corrects potential error accumulation issues before they affect actual control precision, maintaining accuracy without requiring fundamentally more complex control architecture
Solution Approach 2:
The trajectory simulation and performance evaluation components serve as intermediaries between the simple control agent and the actual machine control. This intermediary layer validates and refines the control agent's predictions by simulating trajectories and comparing them against training data, effectively filtering out accumulated prediction errors while keeping the core control agent relatively simple
3Manufacturing precision
If extensive retraining is performed to adapt to changing performance goals, then control precision is improved, but loss of time increases
Solution Approach 1:
The system creates a copy of the training data patterns through the performance evaluator, which stores and references historical training trajectories. When performance goals change, instead of retraining the control agent from scratch, the system queries this copied data structure to identify relevant patterns and adjust control strategies, maintaining precision while avoiding time-consuming retraining processes
Solution Approach 2:
The system achieves adaptation to changing performance goals by modifying parameters such as the performance evaluator's target criteria and trajectory evaluation weights, rather than retraining the entire control agent. This parameter-based adaptation allows rapid adjustment to new goals by changing evaluation parameters while keeping the trained control agent intact, significantly reducing adaptation time while maintaining control precision
Data Source
Figure 1~2
Figure 3~4
Figure 5
AI summary
A large number of training datasets (TDs) are read in, each specifying a machine state, a control action, and a resulting subsequent state. For each machine state, a training action trajectory (TAT) is derived from the training datasets. An action trajectory (AT) is then predicted using a control agent (POL), and a deviation (DA) of the predicted action trajectory (AT) from the training action trajectory (TAT) is determined. The deviation (DA) is accumulated over several trajectory time steps. The control agent is then trained to reduce the determined deviations (DA). Additionally, a performance evaluator (PEV) is provided, which calculates a trajectory performance value (RET) for each action trajectory (AT).To control the machine (M), a current operating state (BS) of the machine is recorded. Using the trained control agent (POL), a multitude of test action trajectories (TTA) originating from the recorded operating state are predicted. For each test action trajectory, a trajectory performance value (RET) is determined by the performance evaluator (PEV). Based on the determined trajectory performance values (RET), a performance-optimizing action trajectory (PA) is selected from the test action trajectories (TTA). The machine is then controlled based on the selected action trajectory (PA).