Machine Control Agent Selection for Reliable State Space Coverage
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing machine learning control agents for complex machines often perform unfavorably in operating states with limited training data, and stochastic policies can be unpredictable, making validation challenging.
Innovation Solution
Training control system-specific agents using diverse training data sets from various control systems, allowing for the generation of deterministic test agents within defined parameter spaces, and selecting a performance-optimizing agent that closely resembles the original system-specific agents to ensure reliable control behavior.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If training data from multiple control systems is aggregated, then coverage of state space is improved, but control actions become inconsistent for the same machine states
Solution Approach 1:
The patent segments the training data by control system origin, creating separate training datasets for each control system. Control agents are then trained individually on their respective control system's data, ensuring consistency within each agent while maintaining diverse state space coverage across multiple agents.
Solution Approach 2:
The patent changes the parameter of data attribution by adding control system identifiers to training data. This allows the training process to recognize and separate data from different control systems, enabling the creation of multiple specialized control agents rather than a single generalist agent.
2Productivity
If stochastic policies are used to explore state space, then efficiency of exploration is improved, but predictability and validity of control actions deteriorate
Solution Approach 1:
The patent segments the exploration process by creating multiple specialized control agents, each trained on specific control system data. This allows deterministic behavior within each agent's domain while collectively covering the entire state space efficiently.
Solution Approach 2:
The patent creates multiple copies (control agents) of the neural network architecture, each specialized for a particular control system. These copies collectively perform the exploration function that a single stochastic agent would provide, but with deterministic and predictable individual behaviors.
3Device complexity
If a single control agent is trained on aggregated data, then device complexity is reduced, but training precision and control accuracy deteriorate
Solution Approach 1:
The patent segments the training process into multiple specialized agents, each focusing on a specific control system's data. This segmentation improves training precision by reducing noise and inconsistencies in the training data, even though it increases the number of control agents.
Solution Approach 2:
The patent creates a universal framework where multiple specialized control agents can be deployed together. Each agent is simple and focused, but the collective system provides comprehensive control coverage across all machine states and control systems.
Data Source
Figure 1~2
Figure 3
Figure 4
AI summary
According to the invention, training data sets (TD) acquired by controlling the machine (M) using various control systems are read in. Each TD comprises a state data set (S) and an action data set (A). Furthermore, a performance evaluator (PEV) is provided, which determines the performance of a control agent (TP1, TP2,...) for controlling the machine. According to the invention, for each of the different control systems, a control system-specific control agent (P1, P2,...) is trained using the training data sets assigned to the respective control system to reproduce an action data set (A) based on a state data set (S). In addition, within a parameter space of the control system-specific control agents (P1, P2,...), a respective environment (U1, U2,...) is defined around the trained control system-specific control agents using a distance measure (D). Within the environments (U1, U2,...A large number of test control agents (TP1, TP2,...) are then generated, for each of which a performance value is determined by the performance evaluator (PEV). Depending on the determined performance values, a performance-optimizing control agent (PO) is finally selected from the test control agents (TP1, TP2,...) by means of which the machine (M) is controlled.