Machine Learning Parameter Tuning with Simulation Feedback
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Tuning parameters for industrial application programs, such as robot applications, is time-consuming and often delayed due to the difficulty in obtaining sufficient data for machine learning, especially when explorative learning is expensive, dangerous, and not feasible on real hardware, and transferring simulation-based machine learning results to real applications is challenging due to the reality gap and long learning times.
Innovation Solution
A method that generates candidate parameters through machine learning, executes programs in both simulated and real applications, collects performance data, and modifies the simulated application based on real-world measurements to reduce the reality gap, using meta-heuristic algorithms and physics parameters for optimization, allowing for simultaneous execution in both environments to accelerate the learning process.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If machine learning is applied to real industrial applications for parameter tuning, then learning accuracy and reliability are improved, but the process becomes expensive, dangerous, and time-consuming
Solution Approach 1:
The patent performs preliminary machine learning actions in a virtual environment before applying them to real hardware. The system executes the application program in a virtual application that mimics the real industrial application, allowing parameter tuning to be performed in advance without risking actual equipment. This preliminary learning phase generates candidate parameters that can then be safely transferred to real devices.
Solution Approach 2:
The patent introduces a virtual application as an intermediary between the machine learning process and the real industrial application. This virtual environment acts as a mediator that allows explorative learning to occur without direct interaction with physical hardware, thereby eliminating the dangers and costs associated with real-world experimentation while still providing relevant training data.
2Adaptability or versatility
If explorative learning is used to search for new possibilities and adapt to environmental changes, then adaptability is improved, but the process becomes expensive and dangerous on real hardware
Solution Approach 1:
The patent creates a copy of the real industrial application in the form of a virtual application. This virtual copy replicates the essential characteristics and behavior of the real system, allowing explorative learning to be performed on the copy rather than the original. The virtual application can be modified and experimented with without affecting the actual industrial equipment, thus eliminating the harmful factors associated with real-world experimentation.
3Object-affected harmful factors
If simulation-based machine learning is used to avoid real hardware risks, then safety is improved, but the reality gap between simulation and reality causes severe consequences
Solution Approach 1:
The patent implements a feedback mechanism where the virtual application is modified based on reality data obtained from actual machine executions. The system collects performance information from real applications and uses this feedback to adjust and improve the virtual application's accuracy. This continuous feedback loop reduces the reality gap by ensuring the virtual environment remains synchronized with real-world conditions.
Solution Approach 2:
The patent makes the virtual application dynamic by allowing it to be modified and adapted based on collected reality data. Rather than using a static simulation model, the system continuously updates the virtual application's parameters and characteristics to match real-world behavior, thereby maintaining high transfer accuracy while preserving safety benefits.
4Reliability
If sufficient data is collected for machine learning training, then learning quality is improved, but the time required for data collection and learning process becomes too long
Solution Approach 1:
The patent performs data collection and learning actions preliminarily in the virtual environment before they would be needed in the real application. By executing the application program multiple times in the virtual application with different candidate parameters, the system accumulates training data and performs learning iterations in advance, thus preparing high-quality model parameters without delaying real-world deployment.
Data Source
Figure 1
Figure 2~4
AI summary
A method for applying machine learning, in particular parameter based machine learning,to an application, comprising the steps: a)Generating a set of candidate parameters (PC) by a learner (20); b)Executing a program (80) in at least one simulated application (51, 52) based on the set of candidate parameters (PC) and providing interim results (R1, R2) of tested sets of candidate parameters (PC1, PC2) based on a measured performance information of the execution of the program (80); c)Collecting a predetermined number of interim results (R1, R2) and providing an end result (R) based on a combination of the candidate parameters (PC) and the measured performances information by a trainer (30); and d)Generating a new set of candidate parameters (PC) by the learner (20) based on the end result (R), for being executed by the unchanged program (80).