Imperfect Emulation of Model States Using Reinforcement Learning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current emulation methods, particularly in cloud computing, face challenges in imperfectly emulating the state of a model, especially when the original system is complex or not fully understood, such as an internal combustion engine or a human heart, where exact emulation is not feasible, and there is a need for alternative approaches to converge the model and actual systems effectively.
Innovation Solution
The method involves determining the current state of an actual model, comparing it with a reference model, calculating deviations, and modifying the actual model's state to emulate the target state using reinforcement learning and neural networks, with parameter sharing between models to improve convergence and performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If exact emulation is attempted for complex systems (e.g., internal combustion engine, human heart), then emulation accuracy is improved, but device complexity and computational requirements increase significantly making it infeasible
Solution Approach 1:
The patent applies partial emulation by focusing on emulating only the critical or most influential parameters of the actual model rather than attempting to replicate the entire system. The system identifies and emulates key states and parameters that have the greatest impact on system behavior, accepting that not all aspects will be perfectly replicated. This approach achieves sufficient emulation accuracy for practical purposes while keeping computational complexity manageable.
Solution Approach 2:
The patent transforms the emulation problem from state-space replication to parameter-space optimization. By changing the approach from emulating complete system states to optimizing specific parameters that characterize system behavior, the solution reduces computational complexity while maintaining essential emulation accuracy for decision-making purposes.
2Productivity
If reinforcement learning with reward maximization is used to transition internal hidden state, then learning efficiency is improved, but convergence speed may be reduced compared to supervised learning
Solution Approach 1:
The patent applies preliminary action by pre-training the neural network using supervised learning methods before transitioning to reinforcement learning. This initial supervised training phase establishes a good starting point for the reinforcement learning process, providing preliminary knowledge that accelerates subsequent convergence. The system performs required action in advance (supervised pre-training) to reduce the time needed for final convergence during reinforcement learning.
3Adaptability or versatility
If neural networks are used to model complex systems, then ability to solve unsolvable problems is improved, but requirement for training data and computational resources increases
Solution Approach 1:
The patent applies partial action by focusing neural network training on only the most critical parameters and states needed for effective emulation, rather than attempting to learn and replicate all aspects of the complex system. This selective approach reduces training data requirements while maintaining the ability to solve complex problems that require understanding of key system behaviors.
Data Source
AI summary
Present disclosure provides a method and an electronic device for performing imperfect emulation of a state of a model in an electronic device. The method includes determining, by an electronic device, a current state of at least one actual model of the electronic device and comparing, by the electronic device, the current state of the at least one actual model with at least one state of a reference model. The method also includes determining, by the electronic device, a target state to be achieved by the at least one actual model based on the at least one state of the reference model; determining, by the electronic device, a deviation of the current state of the at least one actual model with respect to the target state to be achieved by the actual model; and modifying, by the electronic device, the current state of the at least one actual model to emulate the target state to be achieved by the at least one actual model based on the at least one state of the reference model.