Lithography Control Using Expected Manipulated Variables
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing control systems using reinforcement learning suffer from a deterioration in control performance during operation due to stochastic behavior when determining manipulated variables by sampling, which affects quality assurance.
Innovation Solution
A control device that includes a generator to produce a probability distribution for determining manipulated variables and a determinator to select these variables based on the expectation value of the distribution, rather than the maximum probability, to maintain control performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If the manipulated variable is determined by sampling using a random number during operation, then control performance is improved, but stochastic behavior affects quality assurance
Solution Approach 1:
The patent applies dynamics by switching between two different determination methods based on the operational phase. During learning phase, sampling with random numbers is used to explore and improve control performance. During operation phase, expectation value calculation is used to ensure stability and quality assurance. This dynamic switching resolves the contradiction by allowing both stochastic exploration and deterministic operation.
Solution Approach 2:
The patent changes the parameter determination method from stochastic sampling to deterministic expectation value calculation based on the operational phase. By changing how the manipulated variable is determined (from random sampling to expectation value), the system maintains control performance while eliminating stochastic behavior during operation, thus resolving the quality assurance issue.
2Reliability
If the manipulated variable maximizing the probability value is continuously selected, then quality assurance is maintained, but control performance deteriorates compared to sampling methods
Solution Approach 1:
The system dynamically adjusts the determination method based on phase. In learning phase, it uses sampling to achieve better control performance. In operation phase, it switches to expectation value calculation to maintain quality assurance. This dynamic adaptation allows the system to optimize for different objectives at different times, resolving the performance-quality tradeoff.
Solution Approach 2:
The patent performs preliminary learning during the learning phase where sampling methods can be used to optimize control performance without affecting quality assurance. The expectation value parameters are pre-calculated during learning, so during operation, the system can use these pre-determined values to maintain both performance and quality assurance simultaneously.
Data Source
AI summary
A control device controls an object to be controlled. The device includes a generator configured to generate a probability distribution used to determine a manipulated variable, and a determinator configured to determine the manipulated variable based on the probability distribution generated by the generator. In an operation phase, the determinator determines the manipulated variable in accordance with an expectation value of the probability distribution.