Learning Device Optimizing Pareto Solutions via Contribution Degree
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing multi-objective optimization techniques, such as those used in video games and medical simulations, face challenges in efficiently acquiring a Pareto solution set when there is no domain knowledge available for each indicator, leading to uncertainty in obtaining a Pareto solution for all indicators.
Innovation Solution
A learning device with a circuit configuration that processes input, selects actions, acquires rewards, calculates cumulative rewards, and updates Pareto solution sets based on contribution degrees, using a Q-network and batch learning to optimize multiple indicators, including elapsed time and execution results, to achieve a Pareto solution in a simulated environment.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If existing multi-objective optimization techniques are used without domain knowledge, then the optimization process can be applied generally, but the reliability of obtaining a true Pareto solution set deteriorates
Solution Approach 1:
The patent implements feedback mechanisms where the AI agent continuously receives reward signals from the environment based on its actions. The reinforcement learning algorithm uses this feedback to update its policy and improve its decision-making over time, enabling reliable convergence to Pareto solutions even without domain knowledge. The feedback loop allows the system to learn from trial and error, gradually refining its understanding of the objective space.
Solution Approach 2:
The patent employs preliminary actions by initializing the reinforcement learning agent with an exploration strategy that systematically samples the state space before optimization begins. This preliminary exploration phase allows the agent to gather information about the environment and objective functions, establishing a foundation for subsequent exploitation and convergence to reliable Pareto solutions.
2Adaptability or versatility
If reinforcement learning is used for multi-objective optimization, then the AI can learn strategies for complex environments, but the computational complexity and training time increase
Solution Approach 1:
The patent segments the multi-objective optimization problem into multiple single-objective subproblems by maintaining a set of Pareto solutions rather than optimizing a single objective. Each state-action pair is evaluated against multiple objectives simultaneously, and the reinforcement learning algorithm learns to balance these objectives through the segmentation of the value function into multiple Q-values, one for each objective. This segmentation reduces the computational burden compared to evaluating all possible Pareto combinations.
Solution Approach 2:
The patent applies partial action by focusing the reinforcement learning agent on learning policies for a subset of critical objectives rather than exhaustively optimizing all objectives simultaneously. The agent learns to prioritize certain objectives based on the current state and the existing Pareto solution set, performing partial optimization that is sufficient for practical applications without the excessive computational cost of complete multi-objective optimization.
3Manufacturing precision
If the AI learns to balance multiple indicators, then the quality of operation improves, but the difficulty of learning and training increases
Solution Approach 1:
The patent introduces an intermediary mechanism in the form of a reward function that mediates between multiple conflicting objectives. Instead of directly learning to balance multiple indicators, the AI learns to maximize a unified reward signal that encodes the trade-offs between objectives. This intermediary reward function simplifies the learning task by transforming a complex multi-objective problem into a single-objective reinforcement learning problem, making training more manageable while still achieving high-quality operations.
Data Source
AI summary
A learning device, acquiring a Pareto solution set based on multiple indicators, inputs a first Pareto solution set with at least a non-Pareto solution until a first step indicating a time point of elapsed time, and a first state of the environment in the first step, selects an action in the first state by providing the environment with the first Pareto solution set and the first state, acquires a reward related to the multiple indicators in the first step, and a second state of the environment in a second step, calculates, based on a cumulative reward up to the first step and the first Pareto solution set, a contribution degree which is a cumulative increase amount of a hypervolume since the first step, and updates a second Pareto solution set in the second step by adding the cumulative reward to the first Pareto solution set based on the contribution degree.