Part Extraction Device for Reinforcement Learning Design
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In system design, particularly with reinforcement learning, the probability of successful design is low due to large design scales, leading to sparse rewards and hindered learning, as existing methods fail to effectively generate system requirements that are worth learning.
Innovation Solution
A part extraction device and method that acquires system configuration information in a graph format, extracts and concretizes abstract parts using conversion rules, and employs reinforcement learning to evaluate and select applicable conversion rules, thereby simplifying the design process and increasing the frequency of successful designs.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If system design is performed using reinforcement learning with large design scales, then the system can handle complex design tasks, but the probability of successful design becomes low and rewards become sparse
Solution Approach 1:
The patent divides the large-scale system design problem into multiple smaller sub-problems by extracting parts from the system configuration graph. Each part represents a manageable subset of the overall system, allowing reinforcement learning to operate on smaller, more tractable problems while maintaining the ability to handle complex overall designs through hierarchical composition of parts.
2Adaptability or versatility
If the design scale is large, then the system can cover more functionality, but the learning process is hindered due to sparse rewards
Solution Approach 1:
The patent segments the large design space into extractable parts that can be processed independently. By focusing reinforcement learning on generating and evaluating individual parts rather than complete systems, the learning efficiency improves while the overall system functionality is maintained through composition of these parts.
Solution Approach 2:
The patent applies partial action by extracting and processing only certain parts of the system configuration at a time, rather than attempting to process the entire system simultaneously. This allows the reinforcement learning agent to achieve meaningful progress on partial designs, receiving more frequent rewards and improving learning efficiency.
3Stability of the object's composition
If system configuration is processed as a whole, then the overall system view is maintained, but the design process becomes difficult to manage
Solution Approach 1:
The patent segments the system configuration into extractable parts represented as sub-graphs. This segmentation maintains the overall system view through the parent-child relationship between system-level graphs and part-level graphs, while reducing design process complexity by allowing independent processing and manipulation of individual parts.
Solution Approach 2:
The patent implements a nested structure where parts are extracted from the system configuration graph and can contain further nested parts. This hierarchical nesting allows the design process to operate at multiple levels of abstraction, maintaining system integrity while managing complexity through layered decomposition.
Data Source
AI summary
A part extraction device includes: at least one memory configured to store instructions; and at least one processor configured to execute the instructions to: acquire first information that shows a configuration of a system in a graph format; and extract second information that shows a configuration of a part of the system in a graph format, from the first information.