Reinforcement Learning with Dynamic State-Based Policy Selection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional reinforcement learning methods are inefficient when dealing with wide state changes or non-specific rewards, often requiring a long time to complete or failing to do so effectively, as they rely on single-perspective learning policies.
Innovation Solution
An agent device performs exclusive reinforcement learning by using a statistical analyzer to determine state values based on average and standard deviation, activating either central or peripheral perspective reinforcement learning units depending on these values, allowing for multi-perspective learning and action selection.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If conventional single-perspective reinforcement learning is used, then the learning policy is simple to implement, but the learning efficiency is low and training time is long when dealing with wide state changes or non-specific rewards
Solution Approach 1:
The patent segments the reinforcement learning process into multiple parallel learning units, each specializing in different state ranges (e.g., first learning unit for small state changes, second learning unit for large state changes). This segmentation allows each unit to focus on specific state variations, improving learning efficiency without requiring a single complex policy to handle all scenarios.
Solution Approach 2:
The patent implements dynamic selection of learning units based on the current state characteristics. The system dynamically determines which learning unit to activate by evaluating whether the state change falls within the first or second range, allowing the learning policy to adapt its complexity to the specific situation rather than maintaining fixed high complexity for all cases.
2Reliability
If multiple reinforcement learning units are used to handle different state perspectives, then the learning accuracy and adaptability improve, but the computational resources and system complexity increase
Solution Approach 1:
The system dynamically selects which reinforcement learning unit to execute based on the current state characteristics. By evaluating whether the state change falls within the first range (handled by first learning unit) or second range (handled by second learning unit), the system activates only the necessary learning unit, avoiding unnecessary computational overhead while maintaining high action selection accuracy.
Solution Approach 2:
The patent changes the operational parameters of the reinforcement learning system by introducing state-range-based conditional execution. Instead of running multiple learning units simultaneously or using a single complex unit, the system modifies its behavior based on parameters such as the magnitude of state changes, activating different learning units only when their specific conditions are met, thereby reducing overall computational resource consumption.
Data Source
AI summary
A method and an apparatus for exclusive reinforcement learning are provided, comprising: collecting information of states of an environment through the communication interface and performing a statistical analysis on the states using the collected information; determining a first state value of a first state among the states in a training phase and a second state value of a second state among the states in an inference phase based on analysis results of the statistical analysis; performing reinforcement learning by using one reinforcement learning unit of a plurality of reinforcement learning unit which performs reinforcement learnings from different perspectives according to the first state value; and selecting one of actions determined by the plurality of reinforcement learning unit based on the second state value and applying selected action to the environment.