Multi-Goal Model Training via Difficulty Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional multi-goal reinforcement learning methods require a long time and a large number of training samples to achieve difficult goals, as they typically use a single goal space and generate few samples for high-level difficulty goals.
Innovation Solution
A training device and method that sets and changes sub-goal spaces of varying difficulty levels, allowing for the selection of sampling goals, acquisition of transitions, training of a multi-goal model, and evaluation of success rates to progressively move to higher difficulty levels when a predetermined threshold is met.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If a single goal space is used to train the agent, then the training framework is simple, but the training time is long and the number of training samples required is large
Solution Approach 1:
The patent divides the single goal space into multiple sub-goal spaces with different difficulty levels (easy, medium, hard). Each sub-goal space contains goals of a specific difficulty level, allowing the agent to train on easier goals first and progressively move to harder goals. This segmentation reduces training time by providing easier learning paths while maintaining a relatively simple overall framework.
2Ease of operation
If conventional multi-goal reinforcement learning is used, then the training process is straightforward, but few samples are generated for high-level difficulty goals
Solution Approach 1:
The patent segments the goal space into multiple difficulty levels, ensuring that samples are distributed across easy, medium, and hard goals. This guarantees that sufficient samples are generated for high-level difficulty goals by dedicating specific sub-goal spaces to them, while keeping the training process straightforward through structured organization.
Solution Approach 2:
The patent changes the parameter of goal difficulty by creating multiple sub-goal spaces with different difficulty levels. This allows the system to adjust the distribution of training samples across different difficulty levels, ensuring adequate samples for hard goals without complicating the overall training process.
3Productivity
If multiple sub-goal spaces of different difficulty levels are used, then training efficiency is improved, but the device complexity increases
Solution Approach 1:
The patent divides the goal space into multiple sub-goal spaces organized by difficulty levels, which improves training efficiency by allowing progressive learning from easy to hard goals. The complexity is managed through a clear hierarchical structure that systematically organizes the sub-goal spaces, making the increased complexity manageable and structured.
Data Source
AI summary
A training device and a training method for training a multi-goal model based on goals in a goal space are provided. The training device includes a memory and a processor coupled to the memory. The processor is configured to set the goal space, to acquire a plurality of sub-goal spaces of different levels of difficulty; change a sub-goal space to be processed from a current sub-goal space to a next sub-goal space of a higher level of difficulty; select, as sampling goals, goals at least from the current sub-goal space, and to acquire transitions related to the sampling goals by executing actions; train the multi-goal model based on the transitions, and evaluate the multi-goal model by calculating a success rate for achieving goals in the current sub-goal space.