Multi-model Controller Segmentation for Training Efficiency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for training computational models, such as deep neural networks, often require extensive resources and are inefficient in adapting to changes in input data over time, leading to suboptimal control effectiveness and numerical stability issues.
Innovation Solution
The use of separate but coordinated training processes for recurrent computational models and network computational models, employing supervised and reinforcement-learning update rules, respectively, allows for more efficient resource utilization and improved stability, enabling better control in dynamic environments.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If a single computational model is used for control tasks, then the system structure is simple, but the control effectiveness and adaptability to changing input data deteriorate
Solution Approach 1:
The patent divides the control system into two separate computational models: a recurrent computational model (RCM) for processing sequential input data and a feedforward computational model (FCM) for making control decisions. This segmentation allows each model to specialize in its respective function, improving overall adaptability while maintaining manageable complexity through clear separation of concerns.
2Measurement precision
If extensive resources are allocated for training computational models, then model accuracy improves, but training time and resource requirements increase
Solution Approach 1:
The training process is segmented into two independent phases: training the RCM on sequential data patterns, and training the FCM on control decision-making. This allows parallel training of both models, reducing total training time while maintaining high accuracy through specialized training data and objectives for each model type.
Solution Approach 2:
The RCM is trained first to extract meaningful state representations from sequential input data before the FCM is trained to make control decisions. This preliminary action of feature extraction and state representation learning prepares the data in an optimal format for the second stage of control decision training, improving overall training efficiency.
3Productivity
If traditional training methods are used for deep neural networks, then comprehensive learning is achieved, but numerical stability issues arise
Solution Approach 1:
The patent separates the learning tasks into two distinct computational models with different training objectives: the RCM focuses on learning sequential patterns and state representations without the numerical instability associated with reinforcement learning, while the FCM handles control decisions using stable supervised learning methods. This segmentation eliminates the numerical stability issues that arise when combining both functions in a single model.
Data Source
AI summary
A processing unit can operate a first recurrent computational model (RCM) to provide first state information and a predicted result value. The processing unit can operating a first network computational model (NCM) to provide respective expectation values of a plurality of actions based at least in part on the first state information. The processing unit can provide an indication of at least one of the plurality of actions, and receive a reference result value, e.g., via a communications interface. The processing unit can train the first RCM based at least in part on the predicted result value and the reference result value to provide a second RCM, and can train the first NCM based at least in part on the first state information and the at least one of the plurality of actions to provide a second NCM.