Learning Apparatus for Balancing Model Size and Inference Accuracy
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing learning apparatuses face challenges in achieving desired performance balances between inference accuracy and model size, requiring professional skills and cumbersome operations to adjust training conditions and execute retraining.
Innovation Solution
A learning apparatus that acquires and sets different training conditions to train machine learning models with reduced sizes, determines the necessity of retraining based on inference accuracy comparisons, and performs retraining to achieve higher inference accuracy while maintaining a smaller model size.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Volume of moving object
If the model size is reduced to meet deployment requirements, then the model can be deployed on resource-constrained devices, but the inference accuracy deteriorates
Solution Approach 1:
The patent applies dynamic training condition adjustment by automatically modifying training parameters (such as learning rate, batch size, epoch number) based on the target model size constraints. The system dynamically adapts training conditions to maintain inference accuracy while achieving the desired model size reduction, rather than using static training parameters.
Solution Approach 2:
The patent changes training parameters automatically based on model size requirements. The system adjusts multiple training conditions including learning rate, batch size, and epoch number to optimize the balance between model size and inference accuracy. This parameter transformation approach enables the model to achieve desired size reduction while maintaining performance through automated parameter optimization.
2Measurement precision
If manual adjustment of training conditions is performed to achieve desired performance, then the inference accuracy can be optimized, but the operation complexity and time consumption increase significantly
Solution Approach 1:
The patent implements self-service through an automated determination unit that automatically determines whether retraining is necessary and what training conditions should be adjusted. The system performs self-evaluation of model performance and autonomously decides on retraining actions without requiring professional expertise or manual intervention, thereby simplifying operations while maintaining optimization capability.
Solution Approach 2:
The patent employs feedback mechanisms where the determination unit continuously evaluates inference accuracy and model size, then provides feedback to automatically adjust training conditions. This closed-loop feedback system enables the model to automatically iterate and optimize performance without manual intervention, reducing operation complexity while achieving desired accuracy levels.
3Measurement precision
If multiple retraining iterations are performed to achieve desired performance, then the model accuracy can be improved, but the time consumption and computational resources increase
Solution Approach 1:
The patent applies preliminary action by pre-determining optimal training conditions before actual retraining begins. The determination unit analyzes current model performance and pre-calculates the necessary training condition adjustments, avoiding unnecessary trial-and-error iterations. This preliminary planning reduces the number of retraining iterations needed and accelerates the optimization process.
Solution Approach 2:
The patent implements partial action by performing retraining only when necessary and only for the specific training conditions that need adjustment. Rather than repeatedly training under all possible conditions, the system selectively applies retraining based on determined optimization needs, reducing overall computational waste and time consumption while achieving the desired accuracy improvement.
Data Source
AI summary
According to one embodiment, a learning apparatus includes a processing circuit. The processing circuit acquires a first training condition and a first model trained in accordance with the first training condition, sets a second training condition used to reduce a model size of the first model, different from the first training condition, in accordance with the second training condition and based on the first model, trains a second model whose model size is smaller than that of the first model, and in accordance with a third training condition that is not the same as the second training condition and complies with the first training condition, trains a third model based on the second model.