On-Chip Machine Learning Model Training via Gradient Accumulation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Training machine learning models on devices with limited memory and computation resources is challenging due to the high demand for memory and computation.
Innovation Solution
An integrated circuit with a processor configured to receive data, store model parameters, and update them using running sums of prior loss gradients, allowing for on-chip training of machine learning models.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If machine learning model training is performed on devices with limited resources, then model updates can be performed more frequently and efficiently, but the available memory and computation resources are insufficient to support traditional training methods
Solution Approach 1:
The patent segments the model update process into two distinct phases: inference phase where the model processes input data to generate output, and training phase where only the model parameters are updated using accumulated gradients. This segmentation allows the system to perform frequent updates without requiring full training infrastructure, resolving the contradiction between update frequency and resource availability.
Solution Approach 2:
The patent implements partial training action by accumulating gradients over multiple inference operations and performing a single parameter update step. Instead of completing full training iterations with forward and backward passes for each data point, the system accumulates gradient information and applies updates periodically, enabling frequent model adaptation with minimal computational overhead.
2Reliability
If traditional training methods are used, then model parameters can be optimized effectively, but extensive memory and computation resources are required
Solution Approach 1:
The patent extracts the essential training function from the complete training pipeline by separating gradient accumulation from full model retraining. The system extracts only the parameter update mechanism, using accumulated gradients to adjust model parameters without requiring the full training infrastructure including large batches of training data, multiple epochs, and extensive computational resources.
Solution Approach 2:
The system performs self-service training by maintaining and updating its own model parameters using accumulated gradient information from its own inference operations. The device trains itself continuously in the background using its operational data, eliminating the need for external training infrastructure and enabling reliable parameter optimization with minimal additional resources.
3Measurement precision
If full training procedures are implemented, then model accuracy can be improved, but the training process consumes significant time and computational power
Solution Approach 1:
The patent implements preliminary action by accumulating gradient information during the inference phase before performing parameter updates. Gradient accumulation occurs continuously in the background during normal model operations, so when an update is triggered, the model parameters are already optimized based on accumulated gradients, eliminating the need for time-consuming training waits and enabling rapid model improvement.
Data Source
AI summary
In one example, a method comprises providing first data to a machine learning model to generate second data. The method further comprises determining errors based on the second data and target second data; determining loss gradients based on the errors. The method further comprises updating running sums of prior loss gradients by adding the gradients to the running sums; and updating model parameters of the machine learning model based on the updated running sums.