Sparse Finetuning Hardware Accelerator for Neural Network Weight Updates
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Artificial neural networks (ANNs) face challenges in updating and switching between models due to the slow and power-hungry nature of changing weights in on-chip non-volatile memory, especially in mobile devices, where fast access is required for different ANN models with varying functionality.
Innovation Solution
A hardware accelerator that sparsely-finetunes ANN models by modifying a small number of weights while leaving others unchanged, allowing for efficient updating and switching between models, using a combination of communication bus interfaces, memory, and processing engines to apply sparsely-finetuned convolution operations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of energy
If weights are stored in on-chip non-volatile memory, then storage density and leakage are improved, but updating and context switching become slow and power-hungry
Solution Approach 1:
The patent segments the weight storage into two parts: fixed weights stored in on-chip non-volatile memory (NVM) and finetuned weights stored in volatile memory. This segmentation allows the system to maintain the energy efficiency of NVM for the majority of weights while enabling fast updates through volatile memory for only the necessary finetuned portion, thereby resolving the contradiction between low leakage and fast updating speed.
Solution Approach 2:
Instead of updating all weights in the neural network, the patent applies partial action by only finetuning a small subset of weights that are most critical for adapting to new tasks or contexts. This partial update approach stored in volatile memory enables rapid context switching with minimal power consumption, overcoming the speed and power limitations of updating entire weight sets in NVM.
2Reliability
If all weights are updated for model changes, then model accuracy is improved, but power consumption and update time increase significantly
Solution Approach 1:
The patent implements partial action by identifying and updating only the most critical weights (finetuned weights) that have the greatest impact on model performance for new tasks. These selected weights are stored in volatile memory for efficient updates, while the remaining fixed weights remain in NVM. This approach maintains model accuracy while dramatically reducing power consumption compared to updating all weights.
Solution Approach 2:
The patent applies local quality by treating different weights differently based on their importance and update frequency. Critical weights that require frequent updates are stored in volatile memory with fast access, while less critical weights remain in NVM. This differentiated storage strategy enables selective updates that preserve accuracy while minimizing energy expenditure.
3Adaptability or versatility
If context switching between multiple ANN models is enabled, then adaptability is improved, but access time increases due to NVM limitations
Solution Approach 1:
The patent segments the weight management system into fixed weights in NVM and finetuned weights in volatile memory. This segmentation enables efficient context switching by allowing the system to quickly load different finetuned weight sets from volatile memory for different contexts or tasks, while the fixed weights remain unchanged in NVM. This resolves the contradiction by providing fast access times through volatile memory for the portions of weights that need to change between contexts.
Solution Approach 2:
The patent applies preliminary action by pre-computing and storing finetuned weights in volatile memory before context switching is needed. When context switching is required, the system can immediately access the pre-prepared finetuned weights without performing time-consuming updates during the switch, thereby maintaining fast access times while enabling flexible adaptability across multiple contexts.
Data Source
AI summary
The present disclosure advantageously provides a hardware accelerator for an artificial neural network (ANN), including a communication bus interface, a memory, a controller, and at least one processing engine (PE). The communication bus interface is configured to receive a plurality of finetuned weights associated with the ANN, receive input data, and transmit output data. The memory is configured to store the plurality of finetuned weights, the input data and the output data. The PE is configured to receive the input data, execute an ANN model using a plurality of fixed weights associated with the ANN and the plurality of finetuned weights, and generate the output data. Each finetuned weight corresponds to a fixed weight.