AI Memory Chip Selecting Weight Subsets to Reduce Power
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The high power consumption in portable electronic devices performing convolutional neural network operations is due to the large number of parameters that need to be moved from external DRAM to the processing chip, as the cache memory is insufficient for storing these parameters, resulting in significant energy expenditure.
Innovation Solution
A memory chip with an AI engine that selects only important subsets of weight data for neural network operations, reducing the parameters that need to be moved and the frequency of movement by using a weight index to identify and store only the most critical subsets, thereby minimizing power consumption.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If all weight data are moved from external DRAM to processing chip for neural network operation, then operation accuracy is maintained, but power consumption increases significantly
Solution Approach 1:
The patent extracts and processes only the most important weight data subsets for neural network operations. The weight data is divided into multiple subsets, and only those with the highest importance scores (above a threshold) are selected and moved to the processing chip, rather than moving all weight data. This extraction approach maintains operation accuracy while significantly reducing the volume of data transfer and power consumption.
Solution Approach 2:
The patent applies local quality by differentiating between different subsets of weight data based on their importance. Each subset is assigned an importance score, and the system treats high-importance subsets differently (selecting them for processing) compared to low-importance subsets (discarding them). This localized differentiation allows the system to maintain accuracy for critical operations while reducing overall power consumption.
2Speed
If cache memory on processing chip is used to store weight data, then data access speed improves, but device complexity and cost increase
Solution Approach 1:
The patent segments the weight data into multiple subsets and stores them in the memory array in an organized manner. Each subset can be independently accessed by the AI engine, allowing for efficient data retrieval without requiring a large unified cache memory. This segmentation approach enables selective access to only the necessary data subsets, improving data access efficiency while avoiding the complexity of implementing extensive cache memory.
3Measurement precision
If more parameters are processed in neural network operation, then model accuracy improves, but the number of data movements from external memory increases
Solution Approach 1:
The patent performs preliminary action by pre-calculating importance scores for each weight data subset before the actual neural network operation. The AI engine uses these pre-computed scores to determine which subsets to load into the processing chip. This preliminary classification allows the system to prepare only the necessary data in advance, reducing the frequency of data movements during operation while maintaining model accuracy.
Data Source
AI summary
A memory chip capable of performing artificial intelligence operation and an operation method thereof are provided. The memory chip includes a memory array and an artificial intelligence engine. The memory array is configured to store input feature data and a plurality of weight data. The input feature data includes a plurality of first subsets, and each of the weight data includes a plurality of second subsets. The artificial intelligence engine includes a plurality of feature detectors. The artificial intelligence engine is configured to access the memory array to obtain the input feature data and the weight data. Each of the feature detectors selects at least one of the second subsets from the corresponding weight data as a selected subset based on a weight index, and the feature detectors perform a neural network operation based on the selected subsets and the corresponding first subsets.