Sparse Weight Matrix Processing for Neural Network Energy Efficiency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Convolutional neural networks face challenges in implementing efficient computing architectures for embedded systems due to high electrical consumption and the need to minimize multiply-accumulate operations, especially when dealing with low data density and zero weights, which are not adequately addressed by existing solutions.
Innovation Solution
A computer architecture with a flow management circuit that identifies zero data and synchronizes weight reading using skip information, optimizing MAC operations by processing low-density non-zero data and weights jointly, reducing energy consumption and computing time.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If conventional neural network computing architectures are used, then the network can perform standard MAC operations, but electrical consumption increases and computing efficiency decreases due to processing zero weights and low-density data
Solution Approach 1:
The patent extracts and processes only the non-zero elements from the weight matrix and input data. By identifying and removing zero-weight connections and low-density data elements before processing, the system eliminates unnecessary MAC operations, directly reducing electrical consumption while maintaining computing accuracy for the relevant data subset.
Solution Approach 2:
The patent performs preliminary actions by pre-processing the weight matrix and input data to identify non-zero elements, compute their positions, and reorganize them into compact representations before the main computing phase. This preliminary organization enables the computing units to process only essential data, improving productivity and reducing energy waste during inference and training operations.
2Productivity
If standard MAC operations are performed on all data including zero weights, then complete computational accuracy is maintained, but the number of computing operations increases unnecessarily
Solution Approach 1:
The patent extracts only the non-zero weight elements and their corresponding input data indices, creating a compressed representation that excludes all zero-weight connections. This extraction reduces the number of MAC operations from the full matrix size to only the necessary non-zero elements, directly improving productivity without sacrificing computational accuracy for relevant connections.
Solution Approach 2:
The patent changes the parameter representation by transforming the dense weight matrix into a sparse format with explicit non-zero element indexing. By representing weights as a list of non-zero values with their positions rather than a full matrix, the system reduces the operational parameter count, thereby reducing computing time while maintaining accuracy for non-zero connections.
3Productivity
If dense data representation is used, then all data including zeros is processed uniformly, but memory bandwidth utilization decreases and processing efficiency drops
Solution Approach 1:
The patent extracts and processes only the non-zero data elements by identifying their positions and values, then reorganizing them into a compact format for processing. This extraction eliminates the processing overhead of zero elements while maintaining the complete information content, directly improving data processing efficiency by focusing computational resources on meaningful data.
Solution Approach 2:
The patent performs preliminary data reorganization by pre-processing input data and weight matrices to identify and group non-zero elements, creating optimized data structures before processing. This preliminary action transforms dense representations into sparse, efficiency-optimized formats, improving productivity by ensuring that subsequent processing operates on compact, high-density data subsets.
Data Source
AI summary
A computing circuit for computing a weighted sum of a set of first data using at least one parsimony management circuit includes a first buffer memory for storing all or some of the first data delivered sequentially and a second buffer memory for storing all or some of the second data delivered sequentially. The parsimony management circuit furthermore comprises a first processing circuit able: to analyze the first data in order to search for the first non-zero data and define a first skip indicator between two successive non-zero data, and to control the transfer, to the distribution circuit, of a first datum read from the first data buffer memory on the basis of the first skip indicator. The parsimony management circuit furthermore comprises a second processing circuit able to control the transfer, to the distribution circuit, of a second datum read from the second data buffer memory on the basis of the first skip indicator.