Sparse CNN Accelerator Index Vector Computation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Convolutional neural networks (CNNs) face significant challenges in energy efficiency and computational requirements, particularly in mobile platforms like autonomous vehicles and cameras, due to the large computation needs and energy consumption during inference tasks, especially with deeper networks and higher input sizes.
Innovation Solution
A Sparse Convolutional Neural Network (SCNN) accelerator architecture that exploits weight and activation sparsity by eliminating multiplications with zero operands, using a compressed representation of weights and activations, and optimizing dataflow to reduce energy consumption and improve processing throughput.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional CNN inference is performed on mobile platforms, then classification accuracy can be achieved, but energy consumption becomes prohibitively high
Solution Approach 1:
The patent extracts and processes only the non-zero elements from sparse weight and activation matrices, eliminating computations involving zero operands. This is achieved through index-based representations that track positions of non-zero elements, allowing the system to perform computations only where necessary, thereby reducing energy consumption while maintaining classification accuracy
Solution Approach 2:
The patent applies different processing strategies to different parts of the data based on their sparsity characteristics. Non-zero elements are processed with full computational attention while zero elements are completely skipped. The system dynamically adapts computation based on the local sparsity pattern of each layer and operation, optimizing energy efficiency without sacrificing accuracy
2Adaptability or versatility
If deeper neural networks with larger input sets are deployed, then model capability improves, but computation requirements become prohibitively large
Solution Approach 1:
The patent extracts only the essential computational operations by identifying and processing non-zero elements in weight and activation matrices. This extraction approach allows deeper networks with larger input sets to be executed with reduced computational overhead, as zero-element computations are completely eliminated from the processing pipeline
Solution Approach 2:
The patent segments the computation into distinct phases: sparse matrix multiplication, sparsity pattern extraction, and result aggregation. By dividing the computational task into these segments and processing only the necessary portions (non-zero elements), the system can handle deeper and larger networks without proportionally increasing computation requirements
3Loss of energy
If sparse computation is implemented, then energy efficiency improves, but computational operations become more complex
Solution Approach 1:
The patent introduces index vectors as intermediary data structures that track the positions of non-zero elements. These index vectors act as mediators between the sparse input data and the computation units, enabling the system to efficiently identify and process only the necessary elements without requiring complex conditional logic throughout the entire computation pipeline
Solution Approach 2:
The patent performs preliminary extraction of sparsity patterns and generation of index vectors before the main computation phase. By pre-processing the data to identify non-zero element positions and organizing them into index-based representations, the system simplifies the subsequent computation operations, as the hardware only needs to perform straightforward computations on the indexed non-zero elements without complex zero-checking logic
Data Source
AI summary
A method, computer program product, and system perform computations using a processor. A first instruction including a first index vector operand and a second index vector operand is received and the first index vector operand is decoded to produce first coordinate sets for a first array, each first coordinate set including at least a first coordinate and a second coordinate of a position of a non-zero element in the first array. The second index vector operand is decoded to produce second coordinate sets for a second array, each second coordinate set including at least a third coordinate and a fourth coordinate of a position of a non-zero element in the second array. The first coordinate sets are summed with the second coordinate sets to produce output coordinate sets and the output coordinate sets are converted into a set of linear indices.