Neural Network Acceleration Processor for Sparse Matrix Operations
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current general-purpose processors and GPUs are inefficient in calculating and memory access for large-scale neural networks due to high calculation and memory access requirements, leading to performance bottlenecks and power consumption issues, especially when handling sparse neural networks.
Innovation Solution
A neural network acceleration processor is designed with a connection value generator, pruning module, and computing unit to modify input data and weight values based on connection values, reducing the need for index array retrieval and optimizing calculations by generating connection values and modifying data to eliminate zero-weight connections.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If a general-purpose processor is used to calculate sparse neural networks, then the system can handle various computing tasks, but the calculation and memory access overhead increases significantly
Solution Approach 1:
The processor is segmented into specialized functional units including a sparse matrix multiplication unit, a vector operation unit, and a memory management unit. This segmentation allows each unit to handle specific neural network operations efficiently, reducing overall calculation time while maintaining processing versatility through the coordinated work of multiple specialized units
Solution Approach 2:
An index array is introduced as an intermediary data structure to map non-zero weight values to their corresponding positions. This intermediary allows the system to quickly locate relevant weight values without searching through entire weight matrices, significantly reducing memory access overhead while preserving the ability to handle various neural network configurations
2Adaptability or versatility
If a general-purpose processor is used to calculate sparse neural networks, then the system can handle various computing tasks, but the memory access overhead increases
Solution Approach 1:
Zero-weight connections are extracted and removed from the computation entirely. The sparse matrix multiplication unit only processes non-zero weight values, and the index array selectively maps only these relevant connections. This extraction eliminates unnecessary memory accesses to zero-weight values, reducing power consumption while maintaining versatility through configurable sparse matrix handling
Solution Approach 2:
The index array is pre-computed and stored to map non-zero weight values to their positions before the actual neural network inference or training begins. This preliminary action allows subsequent computations to directly access required weight values without repeated searches, reducing memory access overhead and power consumption during actual processing
3Productivity
If multiple general-purpose processors work concurrently, then calculation capacity increases, but inter-processor communication becomes a performance bottleneck
Solution Approach 1:
Multiple processing units are merged into a unified neural network processing system with shared memory resources and coordinated control. The sparse matrix multiplication units work in parallel on different portions of the computation while sharing the index array and weight value storage, increasing throughput without requiring complex inter-processor communication protocols
4Productivity
If pruning is applied to reduce weight values, then calculation and memory access decrease, but each multiplication operation needs to re-search positions in the index array
Solution Approach 1:
The index array is constructed and stored in advance during the pruning process, mapping the positions of non-zero weight values before computation begins. This preliminary indexing eliminates the need to re-search positions during subsequent multiplication operations, as the mapped positions are directly accessible through the pre-computed index array
Solution Approach 2:
The index array serves as a copied reference structure that preserves the positional information of non-zero weight values after pruning. Instead of searching through the original weight matrix during computation, the system copies and uses the pre-computed index mappings to directly access relevant weight values, reducing search time while maintaining calculation efficiency
Data Source
AI summary
Aspects of data modification for neural networks are described herein. The aspects may include a connection value generator configured to receive one or more groups of input data and one or more weight values and generate one or more connection values based on the one or more weight values. The aspects may further include a pruning module configured to modify the one or more groups of input data and the one or more weight values based on the connection values. Further still, the aspects may include a computing unit configured to update the one or more weight values and/or calculate one or more input gradients.