Neural Network Processing Module for Sparse Convolutional Networks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing neural network operations on GPUs face significant overhead due to the lack of specific support for sparse convolutional neural networks, leading to performance bottlenecks and high power consumption.
Innovation Solution
A neural network operation device and method that processes data to reduce the workload of decoding before performing an artificial neural network operation, utilizing a mapping unit to process input data, storage unit to store processed data, and operation unit to perform neural network operations, thereby reducing extra overhead and increasing operating speed.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If GPU is used for sparse neural network operations, then general computing capability is maintained, but decoding overhead increases and operating speed decreases
Solution Approach 1:
The device segments the neural network processing pipeline into distinct functional units: a mapping unit that handles sparse data identification and reorganization, storage units for intermediate results, and operation units for computation. This segmentation allows each unit to be optimized for its specific function, with the mapping unit eliminating the need for extensive decoding by pre-organizing sparse data in an efficient format before computation.
Solution Approach 2:
The mapping unit acts as an intermediary between the input data and the computation units. It transforms the sparse neural network data into an optimized intermediate representation that reduces decoding overhead downstream, enabling the computation units to operate efficiently without processing the full decoding burden.
2Adaptability or versatility
If GPU is used for sparse neural network operations, then general-purpose computing is maintained, but power consumption increases due to repeated off-chip memory access
Solution Approach 1:
The device implements a nested memory hierarchy with multiple levels of storage units embedded within the processing device. Frequently accessed weight data and intermediate results are stored in on-chip storage units, while less frequently accessed data resides in off-chip memory. This nested structure reduces the frequency of costly off-chip accesses, thereby lowering power consumption while maintaining general-purpose computing capability.
3Productivity
If sparse neural network structure is used, then the number of weights and computation is reduced, but irregular weight distribution increases decoding complexity
Solution Approach 1:
The mapping unit performs preliminary action by pre-processing the sparse neural network data before it reaches the computation units. It identifies the irregular non-zero weight patterns, reorganizes them into a structured format, and prepares the data in advance. This preliminary reorganization eliminates the need for complex real-time decoding during computation, reducing both decoding complexity and computation time.
Data Source
Figure 1a~2
Figure 2a~5a
Figure 5b~6b
AI summary
The present disclosure discloses a neural network processing module, in which a mapping unit is configured to receive an input neuron and a weight, and then process the input neuron and/or the weight to obtain a processed input neuron and a processed weight; and an operation unit is configured to perform an artificial neural network operation on the processed input neuron and the processed weight. Examples of the present disclosure may reduce additional overhead of the device, reduce the amount of access, and improve efficiency of the neural network operation.