Neural Network Accelerator Structured Pruning Low-Bit Quantization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing neural network accelerators face challenges in improving performance and energy efficiency due to the irregularity caused by sparse convolutional neural networks, leading to increased power consumption and complexity in inference operations.
Innovation Solution
A neural network accelerator is designed using structured pruning and low-bit quantization, featuring a master controller, activations selection unit, extensible calculation array, and DMA, which optimizes data reusability, reduces power consumption, and improves throughput by sharing weights and activating data across processing elements.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If sparse convolutional neural networks are used for model compression, then the number of parameters is reduced, but irregularity in inference operations increases leading to higher power consumption
Solution Approach 1:
The patent segments the sparse convolutional computation into multiple processing elements (PEs) organized in an array structure. Each PE handles a specific portion of the computation, allowing parallel processing of sparse operations. This segmentation enables efficient utilization of computational resources while maintaining the benefits of model compression, thereby reducing power consumption compared to irregular sequential processing.
Solution Approach 2:
The patent changes the computational parameters by using low-bit quantization (e.g., 4-bit or 8-bit weights instead of 32-bit floats) and structured pruning to create regular sparse patterns. These parameter changes reduce the computational complexity and memory bandwidth requirements, directly lowering power consumption while preserving model accuracy.
2Loss of energy
If structured pruning and low-bit quantization are applied, then energy efficiency is improved, but device complexity increases
Solution Approach 1:
The patent designs a universal processing element (PE) that can handle multiple functions: sparse convolution operations, low-bit quantized multiplication, and accumulation. Each PE is multi-functional, capable of processing different types of operations within the same hardware structure. This universality reduces overall device complexity by avoiding the need for separate dedicated hardware for each operation type, while still achieving improved energy efficiency through structured pruning and low-bit quantization.
3Ease of manufacture
If conventional processors are used for large-scale convolutional neural networks, then implementation is straightforward, but performance and power consumption are limited
Solution Approach 1:
The patent introduces an intermediary layer between the input data and the processing elements: a weight decomposition unit that breaks down large convolution kernels into smaller blocks. This intermediary structure enables the accelerator to process large-scale neural networks by dividing the computation into manageable chunks that can be handled by the array of PEs, thereby improving performance while maintaining ease of implementation through modular design.
Data Source
AI summary
The present invention discloses a neural network accelerator and a neural network acceleration method based on structured pruning and low-bit quantization. The neural network accelerator includes a master controller, an activations selection unit, an extensible calculation array, a multifunctional processing element, a DMA, a DRAM and a buffer. The present invention makes full use of the data reusability during inference operation of a neural network, reduces the power consumption of selecting input activation and weights of effective calculations, and relieves the high transmission bandwidth pressure between the activations selection unit and the extensible calculation array through structured pruning and data sharing on the extensible calculation array, reduces the number of weight parameters and the storage bit width by combining the low-bit quantization technology, and further improves the throughput rate and energy efficiency of the convolutional neural network accelerator.