Neural Network Processor Weight Compression Architecture
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
General processors are inadequate for multi-layer artificial neural network operations due to low computing performance, and GPUs incur high power consumption and additional costs despite their inefficiencies in supporting these operations.
Innovation Solution
A neural network processor with a floating-point number converter that reduces the bit length of weight values and a computing unit that calculates output neuron data and weight gradients, optimizing performance while minimizing power consumption.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If general processors are used to process multi-layer artificial neural network operations, then the system can handle various computational tasks, but the computing performance is insufficient to meet the needs of neural network operations
Solution Approach 1:
The patent segments the neural network processing into distinct functional units: a computing unit for forward propagation operations, a separate converter for weight value bit-length conversion, and a gradient calculation unit for backpropagation. This segmentation allows each unit to be optimized for its specific function, achieving high computing performance for neural network operations while maintaining the ability to handle various neural network tasks through configurable parameters and data structures.
2Productivity
If multiple general processors work concurrently to improve computing performance, then processing capacity increases, but intercommunication between processors limits performance and increases system complexity
Solution Approach 1:
The patent merges the functions of multiple processors into a single integrated neural network processing system. The computing unit, converter, and gradient calculation unit work together as a unified architecture, eliminating the need for complex inter-processor communication while achieving high processing capacity through internal parallelization and optimized data flow between functional units.
3Productivity
If general processors transcode multi-layer artificial neural network operations into long sequences of instructions, then the operations can be executed, but power consumption increases significantly
Solution Approach 1:
The patent replaces the general-purpose instruction execution mechanism with specialized hardware circuits designed specifically for neural network operations. The computing unit uses dedicated multipliers and adders for dot product calculations, and the converter uses specialized logic for weight value bit-length conversion, eliminating the need for complex instruction decoding and execution sequences, thereby significantly reducing power consumption while maintaining operation execution capability.
4Productivity
If GPUs are used to support multi-layer artificial neural network operations, then the algorithms can be executed, but front-end transcoding is still required which incurs large additional costs and high power consumption
Solution Approach 1:
The patent implements self-service by integrating the weight value bit-length conversion function directly into the neural network processing pipeline. The converter is configured to automatically convert weight values from a first bit length to a second bit length before they are used in computing operations, eliminating the need for external transcoding processes and reducing both device complexity and power consumption while maintaining algorithm execution capability.
5Volume of stationary object
If GPUs with small on-chip cache are used, then the hardware footprint is reduced, but model data must be carried repeatedly from outside the chip which makes off-chip bandwidth the main bottleneck and causes huge power consumption
Solution Approach 1:
The patent applies parameter changes by dynamically adjusting the bit length of weight values during different phases of neural network processing. During forward propagation, weight values are converted to a compressed bit representation for efficient computation, and during backpropagation, the same weights are converted to an expanded bit representation for accurate gradient calculation. This parameter changing approach reduces the memory bandwidth requirement and power consumption while maintaining computational accuracy.
Data Source
AI summary
A compression coding apparatus for artificial neural network, including memory interface unit, instruction cache, controller unit and computing unit, wherein the computing unit is configured to perform corresponding operation to data from the memory interface unit according to instructions of controller unit; the computing unit mainly performs three steps operation: step one is to multiply input neuron by weight data; step two is to perform adder tree computing and add the weighted output neuron obtained in step one level-by-level via adder tree, or add bias to output neuron to get biased output neuron; step three is to perform activation function operation to get final output neuron. The present disclosure also provides a method for compression coding of multi-layer neural network.