CNN Weight Kneading for Faster Zero-Bit Convolution
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing deep convolutional neural network (DCNN) accelerators face challenges in efficiently performing convolution operations due to the high percentage of zero bits in weights, leading to invalid computations and reduced performance on lightweight devices.
Innovation Solution
The proposed solution involves a weight kneading technique that identifies and replaces slack bits in weights with essential bits from subsequent weights, creating a kneading matrix that reduces the number of weights and accelerates convolution operations by utilizing a split accumulator architecture.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If traditional multiply-adder architecture is used for convolution operation, then computing accuracy is maintained, but computing efficiency is reduced due to invalid operations on zero bits
Solution Approach 1:
The patent extracts and removes slack bits (zero bits) from weight representations, creating a compressed weight format that eliminates invalid computation elements. This extraction principle directly addresses the contradiction by removing the source of wasted computational operations while preserving essential information needed for accurate results.
Solution Approach 2:
The patent changes the parameter representation of weights by transitioning from fixed-bit formats to variable-length formats based on essential bit positions. This parameter change allows the system to adaptively represent weights with different precision requirements, eliminating computations on zero bits while maintaining necessary accuracy for the convolution operation.
2Speed
If fixed-point multiplication is decomposed into single-bit multiplications for acceleration, then computing speed is improved, but latency increases due to worst-case cycle design
Solution Approach 1:
The patent introduces dynamic adaptation by adjusting the number of computation cycles based on the actual distribution of essential bits in weights. Instead of designing for the static worst-case scenario, the system dynamically determines the required cycles based on actual weight characteristics, thereby improving speed without incurring unnecessary latency overhead.
3Use of energy by moving object
If 8-bit fixed-point multipliers are used in DCNN accelerator, then power consumption is reduced, but delay increases as multiplier is 3.4 times slower than adder
Solution Approach 1:
The patent extracts and removes slack bits from weights, which reduces the number of multiplication operations required. This extraction directly addresses the delay issue by eliminating operations that would otherwise be performed by slow multipliers, while the reduced computational workload also contributes to lower power consumption.
Solution Approach 2:
The patent changes the effective precision parameter of weight representations based on actual essential bit distribution. By using variable precision instead of fixed 8-bit format, the system reduces unnecessary computations on zero bits, thereby decreasing multiplier usage and associated delays while maintaining energy efficiency.
4Measurement precision
If DCNN architecture uses massive neurons and synapses for high accuracy, then recognition accuracy is improved, but device complexity increases on lightweight devices
Solution Approach 1:
The patent changes the representation parameters of weights from fixed precision to variable precision based on essential bit analysis. This parameter change allows the system to maintain high recognition accuracy by preserving essential weight information while reducing the effective number of parameters that need to be stored and processed, thereby lowering device complexity on lightweight platforms.
Data Source
AI summary
Disclosed embodiments relate to a convolutional neural network computing method and system based on weight kneading, comprising: arranging original weights in a computation sequence and aligning by bit to obtain a weight matrix, removing slack bits in the weight matrix, allowing essential bits in each column of the weight matrix to fill the vacancies according to the computation sequence to obtain an intermediate matrix, removing null rows in the intermediate matrix, obtain a kneading matrix, wherein each row of the kneading matrix serves as a kneading weight; obtaining positional information of the activation corresponding to each bit of the kneading weight; divides the kneading weight by bit into multiple weight segments, processing summation of the weight segments and the corresponding activations according to the positional information, and sending a processing result to an adder tree to obtain an output feature map by means of executing shift-and-add on the processing result.