Bit-Interleaved Convolution Acceleration for Deep Learning Processors
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing deep learning accelerators face inefficiencies and complexity due to limited exploration of bit-level sparsity, particularly in predicting zero bit positions, leading to increased hardware complexity and reduced throughput, especially for floating-point operations.
Innovation Solution
A bit-interleaved computing method and hardware accelerator (Bitlet) that utilizes bit-level sparsity by aligning exponents, removing slack bits, and extracting essential bits for parallel computation, supporting both floating-point and fixed-point operations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If bit-level sparsity is utilized to improve computation efficiency, then productivity is improved, but device complexity increases due to synchronization operations and hardware circuit requirements
Solution Approach 1:
The patent segments the bit-level operations into parallel processing units (PEs), where each PE handles specific bit positions independently. This segmentation allows simultaneous computation across multiple bits without requiring complex synchronization, reducing hardware complexity while maintaining high productivity through parallel processing
Solution Approach 2:
The patent transitions from sequential bit processing to parallel bit processing by introducing a new dimension of computation. Multiple PEs process different bit positions simultaneously, transforming the computation paradigm from 1D sequential to multi-dimensional parallel execution, thereby improving productivity without proportionally increasing complexity
2Measurement precision
If synchronization operations are performed to ensure correct bit-level computation, then measurement precision is improved, but loss of time increases due to computation stopping and throughput reduction
Solution Approach 1:
The patent performs preliminary actions by pre-calculating and storing control signals that determine which bits need synchronization before the actual computation begins. This allows the synchronization to be handled through simple control logic rather than complex runtime coordination, maintaining accuracy while minimizing time loss
Solution Approach 2:
The patent ensures continuity of useful action by designing the parallel PE architecture where multiple computations proceed simultaneously without interruption. The synchronization mechanism is integrated into the continuous flow of computation rather than stopping it, allowing accurate bit-level operations to continue uninterrupted and maintain high throughput
3Productivity
If more computing resources are allocated to handle increasing model scale, then productivity is improved, but use of energy increases due to power consumption constraints
Solution Approach 1:
The patent changes the parameter of bit-width processing to achieve higher productivity without proportionally increasing energy consumption. By processing multiple bits in parallel across multiple PEs, the system achieves scaled-up computation performance while the energy consumption increases only linearly with the number of active PEs, rather than exponentially
Solution Approach 2:
The patent applies partial action by activating only the necessary number of PEs based on the specific computation requirements. Rather than always running maximum computing resources, the system dynamically adjusts the number of active PEs to match the workload, achieving high productivity for small models while maintaining low power consumption for embedded devices
Data Source
AI summary
The present application provides a deep learning convolution acceleration method using bit-level sparsity and a processor. Comprises: selecting the maximum sum of the exponents from all data pairs to be convolved as a maximum exponent; arranging mantissas of the original weights in a computation sequence to form a weight matrix, and uniformly aligning each row of the weight matrix to the maximum exponent and removing slack bits to obtain a reduced matrix, allowing essential bits in each column of the reduced matrix to fill the vacancies according to the computation sequence, after removing null rows in the intermediate matrix, placing zeros at vacancies of the matrix to obtain an interleaved weight matrix, sending the weight segments in each row of the interleaved weight matrix and the mantissa of the corresponding activation to an adder tree for processing summation, by shifting and adding the sum result to obtain a convolution result.