Neural Processor Kernel Decompression via Lookup Tables
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods for performing neural network operations rely heavily on CPUs, leading to significant bandwidth consumption and increased power usage due to the extensive computational requirements of machine learning systems like CNNs, which are not optimized for efficient kernel operations.
Innovation Solution
A neural processor circuit utilizing a look-up table (LUT) to decompress kernel coefficients, where indices from compressed data identify entries in the LUT for assembling uncompressed kernels, and a block sparse mask is used to indicate zero values, reducing computational load and power consumption.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If CPU is used to perform neural network operations, then ease of operation is improved, but power consumption increases
Solution Approach 1:
The patent replaces the general-purpose CPU (mechanical/electronic computing system) with a specialized neural processing unit that uses lookup tables and parallel processing circuits. This substitution maintains operational capability while dramatically reducing power consumption by avoiding the overhead of general-purpose CPU execution for specialized neural network computations.
Solution Approach 2:
The patent uses lookup tables that store pre-computed kernel coefficient data in a compressed format. Instead of performing full computational operations, the system copies relevant data from these lookup tables and assembles the required kernel coefficients, significantly reducing the computational burden and power consumption while maintaining the ability to perform various neural network operations.
2Adaptability or versatility
If CPU is used to perform neural network operations, then adaptability is improved, but bandwidth consumption increases
Solution Approach 1:
The lookup tables store compressed representations of kernel data that can be quickly copied and assembled into full kernels. This copying mechanism allows the system to adapt to different neural network configurations by selecting and assembling appropriate kernel coefficients from the lookup tables, reducing the bandwidth required to transfer full kernel data while maintaining adaptability.
Solution Approach 2:
The patent pre-computes and stores kernel coefficient data in lookup tables in a compressed format during an offline preparation phase. This preliminary action allows the runtime system to efficiently assemble required kernels by copying and combining pre-prepared data, reducing the bandwidth consumption during actual neural network operations while maintaining the ability to handle various configurations.
3Quantity of substance
If compressed kernel data is used, then memory efficiency is improved, but processing complexity increases
Solution Approach 1:
The patent segments the kernel data into multiple lookup tables, where each table stores a portion of the compressed kernel coefficients. The kernel extract circuit then assembles the required kernel by selectively combining data from multiple lookup tables based on indices. This segmentation approach maintains memory efficiency while managing processing complexity through modular organization.
Solution Approach 2:
The patent introduces an intermediary kernel extract circuit that acts as a mediator between the compressed lookup tables and the neural processing units. This intermediary component handles the complexity of decompression and assembly operations, translating the compact stored format into the required kernel format without requiring the neural processing units themselves to handle the complex decompression logic.
Data Source
AI summary
Embodiments of the present disclosure relate to decompressing a kernel for neural network operations in a neural processor circuit, using a look-up table (LUT) with each of its entries associated with a plurality of kernel coefficients. Index data in compressed kernel data includes indices that indicate entries in the LUT. During decompression, all kernel coefficients in entries as indicated by the indices of the index data are retrieved and assembled into the decompressed kernel. A block sparse mask may also be used to indicate a block of locations in the uncompressed kernel to be filled with zero values. Only one or more blocks of locations indicated by the block sparse mask to include at least one none-zero kernel coefficient may be populated with the kernel coefficients from the LUT while remaining blocks of locations are padded with zero.