Kernel Vector Palettization Using LUT Decompression in Neural Processors
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Neural network operations, particularly in machine learning systems like CNNs, consume significant computing system bandwidth and increase power consumption due to extensive multiplication and accumulation operations.
Innovation Solution
A neural processor circuit employs a look-up table (LUT) to store kernel coefficients, using index data to retrieve and assemble kernel data, with a kernel access circuit and neural engine circuit to perform decompression and neural network operations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If kernel data is stored in uncompressed format, then neural network operations can be performed directly without decompression, but computing system bandwidth and power consumption increase significantly
Solution Approach 1:
The patent applies parameter changes by transforming kernel data from uncompressed format to a compressed palettized format with associated index data. This changes the data representation parameters (from raw coefficients to palette indices), enabling more efficient storage and transmission while maintaining the ability to perform neural network operations through on-the-fly decompression using LUTs in the neural processing unit
2Use of energy by moving object
If kernel data is compressed using palettization with LUT, then bandwidth and power consumption are reduced, but additional decompression operations are required
Solution Approach 1:
The patent implements self-service by integrating the decompression functionality directly within the neural processing unit. The LUTs and kernel extract circuit are built-in components that automatically perform decompression as part of the neural network operation pipeline, eliminating the need for separate external decompression hardware and reducing overall system complexity despite the added decompression capability
3Quantity of substance
If palettized kernel data is stored, then storage bandwidth is reduced, but more memory access operations are needed for decompression
Solution Approach 1:
The patent applies segmentation by dividing the kernel data into palettized segments with associated index data. Instead of storing and accessing entire uncompressed kernel matrices, the system stores compact palettized representations and retrieves only the necessary index data and LUT entries needed for specific neural network operations, reducing both storage volume and access time
Data Source
AI summary
Embodiments of the present disclosure relate to decompressing a kernel for neural network operations in a neural processor circuit using a look-up table (LUT) with each of its entries associated with kernel coefficients. Index data in compressed kernel data includes indices, such as a first index and a second index that identify entries in the LUT. A kernel extract circuit is configured to extract the LUT and index data from the kernel data, and assemble an uncompressed kernel data by combining first kernel coefficients identified by the first index with second kernel coefficients identified by the second index.