Neural Network Decompression Decoder for Parallel Weight Expansion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current deep learning model compression techniques, such as pruning and quantization, face challenges in maintaining accuracy while increasing compression rates and enabling parallel processing during decompression, as they require complex processes to restore original weight matrices from quantized data.
Innovation Solution
A decompression apparatus and method that includes a memory for storing compressed data, a decoder with logic circuits for decompressing data, and a processor for obtaining neural network processible data, utilizing representative value matrices, pruning index matrices, and patch information to update and de-interleave data, allowing for efficient neural network processing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If quantization and pruning are applied to compress deep learning models, then compression rate increases, but the process to restore original weight matrices becomes complex and parallel processing is hindered
Solution Approach 1:
The weight matrix is divided into multiple sub-matrices, and the decompression process is segmented into parallel operations. Each sub-matrix can be processed independently through the logic circuits, enabling parallel decomposition without requiring complex sequential restoration of the entire matrix.
Solution Approach 2:
A codebook is introduced as an intermediary structure that stores pre-computed decomposition information. The logic circuits use this codebook to efficiently decompress quantized data into approximate weight values without performing complex restoration calculations, thereby simplifying the decompression process while maintaining parallel processing capability.
2Productivity
If quantized data is processed without restoring original weight matrix, then parallel processing is enabled, but processing accuracy may be degraded
Solution Approach 1:
Instead of restoring the exact original weight matrix, the system creates approximate copies using decompressed values from the codebook. These approximate weight matrices are sufficient for inference tasks and enable parallel processing while maintaining acceptable accuracy levels.
Solution Approach 2:
The system changes the precision parameter of weight representation by using quantized values and approximate decompression rather than full-precision original values. This parameter change enables parallel processing through logic circuits while the accuracy degradation is managed through the codebook-based approximation strategy.
Data Source
AI summary
A decompression apparatus is provided. The decompression apparatus includes a memory configured to store compressed data decompressed and used in neural network processing of an artificial intelligence model, a decoder configured to include a plurality of logic circuits related to a compression method of the compressed data, decompress the compressed data through the plurality of logic circuits based on an input of the compressed data, and output the decompressed data, and a processor configured to obtain data of a neural network processible form from the data output from the decoder.