Neural Network Parameter Compression With Shorter Codeword Tables
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional lossless compression algorithms for neural network parameters result in excessive storage space due to long maximum codeword lengths, which is not optimal and can be further optimized to reduce storage requirements.
Innovation Solution
A data compression method that involves obtaining parameter values and their occurrence probabilities, comparing them to a predetermined threshold to separate values for preprocessing and encoding, thereby controlling and shortening the maximum codeword length by using variable length encoding and padding techniques to form a balanced code table.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If conventional lossless compression algorithms (e.g., Huffman encoding) are used to compress neural network parameters, then the average codeword length is reduced and encoding efficiency is improved, but the maximum codeword length becomes excessively long, requiring large storage space for the code table
Solution Approach 1:
The patent segments the code table into multiple sub-code tables based on different codeword length ranges. Each sub-code table stores only the codewords within a specific length range, allowing the decoding process to selectively access only the relevant sub-code table based on the current codeword length, thereby reducing the storage space required for each individual code table while maintaining comprehensive coverage of all possible codewords
Solution Approach 2:
The patent introduces a new dimension for organizing the code table by adding a length-based classification structure. Instead of using a single flat code table, the system organizes codewords into multiple dimensions based on their length characteristics, allowing efficient access and reduced storage requirements through this hierarchical organization
2Volume of stationary object
If the maximum codeword length is reduced to decrease code table storage space, then the storage requirement is reduced, but the encoding efficiency and compression ratio deteriorate
Solution Approach 1:
The patent implements a dynamic code table selection mechanism where the system chooses the appropriate sub-code table based on the actual codeword length being decoded. This dynamic approach allows the system to maintain small, efficient sub-code tables for each length range while collectively providing comprehensive encoding efficiency across all possible codeword lengths
Solution Approach 2:
The patent changes the organizational parameter of the code table from a single unified structure to multiple structures differentiated by codeword length parameters. This parameter-based segmentation allows each sub-code table to be optimized for its specific length range, maintaining high encoding efficiency while reducing individual table sizes
3Adaptability or versatility
If a large code table is used to accommodate long maximum codeword lengths, then all possible codewords can be decoded, but the decoding complexity and hardware implementation cost increase significantly
Solution Approach 1:
The patent segments the large code table into multiple smaller sub-code tables organized by codeword length ranges. The decoding process first determines the length of the incoming codeword, then selects only the relevant sub-code table for decoding, significantly reducing the search space and computational complexity compared to searching through a single large code table
Solution Approach 2:
The patent performs preliminary classification of codewords by length before the actual decoding process. By pre-organizing codewords into length-based groups and determining the appropriate group first, the system eliminates the need to search through unrelated codewords, thereby reducing decoding complexity and improving processing efficiency
Data Source
AI summary
A data compression method, comprising: obtaining a plurality of values of a parameter and an occurrence probability of each of the plurality of values (S101); comparing the occurrence probability with a predetermined threshold, wherein values with the occurrence probability less than the predetermined threshold are first set of values, and values with the occurrence probability greater than or equal to the predetermined threshold are second set of values (S102); performing pretreatment on the first set of values (S103); and encoding the second set of values and the pretreated first set of values (S104). By means of the data compression method, the maximum codeword length can be effectively reduced, so as to reduce the requirements of a code table to the storage space.