DNN Weight Bit Segmentation for Accuracy-Preserving Transmission
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The challenge lies in effectively compressing deep neural network (DNN) model parameters for transmission while preserving model accuracy, especially due to the large size of DNN models and the need to reduce bit precision during transmission.
Innovation Solution
A data processing system and method that quantize DNN model weights into specific bit types, interleave these bits, and apply different compression rates based on bit type compressibility, ultimately encoding and transmitting the compressed data.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If DNN model parameters are compressed by reducing bit precision during transmission, then transmission efficiency and storage requirements are improved, but model accuracy deteriorates due to information loss
Solution Approach 1:
The patent segments the 32-bit floating-point weight parameters into four distinct bit sets based on their compressibility characteristics: first bit set (highly compressible), second bit set (compressible), third bit set (less compressible), and fourth bit set (incompressible). This segmentation allows different compression strategies to be applied to different parts of the data, achieving overall compression while preserving critical information for model accuracy.
Solution Approach 2:
The patent applies different compression rates to different bit sets based on their local characteristics. The first bit set uses a first compression rate, the second bit set uses a second compression rate, and the third bit set uses a third compression rate. This local quality approach ensures that bits with different importance and compressibility are treated differently, optimizing the balance between compression ratio and accuracy preservation.
2Device complexity
If uniform compression rate is applied to all bit sets, then processing complexity is reduced, but compression efficiency deteriorates due to inability to exploit different compressibility characteristics
Solution Approach 1:
The patent introduces dynamic compression rates that adapt to the characteristics of different bit sets. Instead of using a fixed uniform compression rate, the system dynamically selects appropriate compression rates (first, second, and third compression rates) for different bit sets based on their compressibility. This dynamic approach significantly improves compression efficiency while the automated classification process keeps the additional processing complexity manageable.
Solution Approach 2:
The patent changes the compression rate parameter based on the bit set being processed. By adjusting the compression rate parameter to match the compressibility characteristics of each bit set, the system achieves optimal compression efficiency. The first bit set uses a first compression rate, the second bit set uses a second compression rate, and the third bit set uses a third compression rate, allowing each parameter group to be compressed at its optimal rate.
3Productivity
If important bits are compressed with high compression rates, then compression efficiency is improved, but reliability deteriorates due to potential loss of critical information
Solution Approach 1:
The patent converts the potential harm of compressing important bits into a benefit by first classifying bits into different sets based on their compressibility and importance. The incompressible bits (fourth bit set) and less compressible bits (third bit set) that contain critical information are either not compressed or compressed with lower compression rates. The highly compressible bits (first bit set) that are less critical are compressed with higher compression rates, thus converting the potential harm into beneficial compression efficiency.
Solution Approach 2:
The patent applies beforehand cushioning by identifying and protecting critical information bits before compression. The fourth bit set (incompressible) and third bit set (less compressible) are identified as containing important information and are either excluded from compression or compressed with conservative rates. This prior protection ensures that even if compression introduces errors, the critical information remains intact, maintaining model reliability.
Data Source
AI summary
A data processing method for a DNN model includes: reading weights of transmission data; quantizing each weight into bits sequentially including first, second, third, and fourth-type bits; sequentially interleaving the first-type bit into a first bit set; sequentially interleaving each second-type bit into second bit sets and reading a second compression rate of each second bit set in response to the compressible second bit sets; interleaving the third-type bit into a third bit set and reading a third compression rate of the third bit set in response to the compressible third bit set; compressing each second bit set with the second compression rate, and compressing the third bit set with the third compression rate; sequentially coding the first bit set, each compressed second bit set, and the compressed third bit set to generate first encoded data corresponding to the transmission data; transmitting the first encoded data to an external device.