Bit-Field Masking Compression for Sparse ML Data Packets
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current machine learning networks face challenges in reducing model size and bandwidth requirements while maintaining high inference performance and low power consumption, particularly due to the inefficiencies in compressing sparse data.
Innovation Solution
The implementation of universal bit-field masking coding (UBFMC) for compressing machine learning data and parameters, which leverages sparsity information to generate compressed sub-packets with reduced header overhead, supporting both random and structured sparsity, and utilizing a two-stage packet header for efficient data packing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If conventional compression methods are used for machine learning data, then model size is reduced, but inference performance and power efficiency deteriorate due to loss of sparsity information
Solution Approach 1:
The patent extracts sparsity information from machine learning data by identifying and separating sparse values (zeros or near-zeros) from non-sparse values. This extracted sparsity information is then used to create a compressed representation that maintains the essential characteristics of the original data while reducing model size, thereby resolving the contradiction between compression and performance preservation
Solution Approach 2:
The patent changes the representation parameters of machine learning data by transforming it into a compressed format that explicitly encodes sparsity patterns. This parameter transformation allows the data to be stored more efficiently while retaining the information needed for accurate inference, thus reducing model size without sacrificing performance
2Quantity of substance
If conventional compression methods are used for machine learning data, then bandwidth requirements are reduced, but power consumption increases due to inefficient processing of compressed data
Solution Approach 1:
The patent performs preliminary compression of machine learning data using sparsity-aware encoding before transmission or storage. By pre-processing the data into a compressed format that preserves sparsity information, the system reduces the amount of data that needs to be transmitted over the network, thereby reducing bandwidth usage and associated power consumption without requiring additional processing during inference
3Measurement precision
If sparsity information is preserved in compressed data, then inference accuracy is maintained, but data structure complexity increases
Solution Approach 1:
The patent segments compressed data into distinct components: sparsity information (indicating positions of non-sparse values) and actual data values. This segmentation allows the system to maintain inference accuracy by preserving both components while managing complexity through structured organization, where each segment has a specific purpose and can be processed independently
4Quantity of substance
If model size is reduced through compression, then power budget efficiency improves, but computational performance may deteriorate due to decompression overhead
Solution Approach 1:
The patent designs the compressed data structure to be self-descriptive, where the sparsity information and data values are organized in a format that can be directly utilized by the inference engine without requiring complex decompression algorithms. The compressed structure essentially serves itself by maintaining the necessary information in an accessible format, thereby improving power budget efficiency while minimizing decompression overhead and preserving computational performance
Data Source
AI summary
Systems and techniques are provided for compressing data. A process can include generating a compressed sub-packet by removing one or more sparsity bytes from a sequence of values corresponding to a sub-packet, the sequence of values including one or more sparsity bytes each equal to a configured sparsity value and one or more non-sparsity bytes each corresponding to a respective data value different from the configured sparsity value. A sub-packet header can be generated for the compressed sub-packet, and indicative of a respective location within the sequence of values of each non-sparsity byte. A packet header can be generated for a plurality of compressed sub-packets, and indicative of the configured sparsity value and respective coding information for each compressed sub-packet. A compressed data packet can be generated to include at least the packet header, the sub-packet header, and the one or more non-sparsity bytes included in the sequence of values.