Neural Network Activation Compression Using Block-Wise Scheme Selection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Neural networks require significant processing bandwidth, power, and energy due to the large amount of uncompressed activation data, making efficient compression of this data a challenging task.
Innovation Solution
A processor-based system that splits neural network activation data into blocks, generates metadata for each block, selects an appropriate compression scheme based on the metadata, and applies lossless compression techniques such as masking, look-up table, value packing, and position packing to produce compressed data, optimizing storage and energy usage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If neural network activation data is transmitted uncompressed, then data accuracy and processing speed are maintained, but processing bandwidth, power, and energy consumption increase significantly
Solution Approach 1:
The patent divides neural network activation data into multiple blocks, allowing different compression schemes to be applied to different blocks based on their characteristics. This segmentation enables selective compression that maintains accuracy for critical data while compressing less critical data, thereby reducing overall energy consumption without sacrificing data integrity.
Solution Approach 2:
The patent applies different compression schemes to different blocks of activation data based on local characteristics such as sparsity patterns and value distributions. By tailoring the compression approach to each block's specific properties, the system optimizes the balance between compression efficiency and data reconstruction accuracy, reducing energy usage while maintaining where necessary.
2Quantity of substance
If compression is applied to neural network activation data, then storage requirements and energy consumption are reduced, but compression accuracy and processing complexity increase
Solution Approach 1:
The patent performs preliminary analysis of activation data blocks to identify their characteristics (such as sparsity and value ranges) before applying compression. This preliminary action includes generating metadata about each block's properties, which guides the selection of appropriate compression schemes, thereby simplifying the overall processing by avoiding trial-and-error approaches.
Solution Approach 2:
The patent changes compression parameters based on the characteristics of each data block. Different blocks may use different compression schemes (such as sparse representation, quantization, or encoding methods) depending on their sparsity patterns and value distributions. This parameter adaptation reduces processing complexity compared to applying a single complex compression algorithm to all data.
3Productivity
If a single compression scheme is applied to all activation data, then processing simplicity is maintained, but compression efficiency and adaptability to different data characteristics are reduced
Solution Approach 1:
The patent implements a dynamic compression system where the compression scheme is selected based on the characteristics of each data block. The system adapts its compression approach in real-time based on metadata analysis of sparsity patterns, value ranges, and other block-specific properties, thereby achieving high compression efficiency without requiring a fixed complex system architecture.
Solution Approach 2:
The patent creates a universal compression framework that can handle different types of activation data blocks using multiple compression schemes. The system is designed to select and apply the most appropriate compression method for each block based on its characteristics, making the system versatile and adaptable to various data patterns without requiring separate specialized systems for each data type.
Data Source
AI summary
A processor arranged to compress neural network activation data comprising an input module for obtaining neural network activation data. The processor also comprises a block creation module arranged to split the neural network activation data into a plurality of blocks; and a metadata generation module for generating metadata associated with at least one of the plurality of blocks. Based on the metadata generated a selection module selects a compression scheme for each of the plurality of blocks, and a compression module for applying the selected compression scheme to the corresponding block to produce compressed neural network activation data. An output module is also provided for outputting the compressed neural network activation data.