Neural Network Weight Compression Using Padding-Bit Extraction

Solution Overview

Problem

Deep neural network processing requires large amounts of weight data, leading to high memory, system bandwidth, and power consumption due to the large amount of weights in quantized neural network models.

Innovation Solution

A method for compressing weight data by identifying patterns of padding bits in weight values, removing them to generate compressed frames, and using normalization factors to decompress the data efficiently, reducing the memory and power requirements.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If quantized neural network models use full precision weight values, then processing accuracy is maintained, but memory usage and power consumption increase significantly

Engineering Contradiction:

Improveprocessing accuracyVSAvoidpower consumption

Core Design Contradiction:

ReliabilityVSUse of energy by moving object

Solution Approach 1:

The patent extracts and removes redundant padding bits from weight values that do not contribute to the actual numerical value. By identifying and eliminating these unnecessary bits through normalization factors, the system reduces memory storage requirements and power consumption while preserving the essential information needed for accurate neural network processing.

Inventive Principle:

Principle #2Taking out (Extraction)

Solution Approach 2:

The patent changes the representation parameters of weight values by applying normalization factors that remove redundant padding bits. This parameter transformation allows the system to store and process weight values in a more efficient format, reducing the number of bits required per weight while maintaining processing accuracy through lossless compression.

Inventive Principle:

Principle #35Parameter changes

2Reliability

If quantized neural network models use full precision weight values, then processing accuracy is maintained, but memory usage increases significantly

Engineering Contradiction:

Improveprocessing accuracyVSAvoidmemory usage

Core Design Contradiction:

ReliabilityVSQuantity of substance

Solution Approach 1:

The patent extracts and removes redundant padding bits from weight values that do not contribute to the actual numerical value. By identifying and eliminating these unnecessary bits through normalization factors, the system reduces memory storage requirements and power consumption while preserving the essential information needed for accurate neural network processing.

Inventive Principle:

Principle #2Taking out (Extraction)

Solution Approach 2:

The patent changes the representation parameters of weight values by applying normalization factors that remove redundant padding bits. This parameter transformation allows the system to store and process weight values in a more efficient format, reducing the number of bits required per weight while maintaining processing accuracy through lossless compression.

Inventive Principle:

Principle #35Parameter changes

3Quantity of substance

If padding bits are removed from weight values for compression, then memory efficiency improves, but data representation complexity increases

Engineering Contradiction:

Improvememory efficiencyVSAvoiddata representation complexity

Core Design Contradiction:

Quantity of substanceVSDevice complexity

Solution Approach 1:

The patent applies preliminary normalization factors during the weight encoding stage to remove padding bits before storage. By performing this compression action in advance during model training or preprocessing, the system eliminates the need for complex decompression operations during inference, thereby improving memory efficiency without significantly increasing operational complexity during deployment.

Inventive Principle:

Principle #10Preliminary action

Data Source

PatentUS11757469B2Compression technique for deep neural network weights

Publication Date: 2023.09.12 QUALCOMM INC

AI summary

Various embodiments include methods and devices for compression and decompression of weight data sets. Some embodiments may include compressing weight data by receiving a weight data set of binary numbers representing weight values, generating a frame payload including a compressed first frame of a first subset of the weight values in the weight data set, and generating a block of compressed weight data having the frame payload. Some embodiments may include decompressing weight data by retrieving a block of compressed weight data, in which the block of compressed weight data includes a frame header associated with a frame payload, in which the frame header includes a normalization factor indicator, and in which the frame payload includes compressed weight values, and generating a first decompressed frame comprising decompressed weight values of the compressed weight values of the frame payload.