Floating-Point Compression Using Exponent Deduplication in Neural Networks

Solution Overview

Problem

Deep neural networks face challenges in balancing low power consumption and low memory bandwidth while maintaining high accuracy, with memory bandwidth often becoming a bottleneck due to the high data requirements of floating-point operations in modern computing.

Innovation Solution

A method and system for compressing floating-point numbers by re-ordering and removing duplicate exponents in neural network computations, allowing for lossless compression and decompression of data sequences, which reduces memory bandwidth without affecting accuracy.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If floating-point numbers with large bit width (16 or 32 bits) are used for high accuracy computations, then measurement precision is improved, but use of energy and memory bandwidth increase

Engineering Contradiction:

Improvecomputational accuracyVSAvoidpower consumption

Core Design Contradiction:

Measurement precisionVSUse of energy by moving object

Solution Approach 1:

The patent segments the floating-point number into three separate sequences: sign bits, exponents, and fractions. This segmentation allows independent compression of each sequence, particularly targeting the exponent sequence for deduplication. By separating these components, the patent reduces the overall bit width required for representation while maintaining the ability to reconstruct the original high-precision values for computation.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent changes the parameter representation by transforming the standard floating-point format into a compressed format where exponents are stored as differences from a base exponent. This parameter transformation enables significant compression when multiple operands share the same or similar exponents, reducing the bits required per operand while preserving the ability to recover the original precision when needed for computations.

Inventive Principle:

Principle #35Parameter changes

2Measurement precision

If floating-point numbers with large bit width (16 or 32 bits) are used for high accuracy computations, then measurement precision is improved, but memory bandwidth increases

Engineering Contradiction:

Improvecomputational accuracyVSAvoidmemory bandwidth

Core Design Contradiction:

Measurement precisionVSQuantity of substance

Solution Approach 1:

The patent segments the floating-point number into three separate sequences: sign bits, exponents, and fractions. This segmentation allows independent compression of each sequence, particularly targeting the exponent sequence for deduplication. By separating these components, the patent reduces the overall bit width required for representation while maintaining the ability to reconstruct the original high-precision values for computation.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent changes the parameter representation by transforming the standard floating-point format into a compressed format where exponents are stored as differences from a base exponent. This parameter transformation enables significant compression when multiple operands share the same or similar exponents, reducing the bits required per operand while preserving the ability to recover the original precision when needed for computations.

Inventive Principle:

Principle #35Parameter changes

3Use of energy by moving object

If compression is applied to reduce memory bandwidth, then use of energy and memory bandwidth are reduced, but device complexity increases due to compression and decompression operations

Engineering Contradiction:

Improvepower consumptionVSAvoidcompression circuit complexity

Core Design Contradiction:

Use of energy by moving objectVSDevice complexity

Solution Approach 1:

The patent applies local quality by treating different sequences (sign bits, exponents, fractions) differently during compression. The exponent sequence receives dedicated deduplication processing since it contains the most redundancy, while sign bits and fractions are handled with simpler operations. This localized approach optimizes compression effectiveness for the most redundant component without unnecessarily complicating the processing of all data.

Inventive Principle:

Principle #3Local quality

Solution Approach 2:

The patent performs preliminary re-ordering of operands to group those with identical exponents together before compression. This preliminary action enables the subsequent deduplication step to work more efficiently by ensuring that duplicate exponents are adjacent and easily identifiable. The base exponent is also determined in advance, allowing difference encoding to proceed systematically without requiring complex real-time analysis during compression.

Inventive Principle:

Principle #10Preliminary action

Data Source

PatentUS11742875B1Compression of floating-point numbers for neural networks

Publication Date: 2023.08.29 MEDIATEK INC

AI summary

Floating-point numbers are compressed for neural network computations. A compressor receives multiple operands, each operand having a floating-point representation of a sign bit, an exponent, and a fraction. The compressor re-orders the operands into a first sequence of consecutive sign bits, a second sequence of consecutive exponents, and a third sequence of consecutive fractions. The compressor then compresses the first sequence, the second sequence, and the third sequence to remove at least duplicate exponents. As a result, the compressor can losslessly generate a compressed data sequence.