Hardware Entropy Coding Between Memory and Processor

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing data compression and decompression techniques often implemented in software fail to effectively address the memory bandwidth bottleneck in processing systems, particularly when handling machine learning workloads, and assume a minimum sparsity in data that may not be applicable to all neural networks.

Innovation Solution

A hardware-based system that uses entropy encoding to compress and decompress data by generating a probability distribution of codewords and storing them in a code table, allowing for efficient transfer of data between processors and memory devices, and includes a decompressor device to process compressed data and send it to processors, while a compressor device compresses data for storage in memory devices.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Productivity

If data is transferred uncompressed between memory devices and processors, then transfer speed is maintained, but memory bandwidth is bottlenecked and energy consumption increases

Engineering Contradiction:

Improvememory bandwidthVSAvoidcompression system complexity

Core Design Contradiction:

ProductivityVSDevice complexity

Solution Approach 1:

A dedicated compression device is introduced as an intermediary component between the processor and memory device. This compression device includes a compression engine that compresses data before it is stored in memory and a decompression engine that decompresses data when retrieved from memory. This intermediary handles the compression/decompression operations, allowing the processor and memory device to operate at their full bandwidth without being bottlenecked by compression overhead.

Inventive Principle:

Principle #24Intermediary (Mediator)

Solution Approach 2:

Data is compressed in advance before being written to memory, and decompression is performed in advance before data is processed by the processor. The compression engine compresses data as it is being written to memory, and the decompression engine decompresses data as it is being read from memory, so that the processor always receives uncompressed data without waiting for compression or decompression operations.

Inventive Principle:

Principle #10Preliminary action

2Quantity of substance

If entropy encoding with code tables is used, then compression ratio improves, but processing time and computational expense increase

Engineering Contradiction:

Improvedata compression ratioVSAvoidprocessing time

Core Design Contradiction:

Quantity of substanceVSLoss of time

Solution Approach 1:

The patent replaces software-based entropy encoding algorithms with hardware-based compression engines that use dedicated logic circuits and lookup tables. The compression engine includes a probability distribution calculator and code table generator that operate in hardware, and the decompression engine uses hardware-based lookup tables to quickly decode compressed data. This hardware implementation eliminates the computational overhead of software processing while maintaining the compression efficiency of entropy encoding.

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

Solution Approach 2:

Code tables are generated in advance based on the probability distribution of codewords in the data being compressed. The compression engine calculates the probability distribution and generates optimized code tables before compression, allowing for efficient encoding. The decompression engine uses pre-generated code tables to quickly decode data without real-time computation.

Inventive Principle:

Principle #10Preliminary action

3Quantity of substance

If more memory devices are used to increase storage capacity, then storage capacity increases, but system cost and energy consumption increase

Engineering Contradiction:

Improvestorage capacityVSAvoidenergy consumption

Core Design Contradiction:

Quantity of substanceVSUse of energy by moving object

Solution Approach 1:

The patent changes the state of data from uncompressed to compressed form before storage. By compressing data using the compression engine, the same amount of information is stored in fewer bits, effectively increasing the storage capacity of existing memory devices without adding more memory. This parameter change (from uncompressed to compressed) allows more data to be stored in the same physical memory space.

Inventive Principle:

Principle #35Parameter changes

Data Source

PatentUS11728826B2Compression and decompression in hardware for data processing

Publication Date: 2023.08.15 GOOGLE LLC

AI summary

Methods, systems, and apparatus, including computer-readable storage media for hardware compression and decompression. A system can include a decompressor device coupled to a memory device and a processor. The decompressor device can be configured to receive, from the memory device, compressed data that has been compressed using an entropy encoding, process the compressed data using the entropy encoding to generate uncompressed data, and send the uncompressed data to the processor. The system can also include a compressor device configured to generate, from uncompressed data, a probability distribution of codewords, generate a code table from the probability distribution, and compress incoming data using the generated code table.