Sparse Matrix Compression Using Relaxed Pruning and Huffman Coding

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Existing neural network models face high costs in transmitting and processing matrix data, especially when dealing with sparse matrices, as the compression rate decreases with increasing matrix size, limiting efficiency in acceleration hardware.

Innovation Solution

A method and apparatus that determine 'don't-care' elements in a matrix based on threshold values, generate bitstreams, and apply Huffman coding to efficiently process and compress matrix data, improving data classification and processing speed by utilizing a range of threshold values for relaxed pruning.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Productivity

If compressed sparse column (CSC) format is used to process sparse matrices, then processing efficiency is improved, but compression rate decreases as matrix size increases

Engineering Contradiction:
Improveprocessing efficiencyVSAvoidcompression rate
Core Design Contradiction:
ProductivityVSQuantity of substance

Solution Approach 1:

The patent divides the matrix processing into multiple levels using a hierarchical structure. Large sparse matrices are segmented into smaller sub-matrices or blocks, which are then processed independently. This segmentation allows the CSC format to maintain effective compression rates by applying compression algorithms to smaller, more manageable units rather than attempting to compress the entire large matrix at once, thus resolving the contradiction between processing efficiency and compression rate.

Inventive Principle:
Principle #1Segmentation

2Quantity of substance

If traditional pruning methods are used to handle sparse elements, then memory usage is reduced, but processing speed decreases due to loss of information

Engineering Contradiction:
Improvememory usageVSAvoidprocessing speed
Core Design Contradiction:
Quantity of substanceVSSpeed

Solution Approach 1:

The patent transforms the representation parameters of sparse matrix elements by encoding them in a hierarchical format that preserves essential information while reducing memory footprint. Instead of traditional binary or fixed-precision representations, the invention uses variable-length encoding schemes that adapt to the sparsity pattern, maintaining processing speed by preserving critical numerical relationships while significantly reducing memory usage through optimized parameter encoding.

Inventive Principle:
Principle #35Parameter changes

3Reliability

If matrix size is increased to improve model accuracy, then neural network performance is improved, but transmission and processing costs increase significantly

Engineering Contradiction:
Improvemodel accuracyVSAvoidtransmission and processing costs
Core Design Contradiction:
ReliabilityVSLoss of energy

Solution Approach 1:

The patent introduces a hierarchical dimension to matrix representation, organizing sparse matrix data in a multi-level structure that adds a dimensional layer to traditional flat matrix storage. This hierarchical dimensionality allows large matrices to be decomposed into nested levels of sub-matrices, enabling efficient compression and selective processing that reduces transmission and processing costs while maintaining the accuracy benefits of larger matrix sizes through intelligent data organization and retrieval.

Inventive Principle:
Principle #17Another dimension (Dimensionality change)

Data Source

PatentUS12093341B2Method and apparatus for processing matrix data through relaxed pruning
Publication Date: 2024.09.17 SAMSUNG ELECTRONICS CO LTD
  • US12093341B2 patent drawing
  • US12093341B2 patent drawing
  • US12093341B2 patent drawing

AI summary

A matrix data processing method performed by a computing device which performs a matrix multiplication operation includes, with respect to each of one or more elements included in a matrix, when a value of each element satisfies a designated condition, determining the element to be a don't-care element and determining an output value of the don't-care element, generating a bitstream based on the output value of the don't-care element and index values of valid elements included in the matrix, and equally dividing the bitstream into pieces of a designated number, and generating a Huffman code corresponding to each of a plurality of lower bitstreams that are generated as a result of the equal division.