Global Base-Delta Memory Compression for Shared Value Encoding
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data compression techniques in computer memories face challenges in selecting shared base values across data blocks to minimize metadata and effectively manage compression and decompression processes, particularly when data values are not numerically similar, leading to reduced compression effectiveness and high error rates, especially for floating-point values.
Innovation Solution
A method and device that determine a set of global base values common to multiple data blocks to minimize delta values, using base-delta encoding with entropy-based and deduplication techniques to encode individual data values and delta values compactly, while maintaining high compression ratios and low error rates.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If data blocks are compressed using traditional base-delta encoding with locally selected base values, then compression can be implemented with simple hardware, but compression effectiveness is reduced when data values are not numerically similar and metadata size increases
Solution Approach 1:
The patent divides the data compression process into two distinct stages: a compression stage that selects optimal global base values across multiple data blocks to maximize compression effectiveness, and a decompression stage that uses pre-stored base value tables for fast reconstruction. This segmentation allows the system to achieve high compression ratios without requiring complex real-time computation during decompression, thus resolving the contradiction between hardware simplicity and compression effectiveness.
Solution Approach 2:
The patent performs base value selection and base value table construction in advance during the compression stage, before actual data compression occurs. By pre-computing optimal base values and storing them in lookup tables, the system eliminates the need for complex real-time calculations during decompression, maintaining simple hardware requirements while achieving superior compression effectiveness through globally optimized base value selection.
2Productivity
If more memory capacity is allocated to store uncompressed data, then system performance can be improved, but physical memory size and cost increase
Solution Approach 1:
The patent merges multiple data blocks into a unified compression structure by selecting global base values that are common across multiple blocks. This consolidation allows the system to represent data from multiple blocks more compactly by sharing common base values, thereby reducing the total memory capacity required while maintaining the ability to retrieve and process the same amount of information, thus improving productivity without increasing physical memory.
Solution Approach 2:
The patent changes the representation parameters of data by transforming absolute data values into relative delta values with respect to selected base values. This parameter transformation enables more efficient encoding where frequently occurring patterns are represented compactly, reducing the memory capacity needed to store the same information and thereby improving system performance without requiring additional physical memory resources.
3Quantity of substance
If lossy compression is applied to reduce data size, then memory capacity can be reduced, but error rates increase especially for floating-point values
Solution Approach 1:
The patent incorporates a feedback mechanism where the compression ratio and error characteristics are analyzed to dynamically adjust compression parameters. By monitoring the relationship between compression level and reconstruction accuracy, the system can optimize the balance between memory capacity reduction and error rate control, ensuring that compression is applied at levels that maintain acceptable precision for the specific data being compressed.
Data Source
AI summary
A data compression method is disclosed which involves obtaining a plurality of data blocks, each data block comprising a plurality of data values. The method performs base-delta encoding of the obtained plurality of data blocks, wherein a delta value means a difference between a data value and a base value. This involves determining, among the data values of the plurality of data blocks, a set of global base values common to said plurality of data blocks. The set of global base values is selected to minimize delta values for the data values of the plurality of data blocks with respect to the global base values in said set of global base values. The method further involve encoding individual data values of the plurality of data blocks by selecting, in the set of global base values, for each individual data value a global base value that is numerically closest to the individual data value and thus results in a smallest delta value, and then generating metadata for the encoded individual data value to represent the selected global base value and the resulting delta value.