Semantic-Field Data Compression for Parallel Low-Latency Decoding
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data compression and decompression methods are inefficient when dealing with semantically meaningful data fields, particularly in computer systems and communication networks, as they typically compress data at the value level rather than exploiting value locality at the field level, leading to increased latency and reduced compression efficiency.
Innovation Solution
The approach involves dividing data values into semantically meaningful fields, compressing each field independently, and grouping fields with shared semantic meaning for parallel processing, using techniques like Huffman encoding to enhance compression and decompression efficiency while minimizing metadata overhead.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If data compression is applied at the value level using conventional methods, then the compression process is simple to implement, but the compression efficiency is reduced and latency increases
Solution Approach 1:
The patent segments data values into multiple semantically meaningful fields (e.g., integer fields, floating-point fields, boolean fields, string fields) and applies compression independently to each field type. This segmentation enables specialized compression algorithms to be applied to each field, improving overall compression efficiency while maintaining implementation simplicity through modular processing
2Ease of operation
If data compression is applied at the value level using conventional methods, then the processing approach is uniform and simple, but the latency is increased
Solution Approach 1:
By segmenting data into distinct field types and processing each field type separately with optimized algorithms, the patent reduces the time required for compression. Each field can be compressed in parallel, and the segmented approach allows for more efficient processing compared to uniform value-level compression
Solution Approach 2:
The patent applies different compression algorithms and strategies tailored to each specific field type (e.g., delta encoding for integer fields, specialized handling for floating-point fields). This local optimization reduces latency by using the most efficient compression method for each field rather than applying a generic approach to all data
3Quantity of substance
If data compression is applied at the value level, then metadata overhead is minimized, but compression efficiency is reduced
Solution Approach 1:
The segmentation of data into field types enables the system to track and manage metadata efficiently at the field level rather than at the individual value level. This reduces overall metadata overhead while allowing specialized compression to improve efficiency
Solution Approach 2:
The patent merges multiple values of the same field type into contiguous memory regions and applies compression across these grouped values. This merging approach reduces metadata overhead by handling groups of values together while maintaining the ability to apply efficient field-specific compression algorithms
Data Source
Figure 1~2
Figure 3~4
Figure 5
AI summary
Methods, devices and systems enhance compression and decompression of data values when they comprise a plurality of semantically meaningful data fields. According to a first inventive concept of the present invention disclosure, compression is not applied to each data value as a whole, but instead to at least one of the semantically meaningful data fields of each data value, and in isolation from the other ones. A second inventive concept organizes the data fields that share the same semantic meaning together to accelerate compression and decompression as multiple compressors and decompressors can be used in parallel. A third inventive concept is a system where methods and devices are tailored to perform compression and decompression of the semantically meaningful data fields of floating-point numbers after first partitioning further at least one of said data fields into two or a plurality of sub-fields to increase the degree of value locality and improve compressibility of floating-point values.