Semantic Field Compression for Faster Data Decompression
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data compression and decompression methods are inefficient when dealing with data sets containing semantically meaningful data fields, as they typically apply compression schemes to entire data values rather than exploiting value locality at the field level.
Innovation Solution
The approach involves dividing each data value into semantically meaningful fields and applying compression schemes to these fields in isolation, grouping fields with shared semantic meaning for improved compression and decompression efficiency, allowing for parallel processing and reduced metadata complexity.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If compression schemes are applied to entire data values, then the compression process is simple to implement, but compression efficiency is reduced when data sets contain semantically meaningful fields
Solution Approach 1:
The patent divides each data value into multiple semantically meaningful fields (e.g., timestamp, sensor ID, measurement value). This segmentation allows the compression system to identify and exploit value locality within specific fields, thereby improving compression efficiency without significantly complicating the overall implementation process.
Solution Approach 2:
The patent applies different compression strategies to different semantic fields based on their specific characteristics. For example, timestamp fields may use delta encoding while sensor ID fields use dictionary encoding. This localized approach optimizes compression efficiency for each field type while maintaining overall system simplicity.
2Productivity
If compression is applied to data sets with semantically meaningful fields, then compression efficiency improves, but the complexity of the compression and decompression processes increases
Solution Approach 1:
The patent performs preliminary actions by pre-defining semantic field structures and value locality patterns during system initialization. This allows the compression algorithm to operate more efficiently during actual compression tasks, as the framework for handling different field types is already established, thereby reducing runtime complexity.
Solution Approach 2:
The patent changes parameters by adapting compression strategies based on the specific characteristics of each semantic field. The system automatically selects appropriate compression parameters (e.g., dictionary size, delta encoding window) based on field type and data patterns, which optimizes compression efficiency while avoiding the need for complex manual configuration.
3Productivity
If field-level compression is implemented, then value locality is exploited for better compressibility, but the device complexity increases due to field separation and aggregation operations
Solution Approach 1:
The patent merges the field separation and aggregation operations into a unified compression framework. By integrating these operations into the core compression algorithm rather than treating them as separate preprocessing and postprocessing steps, the system achieves field-level compression benefits while minimizing the additional device complexity that would arise from multiple independent components.
Data Source
Figure 1~2
Figure 3~4
Figure 5
AI summary
Methods, devices and systems enhance compression and decompression of data values when they comprise a plurality of semantically meaningful data fields. According to a first inventive concept of the present invention disclosure, compression is not applied to each data value as a whole, but instead to at least one of the semantically meaningful data fields of each data value, and in isolation from the other ones. A second inventive concept organizes the data fields that share the same semantic meaning together to accelerate compression and decompression as multiple compressors and decompressors can be used in parallel. A third inventive concept is a system where methods and devices are tailored to perform compression and decompression of the semantically meaningful data fields of floating-point numbers after first partitioning further at least one of said data fields into two or a plurality of sub-fields to increase the degree of value locality and improve compressibility of floating-point values.