Time Series Serialization Using Schema-Guided Delta Compression
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data compression methods are inefficient for transmitting and decompressing live data, such as motion capture and visual effects, as they lack application-specific knowledge and structured formats, leading to suboptimal compression and transmission.
Innovation Solution
A method for compressing and serializing data using application-specific schemas that define data types, allowing for key and delta frames, and post-processing to achieve high compression efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If general purpose data-types and standard compression methods are used, then data can be transmitted using existing protocols, but compression efficiency is suboptimal and bandwidth usage is high
Solution Approach 1:
The patent applies parameter changes by converting floating-point values to integer representations and transforming data into delta-encoded sequences. This changes the numerical parameters of the data from high-precision floating-point format to compact integer formats with incremental differences, achieving up to 90% bandwidth reduction while maintaining acceptable reconstruction quality
Solution Approach 2:
The patent segments the data transmission into key frames and delta frames. Key frames contain absolute reference data, while delta frames contain only incremental differences from the previous key frame. This segmentation allows receivers to reconstruct data efficiently even when missing some frames, while dramatically reducing the amount of data that needs to be transmitted
2Productivity
If data is compressed using standard methods without application-specific knowledge, then implementation is simple, but compression ratios are insufficient for live data transmission
Solution Approach 1:
The patent applies preliminary action by pre-defining schemas that capture application-specific knowledge about data structures, relationships, and expected patterns. These schemas are established beforehand and used to guide the compression process, enabling the system to achieve high compression ratios by leveraging domain-specific insights without requiring complex runtime analysis
3Measurement precision
If floating point precision is maintained, then data accuracy is high, but data size and transmission bandwidth requirements increase significantly
Solution Approach 1:
The patent transforms the parameter representation from floating-point format to integer delta-encoded format. By changing how numerical values are represented and encoded, the system maintains sufficient accuracy for visual reconstruction while reducing data size by approximately 90%, as the incremental changes between frames require far fewer bits than full precision floating-point values
Data Source
AI summary
A method for compressing and serializing data, the method comprising: inputting raw data; the raw digital data is configured to be define a schema; the schema is a model of data to be transmitted; the raw data is defined as a value, an array, or a structure; the header code of the data is mapped to a special value; serializing; the raw data is converted; serialization tracks whether a key frame or a delta frame is written; special data values have only header code written; post-processing; the written data stream is block compressed; and de-serializing; a received bit stream is converted into a lossy data; the data type is read; the schema is followed; wherein the header code is read.