Metric Data Compression Using Dynamic Graph Dictionaries
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional metric compression systems are inflexible and inefficient due to their reliance on static compression dictionaries and rigid compression tools, which limit their ability to adapt to changing metric metadata and require significant bandwidth for dictionary updates.
Innovation Solution
The use of a graph-based compression dictionary that can be dynamically updated by adding or removing nodes and edges, combined with delta compression across metric values within a time slice or across a window of time, allows for flexible and efficient compression of metric data.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If static compression dictionaries are used, then compression structure is simple, but adaptability to changing metric metadata deteriorates
Solution Approach 1:
The patent transforms the static compression dictionary into a dynamic graph-based structure where nodes and edges can be added, removed, or modified in real-time. This allows the compression system to adapt to changing metric metadata while maintaining a relatively simple overall structure. The graph structure enables incremental updates without requiring complete dictionary replacement.
Solution Approach 2:
The patent changes the fundamental parameters of the compression dictionary by representing it as a graph data structure with configurable nodes, edges, and weights. This parameter transformation allows flexible modification of the dictionary's adaptive properties while keeping the core compression mechanism simple and efficient.
2Device complexity
If static compression dictionaries are used, then system structure is simple, but bandwidth consumption for updates increases
Solution Approach 1:
The patent extracts only the necessary portions of the compression dictionary for updating by identifying specific nodes and edges that need modification. Instead of transmitting entire dictionary updates, the system extracts and transmits only the changed elements, significantly reducing bandwidth consumption while maintaining simple system structure.
Solution Approach 2:
The patent segments the compression dictionary into independent nodes and edges that can be updated individually. This segmentation allows the system to update only specific portions of the dictionary rather than the entire structure, reducing update bandwidth requirements while keeping the overall system structure simple and manageable.
3Power
If conventional compression tools are used, then processing overhead is low, but compression efficiency deteriorates
Solution Approach 1:
The patent introduces a new dimension to compression efficiency by implementing compression at multiple levels: within time slices (horizontal compression) and across time slices (vertical compression). This multi-dimensional approach significantly improves compression efficiency without substantially increasing processing overhead, as the additional compression passes leverage existing data relationships.
Solution Approach 2:
The patent performs preliminary sorting of metric values within time slices before compression. This preliminary action organizes the data in a way that maximizes compression opportunities, enabling more efficient compression in subsequent steps without requiring complex real-time processing during the main compression operation.
4Ease of operation
If metric values are compressed only across time, then processing simplicity is maintained, but compression flexibility deteriorates
Solution Approach 1:
The patent segments the compression process into distinct phases: within-time-slice compression and across-time-slice compression. This segmentation maintains processing simplicity by breaking down the complex compression task into manageable, independent steps while simultaneously improving compression flexibility by allowing different compression strategies to be applied at each level.
Solution Approach 2:
The patent implements continuous compression by maintaining compression state across time slices. The compression process continues seamlessly from one time slice to the next, utilizing relationships between consecutive slices to achieve better compression ratios while keeping the overall processing approach simple and systematic.
Data Source
AI summary
A computing system configured to (i) obtain a set of key-value pairs, wherein each key-value pair corresponds to a respective timestamp in a period of time, (ii) for at least one timestamp in the given period of time: (a) identify a first subset of the key-value pairs corresponding to the timestamp, (b) sort the first subset, and (c) generate a subset of compression values for the sorted first subset, (iii) for at least one key: (a) identify a second subset of the key-value pairs corresponding to the key, (b) sort the second subset, and (c) generate a subset of compression values for the sorted second subset, and (iv) store a set of compression values comprising (a) the subset of compression values that is generated for each of the at least one timestamp and (b) the subset of compression values that is generated for each of the at least one key.