Compact Prefix Tree for Time Series Data Storage
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current data structures for time series data are not tailored to its characteristics, leading to inefficient storage and retrieval, particularly for sporadically changing data, which consumes large memory resources and processing power.
Innovation Solution
Storing time series data in a compact prefix tree data structure, where new data is only added when it differs from previously stored data, and using timestamps as keys to facilitate efficient querying and storage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If time series data is stored using general purpose data structures, then the data can be stored and retrieved, but memory resources are consumed inefficiently and retrieval speed is reduced
Solution Approach 1:
The patent segments time series data into distinct components: metric identifiers, time stamps, and data values. By organizing these segments in a hierarchical prefix tree structure where common prefixes (metric names and time portions) are shared across multiple data points, the system achieves both compact storage and efficient retrieval paths for different query types.
Solution Approach 2:
The patent merges common prefixes of time series data into shared nodes in the prefix tree structure. When multiple time series data points share common metric names or time stamp prefixes, these are consolidated into single nodes, eliminating redundancy and reducing overall memory consumption while maintaining fast access through the merged structure.
2Reliability
If all time series data points are stored individually, then complete data is preserved, but memory resources are wasted on duplicating unchanged data
Solution Approach 1:
The patent extracts and stores only the varying components of time series data in the prefix tree structure. By identifying and storing only the differences between consecutive data points (the non-repeating portions), the system maintains complete information about data changes while eliminating storage of redundant unchanged values.
Solution Approach 2:
The patent uses reference copying instead of duplicating full data structures. When time series data points are identical or unchanged, the system creates references to existing nodes in the prefix tree rather than creating new duplicate entries, thereby preserving data completeness through reference integrity while minimizing memory consumption.
3Productivity
If new time series data is added continuously without checking for duplicates, then data is captured in real-time, but processing power is wasted on redundant operations
Solution Approach 1:
The patent performs preliminary checks during the data insertion process by comparing incoming time series data against existing entries in the prefix tree before committing storage operations. This preliminary comparison action identifies duplicate or unchanged data points early, allowing the system to skip redundant insertion operations and conserve processing power while maintaining real-time data ingestion capability.
Data Source
AI summary
A device stores time series data, based on time stamps, in a compact prefix tree, and receives new time series data to be added to the compact prefix tree. The device determines whether the new time series data is different than previously stored time series data in the compact prefix tree. The device selectively stores the new time series data in the compact prefix tree by storing the new time series data in the compact prefix tree when the new time series data is different than the previously stored time series data in the compact prefix tree, and updates a last time stamp for one of the previously stored time series data, based on the new time series data, when the new time series data is not different than the one of the previously stored time series data.