Time Series Data Aggregation via Compressed Block Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Aggregation of large time series data sets is computationally expensive and often performed offline, leading to inefficiencies as users may not retrieve all pre-computed results and face delays in obtaining results for non-matching queries until the next offline process is completed.
Innovation Solution
A system and process for efficiently aggregating time series data by generating compressed blocks from recorded data points, storing them in documents, and decompressing only the necessary blocks for real-time query results, reducing computational load and improving response times.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Use of energy by moving object
If aggregation is performed offline in advance, then computation is reduced for future queries, but results become stale and users cannot get results for non-pre-selected queries until the next offline process
Solution Approach 1:
The patent divides the time series data into fixed-size windows and pre-computes aggregations only for these segmented time ranges. When a query arrives, the system identifies which pre-computed windows are needed and retrieves only those, rather than re-computing everything or waiting for full offline processing. This segmentation allows partial results to be returned quickly while minimizing computation.
Solution Approach 2:
The system performs preliminary aggregation computations for segmented time windows in advance and stores these results. When queries arrive, the pre-computed results are already available for immediate retrieval, eliminating the need to wait for offline processing to complete while avoiding full re-computation of all data.
2Speed
If all pre-computed results are stored, then query retrieval is fast, but storage space is wasted when users only retrieve a fraction of results
Solution Approach 1:
By segmenting data into fixed-size time windows and only pre-computing aggregations for these segments, the system stores a manageable amount of data. Users can quickly retrieve results for their specific time ranges by accessing only the relevant segmented windows, avoiding the need to store and retrieve all possible aggregation results.
Solution Approach 2:
The system optimizes storage by maintaining pre-computed aggregation results only for specific segmented time windows that are likely to be queried, rather than uniformly storing all possible aggregations. This local optimization of data retention balances storage efficiency with query performance.
3Reliability
If real-time aggregation is performed, then fresh results are provided, but computational cost becomes prohibitive for large data sets
Solution Approach 1:
The patent segments time series data into fixed-size windows and pre-computes aggregations for each segment. When a real-time query arrives, the system identifies which pre-computed segments are needed and retrieves them immediately, providing fresh results without performing full real-time aggregation on the entire data set. This dramatically reduces computational cost while maintaining result freshness.
Solution Approach 2:
Instead of performing complete real-time aggregation on all data or waiting for full offline processing, the system performs partial aggregation by retrieving only the specific pre-computed segments needed for the current query. This partial action provides sufficiently fresh results with minimal computation.
Data Source
AI summary
Efficient aggregation of time series data is disclosed, including: obtaining a first entry value corresponding to an item, wherein the first entry value comprises a first recorded data point that is associated with a first time interval; generating a compressed block based at least in part on compressing the first entry value with at least a second entry value; storing the compressed block in a document corresponding to the item; determining that the item matches an aggregation search query; decompressing the compressed block from the document corresponding to the item to obtain the first entry value and the second entry value; and generating an aggregation result in response to the aggregation search query based on at least a portion of the first entry value and the second entry value.