Compressed Time-Series Querying with Segment Range Indexing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing database systems face inefficiencies in processing value-based queries on compressed time-series data, particularly with piecewise-regression-based compression, which limits the usage of such compression techniques due to high resource consumption when answering queries like 'select all time-series exceeding a threshold'.
Innovation Solution
The implementation of an index table and segments table structure within the database, where the index table specifies ranges of compressed time-series data, allowing for efficient pre-selection and decompression of relevant segments based on query values, along with the use of decompression functions and parameters, enables efficient processing of value-based queries without significantly reducing compression ratios.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If piecewise-regression-based compression is used to store time-series data, then compression ratio is improved, but query processing efficiency deteriorates due to high resource consumption
Solution Approach 1:
The patent pre-computes and stores minimum and maximum values for segments of compressed time-series data in an index table before queries are executed. This preliminary action enables fast query filtering without decompressing entire segments, resolving the contradiction by preparing data in advance to speed up later query operations while maintaining compression.
Solution Approach 2:
The patent introduces an index table as an intermediary structure between the compressed time-series data and query operations. This index table contains pre-computed min/max values that mediate between the compressed data format and query requirements, enabling efficient query processing without full decompression while preserving compression ratios.
2Volume of stationary object
If compressed time-series data is stored in database, then storage cost is reduced, but query complexity increases due to decompression requirements
Solution Approach 1:
The patent divides compressed time-series data into segments and creates an index table that organizes these segments with their respective min/max values. This segmentation allows queries to operate on specific segments rather than entire datasets, reducing query complexity while maintaining storage efficiency through compression.
Solution Approach 2:
The patent pre-computes minimum and maximum values for each segment during data insertion and stores them in an index table. This preliminary computation simplifies subsequent query operations by enabling fast filtering without decompression, reducing query processing complexity while preserving compressed storage benefits.
3Measurement precision
If all segments are decompressed to answer queries, then query accuracy is improved, but resource consumption increases
Solution Approach 1:
The patent applies partial decompression by first using the index table to identify candidate segments that may contain query results, then decompressing only those specific segments rather than all segments. This partial action maintains query accuracy for relevant data while significantly reducing resource consumption compared to full decompression.
Solution Approach 2:
The patent applies different processing qualities to different data segments: segments identified as potential matches through index table filtering are decompressed with full accuracy, while other segments are skipped entirely. This local differentiation of processing quality ensures accuracy where needed while conserving resources elsewhere.
Data Source
AI summary
A query of time series data stored in a database is received that specifies at least one value. The database includes (i) an index table specifying groups of segments of compressed time series data with corresponding ranges each having a lowest value and a highest value, and (ii) a segments table specifying individual segments of compressed time series data. Thereafter, using the index table, at least one group for which the specified at least one value falls within the corresponding range is identified. The segments table is then queried for the segments corresponding to the identified at least one group to generate a new segments table specifying at least one segment. Next, the at least one segment specified by the new segments table is decompressed. Data responsive to the query within the decompressed at least one segment is then identified using the specified at least one value.