Concurrent Binary-Tree Indexing for Real-Time Time-Series Analysis
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Processing large volumes of time series data, such as terabytes or petabytes, can be time-consuming, requiring extensive processing and indexing before analysis can begin, which is impractical for real-time or diagnostic applications.
Innovation Solution
A method for building a binary tree structured index and summary data concurrently with data acquisition and storage, where each binary tree leaf and node is read-only once generated, allowing real-time indexing and analysis without interrupting data reception or storage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of time
If traditional indexing methods are used to process large volumes of time series data, then data can be organized and stored, but extensive processing time is required before analysis can begin
Solution Approach 1:
The patent applies preliminary action by pre-calculating and storing aggregate measurements at multiple hierarchical levels during data ingestion. Summary data structures (such as segment trees or wavelet transforms) are built concurrently with data collection, so that when analysis is needed, pre-computed aggregates are immediately available without requiring full data processing
Solution Approach 2:
The patent segments the time series data into hierarchical levels of aggregation. Data is divided into segments at different temporal resolutions (e.g., individual data points, 1-minute aggregates, 1-hour aggregates, daily aggregates), allowing queries to operate on appropriate granularities. This segmentation enables fast access to summary information while preserving detailed data when needed
2Productivity
If data is stored to storage device without interruption, then continuous data acquisition is maintained, but indexing and analysis must wait until processing can be performed
Solution Approach 1:
The patent maintains continuity of useful action by performing indexing and aggregate calculation operations concurrently with data acquisition and storage. The system continuously ingests data, continuously updates summary structures, and continuously makes data available for analysis without interruption or batch processing pauses
Solution Approach 2:
Indexing operations are performed in advance as data arrives, rather than waiting for data collection to complete. Summary statistics and index structures are built incrementally and continuously, so analysis capabilities are available immediately without post-collection processing delays
3Ease of operation
If conventional indexing structures are updated in real-time, then live data can be indexed, but updating previously generated nodes increases system complexity
Solution Approach 1:
The patent segments the indexing problem into independent hierarchical levels that can be updated separately. Each level of the summary data structure represents a distinct temporal aggregation level that can be computed and updated independently, avoiding the need to reprocess or restructure the entire index when new data arrives
Solution Approach 2:
The patent uses copying by creating summary aggregate copies of the data at different hierarchical levels. Instead of updating the original detailed data structure, the system maintains replicated summary versions that can be updated independently and efficiently, reducing the complexity of real-time index maintenance
Data Source
Figure 1
Figure 2
Figure 3A~3I
AI summary
A method is described which includes receiving or obtaining a time series of data (S1). The method also includes storing the time series of data to a storage device without interrupting the reception of the time series of data (S2). The method also includes, for each of a plurality of base time periods, at the end of a most recently elapsed base time period (S5) and without interrupting the reception or storage of the time series of data, calculating (S6) one or more measurements based on the time series of data corresponding to the most recently elapsed base time period and updating a binary tree structure indexing the one or more measurements and the time series of data. Updating the binary tree structure includes generating a new binary tree leaf (1, 2, 4, 5) corresponding to the most recently elapsed base time period (S7). Updating the binary tree structure also includes, in response to one or more new binary tree nodes (3, 6, 7) could be added to complete a perfect binary sub-tree including the new binary tree leaf 15 (S9), generating (S10) the new binary tree nodes (3, 6, 7). The new binary tree leaf (1, 2, 4, 5) and any new binary tree nodes (3, 6, 7) may be disjoint from one or more existing portions of the binary tree structure. Leaves (1, 2, 4, 5) and nodes (3, 6, 7) of the binary tree structure are read-only once generated. Each binary tree leaf (1, 2, 4, 5) includes one or more pointers to measurements based on the corresponding base time period and each binary tree node (3, 6, 7) comprises one or more pointers to aggregated measurements based on all descendant base time periods.