Time Series Data Querying via Relational Tags
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Managing and synthesizing vast amounts of time series data collected from IT networks becomes increasingly difficult due to its sheer volume, making it challenging to identify relevant subsets for analysis and problem-solving.
Innovation Solution
An IT system utilizes a relational database and a time series database in conjunction, where a relational database management system identifies data subsets based on relational aspects and provides relational identification tags, which are then used by a time series database management system to retrieve corresponding time series data within a specified time range, allowing for efficient filtering and analysis.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If data is collected over time at increasing frequency and volume, then the amount of available data increases, but the difficulty of managing and synthesizing the data increases
Solution Approach 1:
The patent segments the monolithic time series database into multiple sharded databases distributed across different servers. Each shard stores a subset of the time series data, allowing the system to handle large volumes of data while reducing the complexity of managing and querying the entire dataset. The segmentation is achieved through hash-based sharding where data is distributed across multiple nodes based on hash values of device identifiers and time ranges.
Solution Approach 2:
The patent introduces a new dimensional organization of data by implementing a hybrid storage architecture that combines traditional time-series storage with event-based storage. Events are stored as discrete entities with associated time series data, allowing queries to traverse multiple dimensions (time, device, event type) simultaneously. This dimensional approach enables efficient synthesis of data from multiple sources without increasing management complexity.
2Loss of information
If all collected data is stored for analysis, then complete data availability is achieved, but the time and resources required to identify relevant subsets increase
Solution Approach 1:
The patent implements preliminary action by pre-processing and indexing time series data during ingestion, organizing it into event-based structures with associated metadata before queries are executed. Events are tagged with relevant attributes and organized in a way that enables rapid retrieval. This preliminary organization allows the system to maintain complete data availability while reducing query execution time significantly, as the data is already structured for efficient access patterns.
Solution Approach 2:
The patent introduces an event-based intermediary layer between the raw time series data and query operations. Events serve as mediators that aggregate and contextualize time series data, allowing queries to access relevant information through event contexts rather than scanning raw data streams. This intermediary structure enables complete data availability to be maintained while dramatically reducing the time required to identify and retrieve relevant data subsets through event-based query routing.
3Device complexity
If a single database stores all time series data, then data storage is simplified, but query performance and scalability deteriorate
Solution Approach 1:
The patent segments the single database into multiple sharded databases distributed across different servers, with each shard storing a specific subset of time series data. This segmentation improves query performance by allowing queries to be routed to specific shards based on device identifiers and time ranges, avoiding full database scans. The system maintains simplified storage through automated sharding management and provides scalability by adding new shards as data grows.
Solution Approach 2:
The patent merges multiple sharded databases into a unified query interface that presents a single logical database to users. The query system automatically routes requests to appropriate shards and aggregates results, providing the simplicity of a single database while leveraging the performance benefits of distributed storage. This merging approach maintains ease of use while achieving high query performance and scalability through the underlying distributed architecture.
Data Source
AI summary
A server system may include one or more hardware processors that execute computer-readable instructions from a non-transitory memory that cause the processor to receive a time component associated with a query request and one or more relational identification tags associated with a subset of a plurality of data items stored in one or more databases. The processor may then query the one or more databases to identify one or more time series data items based on the time component and the relational identification tags. After querying the databases, the processor may transmit the one or more time series data items to a computing device configured to display the one or more time series data items.