Index Update Handling in Time Series Data Monitoring

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Conventional techniques, such as relational databases, struggle with managing large quantities of time-sensitive data and face scalability issues, leading to delays in index updates and inefficient query processing in time series data monitoring systems.

Innovation Solution

A proactive method is implemented where index updates are communicated from ingestion nodes to query nodes in a time series data monitoring system, ensuring that the local index structures at query nodes are updated concurrently with the index database, maintaining a current state of the data points and improving query processing performance.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Productivity

If conventional relational databases are used to manage time series data, then data management is simple and familiar, but scalability is limited and query processing is inefficient for large quantities of time-sensitive data

Engineering Contradiction:

Improvequery processing speedVSAvoidsystem architecture complexity

Core Design Contradiction:

ProductivityVSDevice complexity

Solution Approach 1:

The system is divided into multiple query nodes, each maintaining its own local index structure. This segmentation allows parallel query processing across multiple nodes, improving overall query processing speed while distributing the computational load to avoid single-point bottlenecks

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

Index structures are pre-computed and maintained at each query node before actual queries are executed. When new time series data arrives, the ingestion node proactively computes index updates and pushes them to query nodes in advance, ensuring query nodes always have current index structures ready for immediate query processing

Inventive Principle:

Principle #10Preliminary action

2Reliability

If index updates are performed using conventional techniques, then data consistency is maintained, but index update delays occur and real-time requirements are not met

Engineering Contradiction:

Improvedata consistencyVSAvoidindex update delay

Core Design Contradiction:

ReliabilityVSLoss of time

Solution Approach 1:

The ingestion node proactively computes index updates as soon as new time series data arrives, before any queries are executed. These pre-computed index updates are immediately pushed to all query nodes, ensuring that index structures are updated in advance and query nodes always have current data without waiting for periodic refreshes

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The system implements a push-based feedback mechanism where the ingestion node monitors incoming time series data, computes the necessary index updates, and automatically pushes these updates to query nodes. This closed-loop feedback ensures data consistency is maintained across all nodes in real-time without requiring query nodes to periodically check for updates

Inventive Principle:

Principle #23Feedback

Data Source

PatentUS11874825B2Handling of an index update of time series data

Publication Date: 2024.01.16 VMWARE INC

AI summary

In a computer-implemented method for handling of an index update, time series data is received at an ingestion node of a time series data monitoring system. An index update is determined based on the time series data. The index update is stored to an index database of the time series data monitoring system. The index update is forward to a plurality of query nodes of the time series data monitoring system.