Multi-Latency Logical Data Store for Consistent Low-Latency and Analytic Access

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Conventional data management systems face challenges in achieving targeted performance levels when handling both low-latency and analytic data, often resulting in complex data management, uneven load and query performance, and inconsistencies across separate data stores.

Innovation Solution

A multi-latency logical data store is introduced, comprising a controller that manages a low-latency data store and an analytic data store with different latencies, using multi-latency tables and data sets to provide coordinated load, query, and update operations, ensuring consistency and transforming data as needed across the two systems.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If two separate data stores are implemented for low-latency and analytic data, then each data store can be optimized for its particular type of data, but data management becomes increasingly complex

Engineering Contradiction:

Improvedata optimizationVSAvoiddata management complexity

Core Design Contradiction:

ReliabilityVSDevice complexity

Solution Approach 1:

The patent combines multiple separate data stores (low-latency and analytic) into a unified multi-latency logical data store. This single data store can present different latency characteristics to different applications simultaneously, eliminating the need for separate management systems while maintaining optimization for both data types.

Inventive Principle:

Principle #5Merging (Combining)

Solution Approach 2:

The multi-latency logical data store serves multiple functions: it acts as both a low-latency data store for real-time applications and an analytic data store for historical analysis, while also providing data consistency management and coordination between different latency requirements through a single unified system.

Inventive Principle:

Principle #6Universality (Multi-functionality)

2Reliability

If two separate data stores are implemented for low-latency and analytic data, then each data store can be optimized for its particular type of data, but load and query performance become uneven

Engineering Contradiction:

Improvedata optimizationVSAvoidload and query performance

Core Design Contradiction:

ReliabilityVSProductivity

Solution Approach 1:

The patent implements different latency characteristics within different portions or contexts of the same data store. The multi-latency logical data store can provide low-latency access for certain operations and higher-latency analytic access for others, with each portion optimized for its specific purpose while managed through a unified system.

Inventive Principle:

Principle #3Local quality

3Reliability

If two separate data stores are implemented for low-latency and analytic data, then each data store can be optimized for its particular type of data, but data consistency across both data stores becomes problematic

Engineering Contradiction:

Improvedata optimizationVSAvoiddata consistency

Core Design Contradiction:

ReliabilityVSStability of the object's composition

Solution Approach 1:

The patent implements a coordination mechanism that monitors and manages data consistency between low-latency and analytic portions of the multi-latency logical data store. The system provides feedback loops that ensure data is properly synchronized and consistent across different latency domains, maintaining stability while allowing optimization.

Inventive Principle:

Principle #23Feedback

4Device complexity

If a single data store is implemented to store both low-latency data and historical data, then data management is simplified, but the data store cannot be optimized for both types of data

Engineering Contradiction:

Improvedata management simplicityVSAvoiddata optimization

Core Design Contradiction:

Device complexityVSReliability

Solution Approach 1:

The patent creates a dynamic data store that can adapt its latency characteristics based on the type of data and operation. The multi-latency logical data store dynamically adjusts its behavior to provide low-latency access for real-time operations and higher-latency access for analytic operations, maintaining both optimization and simplicity.

Inventive Principle:

Principle #15Dynamics

Data Source

PatentUS9600358B1Scalable monitoring and error handling in multi-latency systems

Publication Date: 2017.03.21 EMC IP HLDG CO LLC

AI summary

Example embodiments of the present invention provide a method, an apparatus, and a computer program product for scalable monitoring and error handling in multi-latency systems. The method includes gathering events from a multi-latency logical data store comprising a first data store having a first data latency and a second related data store having a second data latency substantially different than the first data latency. Processing then may be performed on the gathered events, with notification of the processed events provided toward downstream queues for consumption. In certain embodiments, consumption comprises holistic error handling; according, in those embodiments holistic error handling of the multi-latency logical data store may be performed according to the notification of the processed gathered events asynchronously from gathering events from the multi-latency logical data store.