Centralized Cache Query Processing for Distributed Database Latency

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Distributed databases face challenges in reducing remote data accesses, leading to latency and increased load on interconnected nodes, particularly due to the time-consuming and costly process of maintaining a centralized data cache synchronized with underlying data sources.

Innovation Solution

The method involves parsing distributed database queries into component queries, matching them against a central cache to generate cache and local delta queries, and combining these with remote results to create a global delta, thereby reducing the need for remote data accesses and improving throughput.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Speed

If a centralized data cache is maintained to satisfy queries, then query response efficiency is improved, but the cost and time of synchronizing the cache with underlying data sources increases

Engineering Contradiction:

Improvequery response efficiencyVSAvoidcache synchronization time

Core Design Contradiction:

SpeedVSLoss of time

Solution Approach 1:

The system performs preliminary actions by maintaining a centralized data cache that is proactively synchronized with underlying data sources before queries are executed. This advance preparation allows the cache to contain up-to-date information, enabling fast query responses without performing synchronization at the moment of query execution.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The query processing is segmented into two parts: queries that can be satisfied from the centralized cache and queries that require remote data access. By dividing the workload this way, the system can handle cacheable queries efficiently while minimizing expensive remote accesses, thus resolving the contradiction between cache maintenance cost and query response efficiency.

Inventive Principle:

Principle #1Segmentation

2Measurement precision

If remote data accesses are performed to satisfy queries, then data accuracy is improved, but latency and load on interconnected nodes increase

Engineering Contradiction:

Improvedata accuracyVSAvoidquery latency

Core Design Contradiction:

Measurement precisionVSLoss of time

Solution Approach 1:

The centralized cache acts as an intermediary between the query system and the underlying data sources. It intercepts queries and satisfies them from cached data when possible, avoiding direct remote accesses. This intermediary layer reduces query latency and load on interconnected nodes while maintaining data accuracy through periodic synchronization.

Inventive Principle:

Principle #24Intermediary (Mediator)

Solution Approach 2:

The system performs partial remote data accesses only when necessary - specifically for queries that cannot be satisfied from the cache or when cache data is stale. By performing minimal remote accesses only when needed rather than for every query, the system reduces latency and network load while still ensuring data accuracy when cache data is insufficient.

Inventive Principle:

Principle #16Partial or excessive action

3Stability of the object's composition

If centralized cache synchronization is performed frequently, then data consistency is improved, but system cost and processing time increase

Engineering Contradiction:

Improvedata consistencyVSAvoidsystem throughput

Core Design Contradiction:

Stability of the object's compositionVSProductivity

Solution Approach 1:

Instead of continuous or overly frequent synchronization, the system performs cache synchronization periodically or on an as-needed basis. This periodic action maintains data consistency between the cache and underlying sources while avoiding the excessive processing overhead that would occur with continuous synchronization, thus preserving system throughput.

Inventive Principle:

Principle #19Periodic action

4Loss of energy

If remote data accesses are reduced, then network load is decreased, but cache maintenance complexity increases

Engineering Contradiction:

Improvenetwork loadVSAvoidcache maintenance complexity

Core Design Contradiction:

Loss of energyVSDevice complexity

Solution Approach 1:

The centralized cache serves multiple functions: it acts as a query response mechanism, a data synchronization point, and a load reducer for remote accesses. By making the cache multi-functional, the system reduces network load without proportionally increasing complexity, as the same infrastructure handles multiple objectives.

Inventive Principle:

Principle #6Universality (Multi-functionality)

Data Source

PatentUS11704327B2Querying distributed databases

Publication Date: 2023.07.18 INTERNATIONAL BUSINESS MACHINE CORPORATION

AI summary

Embodiments are disclosed for a method. The method includes parsing a distributed database query into component queries corresponding to distributed nodes. The method also includes matching component queries to central cache to determine whether the component queries can be satisfied. This matching generates matched and unmatched component queries. The method further includes transforming the matched component query to a cache query and local delta query for a first node. Additionally, the method includes executing the cache query to generate a cache result. Also, the method includes providing the local delta query for the first node to generate a local delta. Further, the method includes providing the unmatched component query for a second node to generate a remote result. The method also includes generating a global delta based on the local delta. The method additionally includes generating a result by combining the cache result, local delta, and remote result.