Middleware Server Caching Strategy for Database Query Optimization

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Large-scale and distributed databases face challenges in efficiently managing data validity and reducing unnecessary network traffic while ensuring data credibility and compliance with varying application needs.

Innovation Solution

A middleware server system that determines data validity periods dynamically based on application-specific requirements, using staging tables to cache query results and invoking external databases only when necessary, thereby reducing HTTP traffic and costs.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If query requests are continuously sent to the external database to ensure data freshness, then data credibility is improved, but network traffic and costs increase

Engineering Contradiction:

Improvedata credibilityVSAvoidnetwork traffic

Core Design Contradiction:

ReliabilityVSLoss of energy

Solution Approach 1:

The system performs preliminary actions by determining the service validity period in advance and proactively checking whether the current time exceeds the validity period before external queries are needed. This allows the system to prepare and manage data validity proactively, avoiding unnecessary external queries when data is still valid, thereby reducing network traffic while maintaining data credibility.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The system dynamically adjusts its query strategy based on the service validity period. Instead of using a fixed query interval, the system adapts its behavior to the specific validity requirements of each data service, querying externally only when the validity period expires. This dynamic approach optimizes the balance between data freshness and network traffic reduction.

Inventive Principle:

Principle #15Dynamics

2Loss of energy

If data validity periods are extended to reduce query frequency, then network traffic is reduced, but data freshness deteriorates

Engineering Contradiction:

Improvenetwork trafficVSAvoiddata freshness

Core Design Contradiction:

Loss of energyVSReliability

Solution Approach 1:

The system changes the parameter of data validity by explicitly determining and tracking the service validity period for each external database query result. Instead of using a fixed or default validity period, the system dynamically sets and monitors the validity period parameter, ensuring that data is refreshed precisely when it expires. This parameter-based approach maintains data freshness while avoiding premature queries that would increase network traffic.

Inventive Principle:

Principle #35Parameter changes

3Device complexity

If uniform data validity periods are applied to all applications, then system complexity is reduced, but adaptability to different application needs worsens

Engineering Contradiction:

Improvesystem complexityVSAvoidapplication-specific adaptability

Core Design Contradiction:

Device complexityVSAdaptability or versatility

Solution Approach 1:

The system applies local quality by determining service validity periods specific to each application's needs and each external database service combination. Instead of using a single uniform validity period for all applications, the system customizes the validity period locally for each specific case, allowing different applications to have different data freshness requirements met appropriately. This resolves the contradiction by maintaining low system complexity through automated determination while achieving high adaptability through application-specific validity periods.

Inventive Principle:

Principle #3Local quality

Data Source

PatentUS12124451B2System and method for efficient integration with a primary database to reduce unnecessary network traffic

Publication Date: 2024.10.22 SAUDI ARABIAN OIL CO

AI summary

A computer-implemented method that includes: receiving, at a middleware server, an input stream of query requests each requesting an electronic information service provided by an external database of an external trusted party, wherein the query requests are issued by internal enterprise applications; determining, by the middleware server, a first service validity period for a query request from the input stream, wherein the first service validity period is pre-determined; determining, by the middleware server, a gap between a current date for the query request and a corresponding past date when the query request was last successfully answered by the external database; and in response to determining that the gap is smaller than the first service validity period, fetching a corresponding query result from a staging table on the middleware server, wherein the corresponding query result was received when the query request was last successfully answered by the external database.