Distributed Database Query Optimization Using Historical Execution Data

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing database management systems (DBMS) face challenges in optimizing queries in distributed data management systems due to the complexity of query execution processes, which are not adequately addressed by current query optimizers that fail to consider application-specific attributes and dataset distribution.

Innovation Solution

A method that analyzes incoming queries, retrieves information from previous similar queries, and formulates an execution plan using historical data to optimize query execution, considering application-specific attributes and dataset distribution.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Adaptability or versatility

If a general purpose DBMS is used to handle diverse data schemas and formats, then versatility is improved, but query execution complexity increases

Engineering Contradiction:

Improveability to handle diverse data schemasVSAvoidquery execution process complexity

Core Design Contradiction:

Adaptability or versatilityVSDevice complexity

Solution Approach 1:

The query execution process is segmented into distinct phases: query analysis, historical information retrieval, execution plan formulation, and optimization. This segmentation allows each phase to be optimized independently while managing the overall complexity of handling diverse data schemas in a general-purpose DBMS.

Inventive Principle:

Principle #1Segmentation

2Device complexity

If query optimizers apply the same principles to all queries, then simplicity is maintained, but query performance deteriorates due to lack of application-specific optimization

Engineering Contradiction:

Improvequery optimizer simplicityVSAvoidquery execution performance

Core Design Contradiction:

Device complexityVSProductivity

Solution Approach 1:

Historical information about previously executed queries is retrieved and stored in advance. This preliminary action enables the system to leverage past performance data when formulating execution plans for new queries, improving performance without requiring complex real-time analysis of each query from scratch.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The system incorporates feedback from historical query executions by retrieving information about previously executed similar queries. This feedback mechanism allows the query optimizer to learn from past performance and adjust execution plans accordingly, improving query performance while maintaining a relatively simple optimization framework.

Inventive Principle:

Principle #23Feedback

3Productivity

If query execution is optimized for speed, then productivity is improved, but the system becomes less adaptable to different application-specific attributes and data distributions

Engineering Contradiction:

Improvequery execution speedVSAvoidapplication-specific optimization capability

Core Design Contradiction:

ProductivityVSAdaptability or versatility

Solution Approach 1:

The query execution plan formulation process is made dynamic by incorporating historical information about previously executed queries. This allows the system to adapt execution strategies based on actual performance data while maintaining the ability to handle diverse application-specific attributes and data distributions through learned patterns from history.

Inventive Principle:

Principle #15Dynamics

Data Source

PatentUS12353414B2Database query optimization based on analytics

Publication Date: 2025.07.08 INTERNATIONAL BUSINESS MACHINE CORPORATION

AI summary

A method, computer system, and a computer program product are provided for query management optimization in a distributed data management system. In one embodiment, at least one query is received. The query is then analyzed and related information associated with the query is obtained. When information exists in a database relating to previously executed queries similar to the received query, that information is obtained. A query execution plan is then formulated using any existing information and information relating to the similarly previously executed queries.