Parallel Query Optimizers for Database Execution Plans

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Conventional database systems fail to optimize SQL queries efficiently, particularly in large parallel database management systems, as they do not adequately reduce data read from storage drives, processing time, and communication between nodes, and lack integration with data modeling for optimization.

Innovation Solution

A large parallel database management system that optimizes SQL queries using multiple optimizers, including particle swarm optimization, Bloom filters for column cardinality estimation, kernel density estimation for column distribution, and integrated SQL queries with data modeling syntax to reduce data access and processing time, leveraging computer configurations and network throughput.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Productivity

If conventional single-optimizer approaches are used, then device complexity is reduced, but query optimization effectiveness deteriorates

Engineering Contradiction:
Improvequery optimization effectivenessVSAvoidoptimizer system complexity
Core Design Contradiction:
ProductivityVSDevice complexity

Solution Approach 1:

The patent divides the query optimization process into multiple independent optimizers, each specializing in different optimization techniques (cost-based optimization, rule-based optimization, heuristic optimization). Each optimizer processes queries independently and generates candidate execution plans, which are then evaluated to select the optimal plan. This segmentation allows each optimizer to focus on specific optimization strengths without being overwhelmed by the complexity of implementing all optimization strategies in a single system.

Inventive Principle:
Principle #1Segmentation

2Productivity

If multiple optimizers are deployed, then query optimization effectiveness is improved, but system resource consumption increases

Engineering Contradiction:
Improvequery execution efficiencyVSAvoidCPU and memory resources
Core Design Contradiction:
ProductivityVSUse of energy by moving object

Solution Approach 1:

The patent implements a resource-aware optimizer selection mechanism that dynamically determines which optimizers to activate based on available system resources and query characteristics. When resources are constrained, not all optimizers are executed simultaneously; instead, a subset is selected based on expected performance benefits versus resource costs. This partial action approach ensures that multiple optimizers provide their optimization benefits while preventing excessive resource consumption that would occur if all optimizers ran regardless of system state.

Inventive Principle:
Principle #16Partial or excessive action

3Measurement precision

If comprehensive data analysis is performed for optimization, then measurement precision is improved, but loss of time increases

Engineering Contradiction:
Improvecolumn cardinality estimation accuracyVSAvoidoptimization computation time
Core Design Contradiction:
Measurement precisionVSLoss of time

Solution Approach 1:

The patent implements a hybrid approach that combines pre-computed statistical information (such as column cardinality statistics, data distribution histograms) with on-demand detailed analysis. Frequently accessed tables and columns maintain pre-analyzed statistics that can be quickly retrieved during query optimization. When higher precision is needed or statistics are outdated, the system performs targeted re-analysis rather than comprehensive re-computation of all statistics. This preliminary action approach ensures accurate optimization decisions while minimizing the time spent on data analysis.

Inventive Principle:
Principle #10Preliminary action

Data Source

PatentUS11971890B2Database management system for optimizing queries via multiple optimizers
Publication Date: 2024.04.30 OCIENT INC
  • US11971890B2 patent drawing
  • US11971890B2 patent drawing
  • US11971890B2 patent drawing

AI summary

A large highly parallel database management system includes thousands of nodes storing huge volume of data. The database management system includes multiple query optimizers for determining low cost execution plans for queries. The database management system is adapted to receive a data query. An execution plan generator component of the database management system generates an initial execution plan for the query. The initial execution plan is fed as input to more than one query optimizers. Each optimizer starts with the initial execution plan, generates alternative execution plans, and determines a satisfactory execution plan that incurs the lowest cost. The database management system compares the selected execution plans by the optimizers and selects one with the lowest cost. The multiple query optimizers run in parallel.