Distributed Query Cost Updating for Accurate Execution Plans
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In distributed databases, determining an appropriate execution plan from multiple options is challenging due to varying performance states of data nodes and network costs, leading to inaccurate cost estimation and optimization.
Innovation Solution
A data query optimization method that updates execution costs of candidate plans using monitoring data from data nodes, including processor usage rate, network status, and task planning, to accurately determine a final execution plan.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional cost estimation methods are used for execution plans, then the estimation process is simple and fast, but the accuracy of cost estimation is low due to not considering actual data node conditions
Solution Approach 1:
The patent implements feedback by collecting monitoring data from data nodes (CPU usage, memory usage, I/O status, network status) and using this real-time information to adjust and update the cost estimation of execution plans. The cost estimation model continuously incorporates feedback from actual system conditions to improve accuracy.
Solution Approach 2:
The patent changes the parameters used in cost estimation from static historical data to dynamic monitoring data including CPU usage rate, memory usage rate, I/O status, and network status. These parameter changes allow the cost estimation to reflect actual system conditions accurately.
2Reliability
If multiple candidate execution plans are evaluated with detailed monitoring data, then the accuracy of final execution plan selection is improved, but the computational overhead and time consumption increase
Solution Approach 1:
The patent applies partial action by selecting only the top K candidate execution plans for detailed evaluation instead of evaluating all possible plans. This reduces computational overhead while still ensuring the optimal plan is found among the most promising candidates.
Solution Approach 2:
The patent performs preliminary filtering of execution plans before detailed cost estimation by using basic criteria to eliminate obviously suboptimal plans. This preliminary action reduces the number of plans requiring comprehensive monitoring data analysis.
Data Source
AI summary
Provided is a data query optimization method, an electronic device and a storage medium, relating to the field of data processing technology and in particular to the technical fields of distributed database, big data, cloud computing and others. The method includes: determining a plurality of candidate execution plans for a target query request; determining execution costs of the plurality of candidate execution plans; updating the execution costs of the plurality of candidate execution plans based on monitoring data of data nodes involved in the plurality of candidate execution plans, to obtain final costs of the plurality of candidate execution plans; and screening out a final execution plan for the target query request from the plurality of candidate execution plans based on the final costs of the plurality of candidate execution plans.