Query Cancellation via Machine-Learned Cost Prediction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Long-running queries consume significant computing resources, leading to performance degradation and potential outages in computing systems that handle high volumes of data.
Innovation Solution
A computing system employs a machine-learned model to generate a query cost for incoming queries, which indicates the estimated performance impact on the system. Based on this query cost, the system can proactively cancel queries to prevent resource exhaustion and performance issues.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If the computing system executes all incoming queries, then query processing completeness is improved, but system performance and stability deteriorate due to resource exhaustion from long-running queries
Solution Approach 1:
The system performs preliminary analysis of incoming queries using a machine-learned model to predict their resource consumption and performance impact before execution. This allows the system to proactively identify and cancel queries that would cause performance degradation, preventing resource exhaustion before it occurs.
Solution Approach 2:
The system continuously monitors actual query execution performance and uses this feedback to retrain and improve the machine-learned model's predictions. By comparing predicted versus actual resource consumption and performance impact, the system refines its ability to accurately identify problematic queries, improving both reliability and query processing effectiveness.
2Productivity
If the system cancels queries based on predicted cost, then system performance is improved, but unnecessary cancellation of valid queries may occur
Solution Approach 1:
The system implements a feedback mechanism where actual query execution results are used to retrain and refine the machine-learned model. By continuously learning from real-world performance data, the model improves its prediction accuracy over time, reducing false positives and ensuring that only genuinely problematic queries are cancelled.
Solution Approach 2:
The system dynamically adjusts the cancellation threshold and model parameters based on observed system conditions and performance patterns. This allows the system to adapt its cancellation behavior to different workloads and conditions, optimizing the balance between preventing performance degradation and maintaining query processing completeness.
Data Source
AI summary
Systems and methods for query cancellation. The system can receive a query, the query may be associated with one or more requests executable by a computing system. The method includes generating, using a machine-learned model, a query cost for the query, wherein the query cost is indicative of an estimated performance of the computing system. The method includes, based on the query cost, cancelling the query before completing execution of the one or more requests by the computing system.