Query Execution Statistics Sampling and Aggregation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Large organizations face challenges in efficiently tracking query execution statistics across multiple processes due to resource overhead and complexity, especially when dealing with numerous queries that execute briefly, making comprehensive monitoring resource-intensive and costly.
Innovation Solution
A system that captures query execution statistics using a sliding monitoring window and sampling intervals, identifying and logging statistics for queries executing during these intervals, and aggregating data for related queries based on similarity thresholds, reducing the need for continuous resource allocation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If comprehensive monitoring of all query executions is implemented, then measurement precision and reliability of execution statistics are improved, but resource consumption and system overhead increase significantly
Solution Approach 1:
The patent implements partial monitoring by selecting only certain query executions for detailed tracking based on predefined criteria such as query type, resource consumption thresholds, and execution frequency. This allows the system to obtain sufficient execution statistics without monitoring every single query, thereby reducing resource overhead while maintaining measurement precision for critical queries.
Solution Approach 2:
The system employs periodic sampling of query executions at intervals rather than continuous monitoring. By periodically capturing execution statistics and using sliding time windows to aggregate data, the system achieves reliable measurement of execution patterns while significantly reducing the continuous resource consumption associated with comprehensive monitoring.
2Measurement precision
If detailed tracking of each query execution is performed, then measurement precision is improved, but device complexity and processing overhead increase
Solution Approach 1:
The patent merges multiple query execution tracking operations into unified aggregation processes. By combining statistics from multiple queries into consolidated metrics and using shared data structures, the system reduces the complexity of managing individual query tracking while maintaining comprehensive measurement precision through the aggregated data.
Solution Approach 2:
The system introduces intermediary data structures such as sliding time windows and buffer mechanisms that mediate between individual query executions and the final statistics. These intermediaries simplify the complexity by providing standardized interfaces for data collection, processing, and aggregation, reducing the burden on the monitoring system while preserving measurement accuracy.
3Reliability
If continuous monitoring of all processes is implemented, then reliability of execution data is improved, but resource overhead and loss of time increase
Solution Approach 1:
The system applies partial monitoring by focusing resources on tracking only those query executions that meet specific criteria for reliability, such as long-running queries, resource-intensive operations, or frequently executed queries. This selective approach maintains reliable execution data for critical operations while reducing the time overhead associated with monitoring all processes continuously.
Solution Approach 2:
The patent implements self-service mechanisms where the monitoring system automatically adjusts its sampling rate and selection criteria based on observed query patterns and system conditions. This enables the system to maintain reliable execution data dynamically without requiring constant manual intervention or fixed resource allocation, thereby reducing overall monitoring overhead time while preserving data reliability.
Data Source
AI summary
Techniques for capturing query execution statistics are disclosed. The disclosed system determines which queries are executing during a sampling interval. The system identifies related queries that share similarity criteria with the currently executing query. The system obtains query execution statistics for the query executing during the sampling interval and any related queries. The system generates or obtains statistics during a monitoring window that spans a longer period of time than a single intervening period between the current sampling interval and a prior interval. The statistics are stored in permanent storage or transmitted to a requesting computer. The system refrains from analyzing statistics for queries (and any related queries) that did not execute during a current sampling interval.