Query Statistics Capture via Linked List Thresholding
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing query processing systems face challenges in efficiently capturing statistics for long-running queries, as collecting extensive statistics for all queries can lead to excessive memory usage and performance degradation.
Innovation Solution
A computer-implemented method and system that utilize a linked list to store query execution statistics, where statistics are collected only for queries that exceed a predefined execution threshold, and are then stored in a statistics database, while discarding statistics for shorter queries to conserve memory.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If query statistics are collected for all queries, then comprehensive query performance data is obtained, but memory usage increases excessively and system performance degrades
Solution Approach 1:
The patent applies local quality by collecting detailed statistics only for long-running queries (those exceeding a threshold duration) while using simplified tracking for short queries. This selective approach focuses computational resources on queries that are more likely to impact system performance, thereby reducing overall memory usage while maintaining statistical completeness for significant queries.
Solution Approach 2:
The system changes the parameter of query selection by introducing a duration threshold. Queries are categorized into long-running and short queries based on this parameter, and different statistics collection strategies are applied to each category. This parameter-based differentiation resolves the contradiction by adjusting the scope of data collection based on query characteristics.
2Loss of information
If query statistics are collected for all queries, then complete performance analysis is enabled, but system performance degrades
Solution Approach 1:
The patent implements local quality by applying enhanced statistics collection only to long-running queries that exceed a duration threshold, while using minimal tracking for short queries. This selective approach ensures that system resources are not wasted on collecting detailed statistics for queries that complete quickly and have minimal impact on overall system performance, thereby maintaining system productivity.
Solution Approach 2:
The system performs partial action by collecting complete statistics only when necessary (for long-running queries) rather than for all queries. This partial collection strategy is sufficient to maintain system performance while still capturing all performance data that would significantly impact analysis and optimization decisions.
3Loss of information
If detailed statistics are captured for long-running queries, then real-time and historical query insights are provided, but memory consumption increases
Solution Approach 1:
The patent applies local quality by implementing detailed statistics collection specifically for long-running queries (those exceeding a duration threshold) while using simplified tracking for short queries. This selective approach ensures that memory resources are allocated efficiently, capturing comprehensive statistics only where they provide meaningful insights into system performance.
Solution Approach 2:
The system uses parameter changes by introducing a duration threshold to differentiate between query types. This parameter-based classification enables the system to adjust memory allocation dynamically, storing detailed statistics for long-running queries while using minimal memory for short queries, thereby resolving the memory consumption issue.
4Reliability
If query statistics collection is performed continuously, then comprehensive monitoring is achieved, but processing overhead increases
Solution Approach 1:
The patent implements local quality by applying continuous monitoring and detailed statistics collection only to long-running queries that exceed a duration threshold. For short queries, the system uses simplified tracking with minimal processing overhead. This selective approach maintains monitoring reliability for significant queries while reducing overall system complexity.
Solution Approach 2:
The system performs partial action by collecting detailed statistics only when query duration exceeds the threshold, rather than continuously for all queries. This partial collection strategy is sufficient to maintain accurate monitoring of system performance while significantly reducing processing overhead and complexity.
Data Source
AI summary
A computer-implemented method for efficiently capturing statistics on long running queries includes obtaining a query and obtaining a linked list including a plurality of records, each record in the plurality of records including respective query execution statistics. The method includes executing the query and, during execution of the query, obtaining new query execution statistics associated with the executing query. The method includes creating a new record in the linked list, the new record including the new query execution statistics. The method includes determining that a query execution duration of the query satisfies a query execution threshold. Further, the method includes identifying each record in the linked list that corresponds to the query. The method includes storing, for each respective identified record in the linked list that corresponds to the query, the respective query execution statistics of the respective identified record in a statistics database.