Query Aggregation Definitions for Faster Database Workloads
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing database query systems face inefficiencies in computing resources and processing time, particularly when handling large datasets and multiple queries with similar aggregation requirements, leading to repetitive computations and increased resource utilization.
Innovation Solution
Implement a method and system that analyze query parameters' frequencies across a set of database queries to generate aggregation definitions, creating an aggregated dataset that can satisfy multiple queries, thereby reducing redundant computations and optimizing resource usage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If database queries are performed on large datasets with aggregation requirements, then query results can be obtained, but computing resources and processing time increase significantly due to repetitive aggregation computations
Solution Approach 1:
The system performs preliminary analysis of query workloads to identify common aggregation parameters before queries are executed. Aggregation definitions are pre-computed and stored based on frequent query patterns, so when similar queries arrive, the system can reuse these pre-computed aggregation results instead of performing redundant computations, significantly reducing processing time while maintaining query accuracy
Solution Approach 2:
The system creates universal aggregation definitions that can satisfy multiple different queries simultaneously. By identifying aggregation parameters that are common across many queries (e.g., grouping by location, time period), the system generates a single aggregated dataset that serves as a foundation for answering multiple queries, reducing the need for separate aggregation operations for each query
2Stability of the object's composition
If the same aggregation parameters are used across multiple queries, then query consistency is maintained, but computing resources are wasted due to repetitive aggregation steps
Solution Approach 1:
The system merges multiple queries that share common aggregation parameters into a single aggregation operation. By combining queries with similar requirements (e.g., multiple queries grouping by same location and time parameters), the system performs one aggregation computation that satisfies all merged queries, eliminating redundant computations while maintaining the consistency of aggregation results across all queries
Solution Approach 2:
The system dynamically adjusts aggregation parameters based on query frequency and patterns. By monitoring which aggregation parameters are most frequently used across queries, the system prioritizes pre-computation and caching of those specific parameter combinations, optimizing computing resource allocation by focusing on high-frequency aggregation scenarios while maintaining consistency for less frequent queries
Data Source
AI summary
Described are a system, method, and computer program product for accelerated database queries using aggregation definitions. The method includes receiving a first set of database queries and parsing each query to produce a plurality of query parameters. The method also includes determining a plurality of frequencies based on a frequency of each query parameter occurring in the first set of database queries. The method further includes generating a plurality of aggregation definitions based on the plurality of query parameters and the plurality of frequencies. The method further includes determining a candidate set of aggregation definitions from the plurality of aggregation definitions, based on a number of queries that would be at least partially satisfied by each aggregation definition of the plurality of aggregation definitions. The method further includes generating an aggregated dataset based on the candidate set of aggregation definitions and performing a database query using the aggregated dataset.