Multipart Query Processing via Intermediate Result Partitioning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current data warehouse implementations face inefficiencies in processing complex multiqueries, leading to unsatisfactory response times and excessive network traffic due to the need for extensive further processing of intermediate result sets with fine granularity.
Innovation Solution
A computer-implemented method and apparatus that processes multipart queries by partitioning and sorting intermediate results sets into disjoint groups, reducing processing resource consumption and generating a final results set with only the necessary data, utilizing an input processor, engine, and output processor to formulate and sort the results.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If conventional database queries with group by functionality are used to handle multipart queries, then the data repository can process the queries using standard operations, but the intermediate results set becomes excessively large with fine granularity data requiring extensive further processing
Solution Approach 1:
The patent applies segmentation by dividing the intermediate results set into multiple partitions based on the multipart query structure. Each partition corresponds to a specific part of the multipart query, allowing the system to process and filter data in smaller, manageable segments rather than handling the entire large result set at once. This reduces the amount of data that needs further processing while maintaining query accuracy.
Solution Approach 2:
The patent implements preliminary action by pre-processing the query to determine the optimal partitioning strategy before executing the full query. The system analyzes the multipart query structure in advance and prepares partitioning schemes that will minimize the size of intermediate results sets. This preliminary analysis allows the database to efficiently organize data retrieval and grouping operations to avoid generating excessive intermediate data.
2Quantity of substance
If the intermediate results set is extensively processed to identify relevant data and reduce its size, then the final results set contains only necessary data, but processing time and resource consumption increase significantly
Solution Approach 1:
The patent uses segmentation to divide the processing workload into parallel operations on different partitions of the intermediate results set. By processing multiple partitions simultaneously rather than sequentially filtering the entire large result set, the system reduces overall processing time while still achieving the goal of extracting only relevant data for the final results.
Solution Approach 2:
The patent applies preliminary action by pre-determining which partitions of the intermediate results set will contribute to the final results based on the multipart query structure. This allows the system to skip processing of partitions that won't contribute to the final answer, significantly reducing processing time while ensuring all necessary data is included in the final results set.
3Adaptability or versatility
If conventional group by operations are used without partitioning, then the query can be executed using standard database functionality, but network traffic increases due to excessive data transmission
Solution Approach 1:
The patent applies segmentation by partitioning the intermediate results set into multiple smaller groups that correspond to different parts of the multipart query. This segmentation allows the database to transmit only the necessary partitioned data to the application layer rather than transmitting the entire large intermediate results set, significantly reducing network traffic while maintaining compatibility with standard database operations.
4Adaptability or versatility
If the intermediate results set is generated with fine granularity, then all necessary data is available for any possible query variation, but the amount of data requires excessive processing resources
Solution Approach 1:
The patent uses segmentation to divide the fine-granularity intermediate results set into coarse-grained partitions based on the multipart query structure. Each partition aggregates data at an appropriate level of granularity for its specific query part, reducing the total amount of data that needs processing while preserving all necessary information needed to answer any variation of the multipart query.
Data Source
AI summary
A query having multiple parts may be processed to form an intermediate results set. This intermediate results set may be partitioned into a plurality of groups. Thereafter, the groups may be sorted into a plurality of containers so that each container contains data sufficient to calculate one requested result in the multipart query. Related techniques, apparatuses, systems, and computer program products are also described.