Parallel Query Processing Eliminating Redundant Table Scans
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In contemporary parallel processing systems, queries with maximum or minimum equality conditions require scanning a table twice, leading to extensive resource consumption, especially in systems with large tables, due to the lack of an index on the attribute used for these conditions.
Innovation Solution
The system scans the table only once and distributes rows to processing modules, where each module identifies and maintains rows with attribute values equal to the maximum or minimum value, allowing a global aggregation mechanism to compute the query result without a second scan, and can be extended to handle top N queries.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If the system scans the table twice to process queries with maximum or minimum equality conditions, then the query processing is complete and accurate, but the resource consumption becomes extensive and the processing time increases
Solution Approach 1:
The patent segments the table scan process by distributing rows to different processing modules (AMPs) in parallel. Each AMP independently identifies local maximum or minimum values during a single scan, eliminating the need for a second full table scan. This segmentation allows the system to maintain query accuracy while significantly reducing resource consumption and processing time.
2Measurement precision
If the system scans the table twice to identify and then filter rows by maximum or minimum attribute values, then the query result is accurate, but the processing time doubles
Solution Approach 1:
The patent applies preliminary action by having each processing module identify and maintain rows with maximum or minimum attribute values during the initial single table scan. Instead of performing a complete scan to find extremes and then a second scan to filter, the system performs the identification and filtering in parallel during one scan, significantly reducing processing time while maintaining result accuracy.
3Productivity
If the system broadcasts the maximum or minimum attribute value to every processing module after scanning, then the query can be completed, but the communication overhead and resource usage increase
Solution Approach 1:
The patent extracts the maximum or minimum identification function from the central coordinator and distributes it to individual processing modules. Each AMP independently determines local extremes during the scan, eliminating the need for centralized value computation and broadcasting. This extraction reduces communication overhead while maintaining query completion capability.
Data Source
AI summary
A system, method, and computer-readable medium for optimized processing of queries that feature maximum or minimum equality conditions are provided. A table on which the query is applied is scanned a single time. Rows of the table distributed to respective processing modules are scanned by the processing modules. Each processing module maintains identification of any rows distributed to the respective processing module that have attribute values that equal the maximum or minimum attribute value locally identified by the processing module. Subsequently, a global aggregation mechanism is invoked to compute the query result without requiring an additional rescan of the table. Further, the disclosed mechanisms may be extended to compute top N queries featuring maximum or minimum equality conditions.