Partitioned Row Limiting for Hierarchical Query Deduplication
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing database query methods for retrieving hierarchical data suffer from inefficiencies in processor and memory usage due to the need for post-processing and complex sorting operations, particularly when dealing with multi-vector queries and duplicate detection in non-sort keys.
Innovation Solution
Implementing partitioned sorting with duplicate elimination on non-sort keys and hierarchical filtering sorts to optimize database queries, using multiple levels of sorting to efficiently manage and eliminate duplicates, thereby reducing memory footprint and processor cycles.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If traditional post-processing approaches are used to retrieve hierarchical data, then query results can be obtained, but processor cycles and memory usage increase significantly
Solution Approach 1:
The patent applies preliminary action by performing duplicate elimination and hierarchical filtering during the sorting process itself, rather than after query results are retrieved. The system eliminates duplicates on non-sort keys and applies partitioned row limiting while data is being sorted, preventing unnecessary post-processing operations and reducing processor cycles and memory usage.
Solution Approach 2:
The patent segments the query processing into multiple levels of sorting operations, where each level handles a specific hierarchical level of data. This segmentation allows the system to process and eliminate duplicates at each hierarchical level independently, improving efficiency by avoiding the need to process all data at once through post-processing.
2Measurement precision
If complex sorting operations are performed to handle multi-vector queries, then accurate results are achieved, but device complexity increases
Solution Approach 1:
The patent segments the complex sorting operation into multiple simpler sorting levels, where each level sorts data at a specific hierarchical level. This segmentation reduces the complexity of individual sorting operations while maintaining the overall accuracy of multi-vector query results through systematic hierarchical processing.
Solution Approach 2:
The patent introduces partitioned row limiting as an intermediary mechanism between the sorting operations and the final query results. This intermediary structure manages the complexity by controlling which rows are passed between sorting levels, ensuring accurate results while reducing the computational burden through structured data management.
3Measurement precision
If duplicate elimination is performed on non-sort keys, then data quality improves, but processor cycles increase
Solution Approach 1:
The patent applies preliminary action by performing duplicate elimination on non-sort keys during the sorting process itself, rather than as a separate post-processing step. This integrates duplicate removal into the data flow, improving data quality while minimizing additional processor cycle consumption by avoiding redundant operations.
Solution Approach 2:
The patent merges duplicate elimination operations with the sorting operations into a unified process. By combining these operations, the system eliminates duplicates on non-sort keys without requiring separate processing passes, thereby improving data quality while reducing total processor cycle usage through operation consolidation.
Data Source
AI summary
A query can be received for data items that have a plurality of attributes that include a first attribute and a second attribute that has a hierarchical relationship with the first attribute. A first sort of the data items can be performed based on a first set of ordering keys including the first attribute and the second attribute. A second sort of the data items can be performed based on a second set of one or more ordering keys, the second set being a proper subset of the first set. Based on results of the second sort, a subset of results of the first sort can be selected. The subset of results can be returned in response to the query.