Parallel Heapsort Buffer Locking for Top K Queries
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional heapsort operations cannot be effectively used in parallel processing frameworks due to race conditions and false results arising from the lack of synchronization primitives to control access to the heap, which hinders efficient execution of top k queries in database systems.
Innovation Solution
A parallelized heapsort method is implemented using worker threads that update buffers with candidate elements, acquiring a lock only when the buffer reaches a threshold size to update the heap, thereby synchronizing access and reducing synchronization overhead, and generating results for top k queries by maintaining a max-heap or min-heap structure.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If conventional heapsort is used without synchronization primitives, then processing speed is improved, but race conditions occur causing false results
Solution Approach 1:
The patent applies preliminary action by having worker threads populate local buffers with candidate elements from their respective data partitions before attempting to update the global heap. This preliminary buffering stage allows parallel data collection without immediate synchronization conflicts, and only when buffers reach threshold size do threads acquire locks to update the heap, thus maintaining both speed and reliability
2Reliability
If synchronization primitives are added to control heap access, then result accuracy is improved, but synchronization overhead increases
Solution Approach 1:
The patent segments the heapsort operation into distinct phases: (1) parallel buffer population phase where threads independently collect candidate elements without synchronization, and (2) sequential heap update phase where threads acquire locks only when buffers reach threshold size. This segmentation minimizes the duration and frequency of lock acquisitions, reducing synchronization overhead while ensuring result accuracy
Solution Approach 2:
The patent uses partial action by having threads update the heap only when their local buffers reach a threshold size, rather than updating immediately for every element. This threshold-based approach reduces the frequency of synchronization operations while still maintaining the correctness of the top-k result, thereby reducing overall synchronization overhead
3Productivity
If parallel processing is implemented for top k queries, then productivity is improved, but race conditions arise from lack of synchronization
Solution Approach 1:
The patent introduces local buffers as intermediary structures between worker threads and the global heap. Threads first populate their local buffers with candidate elements from data partitions, and only when buffers reach threshold size do they acquire locks to transfer elements to the global heap. This intermediary buffering mechanism enables parallel processing while mediating access conflicts to the shared heap structure
Data Source
AI summary
A method for performing a parallelized heapsort operation may include updating, by a first worker thread, a first buffer while a second worker thread updates a second buffer in parallel. The first worker thread may update the first buffer by adding, to the first buffer, elements from a first partition of a dataset. The second worker thread may update the second buffer by adding, to the second buffer, elements from a second partition of the dataset. Upon the first buffer reaching a threshold size, the first worker thread may acquire a lock for the first worker thread to update a heap based on the first buffer while the second worker thread is prevented from updating the heap based on the second buffer. A result of a top k query comprising a k quantity of smallest elements from the dataset may be generated based on the heap.