A multi-GPU double sorting Top-K optimization method based on shared memory and dynamic threshold queue
By adopting a multi-GPU bitone sorting optimization method based on shared memory and dynamic threshold queues, the computational power and memory bottlenecks of single-GPU platforms and the resource scheduling problem of multi-GPU platforms are solved, achieving efficient Top-K query and suitable for sorting algorithm optimization in multi-GPU systems.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- SOUTHWEAT UNIV OF SCI & TECH
- Filing Date
- 2026-03-27
- Publication Date
- 2026-06-09
AI Technical Summary
Existing Top-K algorithms suffer from computational and memory bottlenecks on single-GPU platforms, making it difficult to scale to multi-GPU platforms. Furthermore, resource scheduling and load balancing are challenging in multi-GPU environments, resulting in low computational speed and efficiency.
A multi-GPU bitone sorting optimization method based on shared memory and dynamic threshold queues is adopted. By reconstructing the bitone sorting into a two-stage iterative logic, a global dynamic elimination threshold queue is constructed, an adaptive update mechanism is designed, invalid data is eliminated, and the final sorting is completed on the host side.
It significantly reduces global communication and synchronization overhead, improves the parallel efficiency of multi-GPU systems, ensures computational correctness and reliability, and is applicable to different programming models for multi-GPU systems.
Smart Images
Figure CN122173311A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of parallel computing and GPU acceleration technology, specifically relating to a multi-GPU bitonetic sorting Top-K optimization method based on shared memory and dynamic threshold queues. Background Technology
[0002] As CMOS technology becomes increasingly difficult to master and manufacturing costs continue to rise, relying on increasing the computing resources of a single GPU system to improve throughput is becoming increasingly challenging. Simultaneously, the rapid proliferation and scaling of data-parallel workloads make it increasingly difficult for current single GPU systems to meet the demands of high-performance computing. Therefore, the industry is exploring the integration of multiple GPUs onto a single platform. Multi-GPU platforms exhibit significant advantages over single-GPU platforms. They can significantly improve computational speed and efficiency by distributing tasks through parallel computing. For example, in sorting algorithms, multiple GPUs can process different data blocks simultaneously, thereby accelerating the overall processing flow. Furthermore, multi-GPU platforms offer excellent scalability, allowing for flexible increases in the number of GPUs to enhance computing power on existing hardware without requiring a change to the existing core system architecture.
[0003] In the field of sorting algorithms, bitonic sort is often used to efficiently solve the Top-K problem (i.e., finding the top K largest or smallest elements in a dataset) due to its highly parallelizable characteristics. To further improve the computational performance of bitonic sort when processing massive amounts of data, it can be ported to a multi-GPU platform. A multi-GPU platform divides the data into multiple independent or overlapping intervals and distributes the sorting tasks for these intervals to multiple GPUs for parallel computation and merging, thereby achieving efficient processing of large-scale Top-K query tasks.
[0004] Regarding methods for solving the Top-K algorithm, Alabi et al. successfully ported the k-selection problem from a CPU serial approach to parallel execution on a single GPU. They not only revealed the limitations of directly porting it to a single GPU for parallel execution but also demonstrated how to refactor the algorithm to maximize thread-level parallelism and memory bandwidth on a single GPU. Tang et al. organized the Top-K algorithm into two stages for parallelism. The first stage divided the data into blocks, performing quicksort within each block to generate a candidate set of size K. The second stage merged the candidate sets of size K generated from all previous data blocks pairwise to generate another candidate set of size K, continuing this merging until the top K elements were generated. This method efficiently improved the Top-K problem. Jeff Johnson et al. used warps as the basic unit for register-level data exchange, significantly reducing the costly shared memory accesses and synchronization in the Top-K algorithm, thereby minimizing global communication and synchronization overhead.
[0005] Their common ground lies in their parallel design of the Top-K selection algorithm. On the one hand, they improve thread utilization and memory bandwidth on a single GPU platform by optimizing data partitioning and parallel strategies; on the other hand, they optimize by reducing global communication and synchronization overhead. These methods collectively drive the algorithm's performance closer to the theoretical limits of hardware.
[0006] While the aforementioned Top-K algorithms on single GPUs achieve high performance on their respective platforms, they still face two major challenges when dealing with massive datasets. First, their optimization heavily relies on specific hardware of a single GPU (such as warp and memory hierarchy), making it difficult to directly scale to multi-GPU platforms, and the computing power and memory of a single device are insufficient. Second, in a multi-GPU environment, effectively scheduling and allocating computing tasks to improve computational speed while ensuring result accuracy and avoiding load imbalance and communication bottlenecks remains a critical unresolved challenge. Summary of the Invention
[0007] The purpose of this invention is to propose a multi-GPU bitonetic sorting Top-K optimization method based on shared memory and dynamic threshold queues, which aims to solve the problems of computing power and memory bottlenecks and difficulties in multi-GPU expansion faced by the current Top-K algorithm on a single GPU platform when processing massive data sorting queries.
[0008] This invention is implemented as follows: A multi-GPU bitonetic sorting Top-K optimization method based on shared memory and a dynamic threshold queue includes the following steps: Step 1: Reconstruct the Top-K bitonic sorting algorithm into a two-stage iterative execution logic; Step 2: Construct a global dynamic elimination threshold queue, and extract candidate elimination thresholds from each data block and store them in this queue; Step 3: Design a global elimination threshold queue adaptive update mechanism to dynamically calculate the elimination threshold for the current round based on the distribution of candidate elimination thresholds in the queue; Step 4: Based on the elimination threshold, eliminate elements in the sorted data that do not meet the conditions, and reorganize the remaining data; Step 5: When the data size is reduced to below the preset threshold, the remaining data is sent back to the host to perform a complete bitonic sort.
[0009] Furthermore, in step one, the complete bitonic sorting network is decomposed into an elimination phase and a complete sorting phase. In the first phase, the original data is divided into multiple data blocks that match the capacity of the GPU shared memory. After each data block independently performs partial bitonic sorting, a candidate elimination threshold is extracted from the shared memory. Then, invalid data across data blocks is eliminated based on this threshold. When the data volume is reduced to below a preset capacity threshold, the second phase is entered to perform complete bitonic sorting.
[0010] Furthermore, step two involves constructing a global dynamic eviction threshold queue, which is used to receive and store candidate eviction thresholds extracted from shared memory after each data block completes a specific sorting stage.
[0011] Furthermore, step three involves designing an adaptive update mechanism for the global elimination threshold queue state. During a specific phase of the bitonic sorting, values are extracted from the data block as candidate elimination thresholds and added to the global queue. The candidate elimination thresholds in the queue are sorted, and the maximum value is selected as the elimination threshold for the current round. When subsequent candidate elimination thresholds are added and the queue again reaches the update trigger condition, the thresholds are reordered and a new elimination threshold is calculated. Whether to update the elimination criteria is determined by comparing the new threshold with the previous round's threshold.
[0012] Furthermore, in step four, based on the elimination threshold determined in the current round, elements with values greater than (in ascending order) or less than (in descending order) the threshold are eliminated within the data block. By reorganizing the remaining data, it is determined whether a new round of elimination iteration or the final complete sorting stage needs to be started based on the amount of reorganized data.
[0013] Furthermore, in step five, when the data size is reduced to below a preset capacity threshold, the recombined remaining data is sent back from the GPU global memory to the host. On the host, a complete bitonic sort is performed on these recombined remaining data, and a globally ordered Top-K sequence is directly output.
[0014] The beneficial effects of this invention are as follows: (1) By introducing a candidate threshold extraction mechanism based on shared memory, this invention utilizes the local ordered features of a specific stage of bitonic sorting to extract representative candidate thresholds without additional sorting, which significantly reduces global communication and synchronization overhead. (2) By eliminating invalid data in stages, the present invention significantly reduces the data size of subsequent sorting, reduces the computation time of each stage in the bitone sort, effectively reduces the running burden of a single GPU, and improves the overall parallel efficiency of a multi-GPU system. (3) The optimization method proposed in this invention is based on the standard parallel mode of bitonic sorting and can be implemented on different multi-GPU systems that support programming models such as CUDA and OpenCL. At the same time, the complete implementation process includes a result verification step, which ensures the correctness and reliability of the algorithm under large-scale parallel execution. Attached image description: Figure 1 is the implementation flow of the bitonetic sorting Top-K optimization method on a multi-GPU platform provided in an embodiment of the present invention; Figure 2 is a schematic diagram of the working principle of the global dynamic threshold queue provided in an embodiment of the present invention.
Claims
1. A multi-GPU bitone sorting Top-K optimization method based on shared memory and a dynamic threshold queue, wherein the multi-GPU bitone sorting Top-K optimization method based on shared memory and a dynamic threshold queue on a multi-GPU platform includes the following steps: Step 1: Reconstruct the Top-K bitonic sorting algorithm into a two-stage iterative execution logic; Step 2: Construct a global dynamic elimination threshold queue, and extract candidate elimination thresholds from each data block and store them in this queue; Step 3: Design a global elimination threshold queue adaptive update mechanism to dynamically calculate the elimination threshold for the current round based on the distribution of candidate elimination thresholds in the queue; Step 4: Based on the elimination threshold, eliminate elements in the sorted data that do not meet the conditions, and reorganize the remaining data; Step 5: When the data size is reduced to below the preset threshold, the remaining data is sent back to the host to perform a complete bitonic sort.
2. The multi-GPU bitone sorting Top-K optimization method with shared memory and dynamic threshold queue on a multi-GPU platform as described in claim 1, characterized in that, Step one decomposes the complete bitonic sorting network into an elimination phase and a complete sorting phase. In the first phase, the original data is divided into multiple data blocks that match the capacity of the GPU shared memory. After each data block independently performs a partial bitonic sort, a candidate elimination threshold is extracted from the shared memory. Then, invalid data across data blocks is eliminated based on this threshold. When the data volume is reduced to below the preset capacity threshold, the second phase is entered to perform a complete bitonic sort.
3. The multi-GPU bitone sorting Top-K optimization method with shared memory and dynamic threshold queue on a multi-GPU platform as described in claim 1, characterized in that, Step two involves constructing a global dynamic eviction threshold queue, which is used to receive and store candidate eviction thresholds extracted from shared memory after each data block completes a specific sorting stage.
4. The multi-GPU bitone sorting Top-K optimization method with shared memory and dynamic threshold queue on a multi-GPU platform as described in claim 1, characterized in that, Step three involves designing an adaptive update mechanism for the global elimination threshold queue state. During a specific phase of the bitonic sorting, each data block retrieves a value from shared memory as a candidate elimination threshold and adds it to the global queue. The candidate elimination thresholds in the queue are sorted, and the maximum value is selected as the elimination threshold for the current round. When subsequent candidate elimination thresholds are added and the queue again reaches the update trigger condition, the thresholds are reordered and a new elimination threshold is calculated. The elimination criteria are then compared with the threshold from the previous round to determine whether to update the elimination criteria.
5. The multi-GPU bitone sorting Top-K optimization method with shared memory and dynamic threshold queue on a multi-GPU platform as described in claim 1, characterized in that, Step four, based on the elimination threshold determined in the current round, eliminates elements within the data block whose values are greater than (in ascending order) or less than (in descending order) the threshold. By reorganizing the remaining data, it is determined whether a new round of elimination iteration or the final complete sorting stage needs to be started based on the amount of reorganized data.
6. The multi-GPU bitone sorting Top-K optimization method with shared memory and dynamic threshold queue on a multi-GPU platform as described in claim 1, characterized in that, In step five, when the data size is reduced to below the preset capacity threshold, the recombined remaining data is sent back from the GPU global memory to the host. On the host, a complete bitonic sort is performed on the recombined remaining data, and a globally ordered Top-K sequence is directly output.