Multi-GPU Query Partitioning and Memory Management
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing systems face challenges in efficiently executing queries on multi-graphics processing unit (GPU) systems, particularly in allocating and managing data partitions across multiple GPUs and CPUs to optimize query processing and result handling.
Innovation Solution
A method and system that identify the number and topology of GPUs and CPUs, allocate data partitions based on query criteria, and manage memory by releasing least recently used data to ensure efficient query execution across multiple GPUs and CPUs, allowing for parallel processing and result consolidation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If data partitions are allocated across multiple GPUs and CPUs for parallel query processing, then query execution speed is improved, but system complexity and memory management overhead increase
Solution Approach 1:
The patent divides the data source into multiple partitions and allocates them across different GPUs and CPUs. Each processing unit works on its assigned partition independently, enabling parallel query execution. This segmentation allows the system to scale horizontally by adding more processing units while maintaining manageable complexity for each individual unit.
Solution Approach 2:
The patent introduces a coordination mechanism that acts as an intermediary between multiple GPUs and CPUs. This mediator manages partition allocation, tracks data locations, and coordinates result consolidation. By centralizing management functions, the system reduces the complexity burden on individual processing units while maintaining efficient parallel operation.
2Productivity
If memory storage is increased to hold all data partitions and result sets, then query processing efficiency is improved, but memory consumption and storage requirements worsen
Solution Approach 1:
The patent pre-allocates data partitions to specific GPUs and CPUs before query execution based on expected query patterns and data access frequencies. Hot partitions that are likely to be accessed frequently are kept in memory, while less frequently accessed partitions are stored on disk. This preliminary arrangement reduces the need for excessive memory while maintaining efficient query processing.
Solution Approach 2:
The patent implements a memory management strategy where least recently used partitions and intermediate result sets are discarded from memory and moved to disk storage when memory capacity is constrained. When these data become needed again, they are recovered from disk back into memory. This approach allows the system to maintain high query processing efficiency for active data while minimizing overall memory consumption.
Data Source
AI summary
A system and method allocates partitions to be used to perform a query to multiple GPUs and each partition is also allocated to a CPU to which the GPU is attached. The GPUs use any assigned data in GPU memory, and otherwise request it from a CPU, which provides it from CPU memory if possible, and otherwise copies it from long term storage, CPU memory and makes it available to GPU memory. The GPUs process the query against the partitions assigned, optionally by performing portions of the query at a time, for example performing an innermost nested query and then performing the next outermost nested query. The results from any given portion of a query may be accumulated across all GPUs and some or all of the results distributed to each of the GPUs. The remaining work may be balanced by redistributing only some of the accumulated results.