Run Length Encoded Data Query Processing on GPUs
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The limited device memory on GPUs and the bandwidth constraints of PCIe buses restrict the amount of data that can be processed concurrently, especially when performing query operations on large datasets.
Innovation Solution
Implementing Run Length Encoding (RLE) compression to reduce the size of data, allowing it to be processed more efficiently on GPUs without the need for decompression, and using hybrid RLE/index representations to optimize processing of data with varying value distributions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If data is transmitted from CPU to GPU over PCIe bus, then data processing can be performed on GPU, but bandwidth limitations restrict the amount of data that can be processed concurrently
Solution Approach 1:
The data is compressed using Run Length Encoding (RLE) before being transmitted from the CPU to the GPU. This preliminary compression action reduces the data size that needs to be transmitted over the PCIe bus, thereby overcoming the bandwidth limitation while maintaining the ability to process large datasets on the GPU
Solution Approach 2:
The data representation is changed from uncompressed format to RLE compressed format. This parameter change in data encoding reduces the quantity of data that must be transmitted and processed, allowing the GPU to handle larger datasets within the constraints of PCIe bandwidth
2Productivity
If device memory on GPU is used to store data, then query operations can be performed, but limited memory capacity restricts the amount of data that can be processed concurrently
Solution Approach 1:
Data is compressed to RLE format before being loaded into GPU device memory. This preliminary compression reduces the memory footprint of the data, allowing the GPU to store and process larger datasets concurrently within the limitations of its device memory capacity
Solution Approach 2:
The data is transformed from uncompressed representation to RLE encoded representation. This parameter change in data encoding reduces the volume of data that occupies GPU memory, enabling the system to handle larger datasets without exceeding memory constraints
3Ease of operation
If data is decompressed before query operations, then query processing can be performed, but additional processing time and resources are required
Solution Approach 1:
The patent extracts and utilizes the compression encoding directly for query operations without performing decompression. By taking out the compressed representation and operating on it directly, the system eliminates the time-consuming decompression step while still being able to perform necessary query operations on the data
Solution Approach 2:
The patent skips the decompression step entirely and performs query operations directly on the compressed RLE data. This skipping of the intermediate decompression phase significantly reduces processing time and resource requirements while maintaining query functionality
Data Source
AI summary
A method, computer program product, and computing system for processing query operations on run length encoding (RLE), data in a parallel processing computing system. Data for query execution is received at a parallel processing computing system, at least a portion of the data being compressed according to RLE, thereby forming RLE data; and a query operation is executed on the RLE data without performing a decompression operation on the RLE data.