Accumulation Buffers for Ordered Parallel Stream Processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional techniques for managing multiple computational operations on GPUs result in increased latency and reduced GPU utilization due to the need for sequential execution to maintain the correct order of outputs, which hinders parallel processing.
Innovation Solution
The implementation of accumulation buffers, where each buffer is assigned to a respective dispatch function (DF), allowing parallel execution of DFs with outputs stored in separate buffers, followed by a merge function to aggregate data in the correct order.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If sequential execution is used to maintain correct order of outputs, then output order accuracy is improved, but processing speed and GPU utilization deteriorate
Solution Approach 1:
The patent divides the output management into multiple independent accumulation buffers, each associated with a specific dispatch function. This segmentation allows parallel execution of multiple DFs while maintaining individual output order within each buffer, resolving the contradiction between parallel processing and output order accuracy.
Solution Approach 2:
The accumulation buffer acts as an intermediary data structure between parallel processing operations and the final output. It temporarily stores intermediate results from multiple DFs, allowing them to execute in parallel while ensuring correct ordering during the merge phase, thus maintaining output accuracy without sacrificing processing speed.
2Manufacturing precision
If sequential execution is used to maintain correct order of outputs, then output order accuracy is improved, but GPU utilization deteriorates
Solution Approach 1:
By segmenting the output storage into multiple accumulation buffers corresponding to different DFs, the system enables simultaneous execution of multiple GPU threads. Each buffer independently manages its associated DF's outputs, allowing full GPU utilization while maintaining correct output ordering through the merge function.
Solution Approach 2:
The patent introduces a new dimension of organization by associating accumulation buffers with specific DFs rather than using a single sequential output buffer. This dimensional change in data organization allows parallel operations to coexist with ordered output maintenance, maximizing GPU resource utilization.
3Productivity
If parallel execution is used, then processing speed is improved, but maintaining correct output order becomes difficult
Solution Approach 1:
The segmentation of outputs into DF-specific accumulation buffers allows parallel execution to proceed freely while each buffer maintains the ordering integrity of its associated DF's outputs. The final merge function combines these segmented outputs in the correct overall sequence, preserving accuracy despite parallel processing.
Solution Approach 2:
The accumulation buffers perform preliminary ordering of intermediate outputs from each DF before the final merge. This preliminary action ensures that when parallel results are combined, the correct global order is achieved without requiring sequential execution during the computationally intensive processing phase.
Data Source
AI summary
Described are apparatuses, systems, and techniques for efficient parallel execution of multiple processes in real-time streaming and latency-sensitive applications. The techniques include but are not limited to executing in parallel multiple processing threads, storing data output by the multiple processing threads in respective accumulation buffers, and applying an aggregation function to the stored data to generate an aggregated data.