GPU Shared-Memory Channels for Faster Inter-Module Data Exchange
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing GPU architectures face inefficiencies due to high demand for global memory bandwidth and long data reading paths, leading to increased computational delays and hardware costs, as working groups on different computing modules cannot directly access each other's shared memory.
Innovation Solution
Establish a direct data transferring channel between the shared memories of different computing modules to facilitate direct data interaction without detours through caches or global memory, reducing the need for bandwidth and accelerating data reading speed.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If data is transferred through global memory and caches between working groups on different computing modules, then data interaction is enabled, but data reading path becomes long and bandwidth demand increases
Solution Approach 1:
The patent segments the data transfer path by introducing intermediate shared memories on different computing modules, breaking the long global memory access path into shorter segments. Each computing module's shared memory acts as a local staging area, reducing the distance data must travel and enabling parallel transfer operations.
Solution Approach 2:
The patent introduces shared memories on computing modules as intermediary structures between working groups. These intermediaries enable direct data exchange without requiring data to traverse the entire global memory path, effectively mediating the data transfer and reducing the reading path length.
2Ease of operation
If data is transferred through global memory and caches between working groups on different computing modules, then data interaction is enabled, but computational delay increases
Solution Approach 1:
By segmenting the data transfer into multiple shorter hops through intermediate shared memories, the patent enables overlapping of transfer operations with computation. Working groups can begin processing locally available data while data transfer continues in the background, masking the transfer time and reducing overall computational delay.
Solution Approach 2:
The patent implements preliminary data loading into computing module shared memories before the actual computation begins. This preliminary action prepares data in advance at the location where it will be needed, so that when computation starts, the data is already available locally, eliminating wait time during the computation phase.
3Productivity
If direct data transfer channel is established between shared memories of different computing modules, then data reading path is shortened and bandwidth demand is reduced, but hardware complexity increases
Solution Approach 1:
The patent makes the shared memories on computing modules serve multiple functions: they act as local data storage for working groups, as intermediary transfer points for inter-module communication, and as buffering structures for data reusability. This multi-functionality justifies the added hardware by maximizing its utility across different operational scenarios.
Solution Approach 2:
The patent merges the functions of global memory, caches, and inter-module transfer buffers into a unified architecture where computing module shared memories perform multiple roles. By combining these functions into existing structural elements rather than adding completely separate components, the hardware complexity increase is minimized while achieving the desired performance improvements.
4Adaptability or versatility
If working groups on different computing modules access each other's shared memory, then data reusability is improved, but global memory access increases
Solution Approach 1:
The patent implements local quality by enabling working groups to access shared memories on other computing modules directly, creating localized data exchange pathways. This allows data to be reused locally at the computing module level without being pulled back through global memory, maintaining high data reusability while minimizing global memory access traffic.
Data Source
AI summary
A data processing method, a data processing apparatus, an electronic device, and a computer-readable storage medium. The data processing method is applied to a data processing apparatus. The data processing apparatus comprises a plurality of computing modules, wherein each of the plurality of computing modules comprises a plurality of thread execution units and a shared memory shared by the plurality of thread execution units, and the plurality of computing modules comprise a first computing module and a second computing module. The data processing method comprises: by means of a data transmission channel established between the shared memory of the first computing module and the shared memory of the second computing module, directly transmitting data to be exchanged between a first workgroup run by the first computing module and a second workgroup run by the second computing module.