Memory Expansion Cards for GNN Sampling Latency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing graph neural network (GNN) distributed systems face bottlenecks in high-latency graph sampling operations, irregular access patterns leading to load imbalance and low storage bandwidth utilization, particularly in large-scale processing.
Innovation Solution
A data operation system and method utilizing a memory expansion unit with interconnected memory expansion cards for near-memory processing, performing sampling and aggregation operations close to storage, reducing communication costs and bandwidth underutilization by optimizing data transmission paths and offloading data operations to near-memory processing modules.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If distributed architecture is used for large-scale GNN processing, then processing capability is improved, but communication latency increases
Solution Approach 1:
The system segments the distributed GNN processing workload by introducing memory expansion cards as intermediate processing nodes between storage and compute units. Each memory expansion card handles local sampling and aggregation operations independently, dividing the overall processing task into distributed segments that can be executed in parallel with minimal inter-node communication.
Solution Approach 2:
The memory expansion card serves as an intermediary component between storage devices and data processing units. It performs sampling and aggregation operations locally before transmitting results to the data operation unit, thereby mediating the data flow and reducing the need for high-latency cross-node communication in the distributed architecture.
2Adaptability or versatility
If irregular access patterns are used for graph data, then flexibility is improved, but storage bandwidth utilization deteriorates
Solution Approach 1:
The memory expansion card performs preliminary sampling and aggregation operations directly on the graph data before it is accessed by the data processing unit. By pre-processing the data in the memory expansion card, the system maintains flexible irregular access patterns while reducing the amount of data that needs to be transferred, thereby improving storage bandwidth utilization.
3Adaptability or versatility
If data operations are performed remotely, then processing flexibility is improved, but communication overhead increases
Solution Approach 1:
The memory expansion card acts as an intermediary that performs sampling and aggregation operations locally between storage and the data processing unit. This local processing eliminates the need for frequent remote communications, reducing communication overhead and energy loss while maintaining processing flexibility through the distributed architecture.
Data Source
AI summary
A data operation system includes: a plurality of data processing units; a memory expansion unit communicatively coupled to the plurality of data processing units; a plurality of data operation units communicatively coupled to the plurality of data processing units and the memory expansion unit; and a plurality of first storage units communicatively coupled to the plurality of data processing units; wherein the memory expansion unit comprises a plurality of memory expansion cards, each of the plurality of data processing units is communicatively coupled to at least one of the plurality of memory expansion cards, and the plurality of memory expansion cards are interconnected.