Memory Extension Cards for Faster GNN Graph Data Access
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional systems face inefficiencies in accessing and processing unstructured graph data, particularly due to random memory accesses and the limitations of PCIe bandwidth and data transfer rates being too slow for applications involving large amounts of data, and the lack of scalability in memory capacity, especially in parallel processing environments.
Innovation Solution
Implement a system with a network of interconnected processing units and memory extension cards, each with a graphic access engine and a graphic access engine, and interconnected by a network of interconnected processing units, each configured to optimize and facilitate efficient memory access and data transfer rates, utilizing a network of interconnected memory extension cards, each configured to optimize and facilitate efficient data transfer rates.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If conventional PCIe interconnects are used for memory access, then system compatibility is maintained, but bandwidth and data transfer rates are too slow for large graph data processing
Solution Approach 1:
The system segments the memory architecture into multiple memory extensions (HBM1, HBM2, HBM3) with different bandwidth capabilities, allowing selective use of high-speed memory interfaces for graph data processing while maintaining PCIe compatibility for system integration
Solution Approach 2:
The patent introduces a new dimension to the memory hierarchy by adding memory extensions with higher bandwidth dimensions beyond conventional PCIe, enabling parallel data transfer paths that significantly increase data transfer rates for graph operations
2Quantity of substance
If memory capacity is extended to handle large graph data, then data processing capability improves, but memory access becomes more random and less efficient
Solution Approach 1:
The system performs preliminary actions by pre-loading and caching frequently accessed graph data into high-speed memory extensions before actual processing, reducing random memory accesses during graph neural network operations
Solution Approach 2:
Different memory extensions provide different local qualities - HBM1 offers high capacity with moderate speed, HBM2 and HBM3 offer higher speed with varying capacities, allowing optimization of memory access patterns based on data access characteristics
3Productivity
If parallel processing units are added to improve processing speed, then computational capability increases, but system complexity and interconnect requirements increase
Solution Approach 1:
Multiple processing units share common memory extensions through unified memory addresses, merging their memory access paths to reduce the number of interconnects required while maintaining parallel processing capability for graph operations
Solution Approach 2:
The memory extensions serve multiple functions - storing graph data, providing high-speed memory cache, and acting as data buffers for parallel processing units, reducing the need for separate dedicated memory per processor
Data Source
AI summary
This application describes systems and methods for facilitating memory access for graph neural network (GNN) processing. An example system includes a plurality of processing units, each configured to perform graph neural network (GNN) processing; and a plurality of memory extension cards, each configured to store graph data for the GNN processing, wherein: each of the plurality of processing units is communicatively coupled with three other processing units via one or more interconnects respectively; the plurality of processing units are communicatively coupled with the plurality of memory extension cards respectively; and each of the plurality of memory extension cards includes a graphic access engine circuitry configured to acceleratre GNN memory access.