Graph Processing System Memory Segmentation for Page Swapping
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Graph processing systems using general-purpose graphics processing units (GPGPU) face performance degradation due to frequent page swapping and data copying between volatile memory devices, and the time required for checkpoint operations is longer than computation time, especially in graph algorithms like Google's PageRank, where iterative computations require efficient failure recovery.
Innovation Solution
A graph processing system utilizing two memory devices with different characteristics, where a nonvolatile memory device stores read-only graph data and a volatile memory device performs read-and-write operations, allowing for efficient checkpointing and iterative computations by rearranging and storing vertex values, and overlapping intermediate storage operations with computation and rearrangement processes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If only a volatile memory device with relatively small capacity is included in a GPGPU system, then the system structure is simple, but frequent page swapping and data copying occur causing performance degradation
Solution Approach 1:
The patent divides the memory system into two segments: a first memory device (non-volatile, large capacity) for storing graph data and a second memory device (volatile, small capacity) for storing vertex values. This segmentation allows each memory type to be optimized for its specific function, eliminating the need for frequent page swapping while maintaining system simplicity.
2Reliability
If a separate storage device using disk/flash-based storage is connected with the GPGPU system for checkpoint operations, then failure recovery is enabled, but the checkpoint time is longer than computation time
Solution Approach 1:
The patent introduces the first memory device (non-volatile memory) as an intermediary between the volatile memory and the external disk/flash storage. This intermediary enables fast checkpoint operations by storing vertex values locally in non-volatile memory, which has both large capacity and fast access speed, thereby reducing checkpoint time while maintaining reliability.
3Device complexity
If graph data is stored in a volatile memory device with small capacity, then the system is simpler, but frequent page swapping occurs degrading performance
Solution Approach 1:
The patent segments the memory system into a first memory device for storing large graph data and a second memory device for storing vertex values during computation. This segmentation allows the graph data to be stored in large-capacity non-volatile memory, eliminating frequent page swapping, while the volatile memory is used only for active computation data.
4Stability of the object's composition
If checkpoint operations are performed using disk/flash-based storage, then data persistence is achieved, but the checkpoint time exceeds the computation time
Solution Approach 1:
The patent changes the storage medium parameter from disk/flash-based storage to non-volatile memory for checkpoint operations. This parameter change maintains data persistence capability while dramatically reducing access time, allowing checkpoint operations to be completed faster than the computation time.
Data Source
AI summary
A method for operating a graph processing system including a first and second memory, comprising: storing in the first memory, a start index, a terminal index and an edge value of graph data; storing in the second memory, a start vertex of the graph data; rearranging the start vertex as an intermediate vertex by using to the start index; performing a graph computation on the intermediate vertex by using to the terminal index and the edge value; storing in the second memory, a terminal vertex as a result of the graph computation; determining whether a graph processing operation is completed by comparing the terminal vertex and the start vertex; setting the terminal vertex as the start vertex when the graph processing operation is not completed; and iterating the rearranging, the performing, the storing the terminal vertex, the determining and the setting until the graph processing operation is completed.