Hybrid On-Off Chip Memory Architecture for Graph Analytics
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current computing hardware systems face inefficiencies in executing graph algorithms due to poor performance and energy inefficiency, particularly with irregular-access patterns and high traffic between cores and off-chip memory, as well as sub-optimal performance for graphs with uniform or sparsely-connected vertices.
Innovation Solution
A hybrid on/off-chip memory architecture is proposed, where high-degree vertices' data is stored in on-chip memory and low-degree vertices' data is stored in off-chip memory, with atomic operations performed by dedicated compute units in both locations, optimizing data access and reducing memory traffic.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If atomic operations are computed by general-purpose cores, then computation flexibility is maintained, but core computation overhead increases and performance decreases
Solution Approach 1:
The patent extracts atomic operation computation from general-purpose cores and relocates it to specialized atomic operation computer units. This separation allows cores to focus on higher-level graph algorithm logic while dedicated units handle atomic operations, eliminating core computation overhead and improving overall productivity.
Solution Approach 2:
The patent introduces atomic operation computer units as intermediary components between cores and memory. These units act as mediators that handle atomic operations efficiently, reducing the time cores spend on such operations while maintaining system coherence through controlled memory access.
2Quantity of substance
If data is stored in off-chip memory, then memory capacity is increased, but memory traffic and access latency increase
Solution Approach 1:
The patent segments memory into on-chip and off-chip portions, with on-chip memory storing frequently accessed graph data structures and off-chip memory providing bulk storage. This segmentation allows the system to leverage both the speed of on-chip memory and the capacity of off-chip memory, optimizing the capacity-speed tradeoff.
Solution Approach 2:
The patent applies local quality by placing different types of data in different memory locations based on access patterns. Hot data that requires frequent access is stored in on-chip memory for fast access, while cold data is stored in off-chip memory, optimizing both capacity utilization and access speed for different data subsets.
3Productivity
If compute engines are co-located with off-chip memory, then atomic operation computation is offloaded from cores, but traffic between compute engines and off-chip memory increases
Solution Approach 1:
The patent merges the atomic operation computer unit with the memory system to enable in-memory computation. By combining computation and storage resources, the system performs atomic operations directly where data is stored, eliminating the need for separate data movement operations and reducing energy consumption associated with memory traffic.
4Speed
If data is stored in on-chip memory, then access latency is reduced, but memory capacity is limited
Solution Approach 1:
The patent implements a nested memory hierarchy where on-chip memory is nested within the chip and off-chip memory is nested outside. This nested structure allows the system to provide fast on-chip access for critical data while maintaining access to larger off-chip memory capacity, achieving both speed and capacity goals simultaneously.
Data Source
AI summary
The increased use of graph algorithms in diverse fields has highlighted their inefficiencies in current chip-multiprocessor (CMP) architectures, primarily due to their seemingly random-access patterns to off-chip memory. Here, a novel computer memory architecture is proposed that processes operations on vertex data in on-chip memory and off-chip memory. The hybrid computer memory architecture utilizes a vertex's degree as a proxy to determine whether to process related operations in on-memory or off-chip memory. The proposed computer memory architecture manages to provide up to 4.0× improvement in performance and 3.8× in energy benefits, compared to a baseline CMP, and up to a 2.0× performance boost over state-of-the-art specialized solutions.