Graph Data Storage Partitioning for NUMA Access Efficiency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing graph data storage technologies suffer from low efficiency due to uniform memory access, where multiple processors accessing the same memory lead to reduced data access efficiency.
Innovation Solution
A three-level partitioning method is implemented for graph data storage in a non-uniform memory access (NUMA) architecture, dividing data into multiple partitions based on communication modes among computing devices, memories, and processors, and storing these partitions in separate NUMA nodes to optimize access efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If multiple processors share a single physical memory for graph data storage, then the system structure is simple and easy to implement, but the data access efficiency deteriorates due to uniform memory access limitations
Solution Approach 1:
The patent applies segmentation by dividing the single physical memory into multiple virtual memory spaces, with each processor assigned a dedicated virtual memory space. This logical segmentation allows processors to access their assigned data without contention, improving data access efficiency while maintaining the simplicity of a single physical memory structure. The virtual memory spaces are managed through a memory management unit that handles address translation and allocation.
2Loss of time
If graph data is stored in a unified memory space accessible by all processors, then the implementation is straightforward, but the communication overhead between processors increases due to memory access conflicts
Solution Approach 1:
The patent implements local quality by allowing each processor to have preferential access to a specific portion of the memory space assigned to its virtual memory space. This creates local memory regions with optimized access characteristics for each processor, reducing communication overhead and memory access conflicts. The memory management unit enforces these local access patterns, improving overall system performance while maintaining manageable memory operations through standardized allocation protocols.
3Productivity
If graph data is partitioned across multiple NUMA nodes, then the data access efficiency for each processor is improved, but the system complexity increases due to non-uniform memory access architecture
Solution Approach 1:
The patent applies dimensionality change by introducing a virtual memory space dimension that overlays the physical NUMA architecture. This virtual dimension provides a unified address space for programmers while the underlying system manages the complexity of mapping virtual addresses to physical NUMA nodes. The memory management unit handles the translation and routing, allowing processors to access data on appropriate NUMA nodes without programmers needing to understand NUMA complexity, thus improving processing throughput while abstracting away the architectural complexity.
Data Source
AI summary
A graph data storage method for non-uniform memory access architecture (NUMA) processing system is provided. The processing system includes at least one computing device, each computing device corresponding to multiple memories, and each memory corresponding to multiple processors. The method includes: performing three-level partitioning on graph data to obtain multiple third-level partitions based on a communication mode among computing device(s), memories, and processors; and separately storing graph data of the multiple third-level partitions in NUMA nodes corresponding to the processors. A graph data storage system and an electronic device are further provided.


