Graph Data Storage Partitioning for NUMA Access Efficiency

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Existing graph data storage technologies suffer from low efficiency due to uniform memory access, where multiple processors accessing the same memory lead to reduced data access efficiency.

Innovation Solution

A three-level partitioning method is implemented for graph data storage in a non-uniform memory access (NUMA) architecture, dividing data into multiple partitions based on communication modes among computing devices, memories, and processors, and storing these partitions in separate NUMA nodes to optimize access efficiency.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Productivity

If multiple processors share a single physical memory for graph data storage, then the system structure is simple and easy to implement, but the data access efficiency deteriorates due to uniform memory access limitations

Engineering Contradiction:
Improvedata access efficiencyVSAvoidmemory architecture complexity
Core Design Contradiction:
ProductivityVSDevice complexity

Solution Approach 1:

The patent applies segmentation by dividing the single physical memory into multiple virtual memory spaces, with each processor assigned a dedicated virtual memory space. This logical segmentation allows processors to access their assigned data without contention, improving data access efficiency while maintaining the simplicity of a single physical memory structure. The virtual memory spaces are managed through a memory management unit that handles address translation and allocation.

Inventive Principle:
Principle #1Segmentation

2Loss of time

If graph data is stored in a unified memory space accessible by all processors, then the implementation is straightforward, but the communication overhead between processors increases due to memory access conflicts

Engineering Contradiction:
Improvecommunication overhead timeVSAvoidmemory management ease
Core Design Contradiction:
Loss of timeVSEase of operation

Solution Approach 1:

The patent implements local quality by allowing each processor to have preferential access to a specific portion of the memory space assigned to its virtual memory space. This creates local memory regions with optimized access characteristics for each processor, reducing communication overhead and memory access conflicts. The memory management unit enforces these local access patterns, improving overall system performance while maintaining manageable memory operations through standardized allocation protocols.

Inventive Principle:
Principle #3Local quality

3Productivity

If graph data is partitioned across multiple NUMA nodes, then the data access efficiency for each processor is improved, but the system complexity increases due to non-uniform memory access architecture

Engineering Contradiction:
Improvegraph data processing throughputVSAvoidNUMA architecture complexity
Core Design Contradiction:
ProductivityVSDevice complexity

Solution Approach 1:

The patent applies dimensionality change by introducing a virtual memory space dimension that overlays the physical NUMA architecture. This virtual dimension provides a unified address space for programmers while the underlying system manages the complexity of mapping virtual addresses to physical NUMA nodes. The memory management unit handles the translation and routing, allowing processors to access data on appropriate NUMA nodes without programmers needing to understand NUMA complexity, thus improving processing throughput while abstracting away the architectural complexity.

Inventive Principle:
Principle #17Another dimension (Dimensionality change)

Data Source

PatentUS12182201B2Graph data storage method, system and electronic device
Publication Date: 2024.12.31 ZHEJIANG TMALL TECH CO LTD
  • US12182201B2 patent drawing
  • US12182201B2 patent drawing
  • US12182201B2 patent drawing

AI summary

A graph data storage method for non-uniform memory access architecture (NUMA) processing system is provided. The processing system includes at least one computing device, each computing device corresponding to multiple memories, and each memory corresponding to multiple processors. The method includes: performing three-level partitioning on graph data to obtain multiple third-level partitions based on a communication mode among computing device(s), memories, and processors; and separately storing graph data of the multiple third-level partitions in NUMA nodes corresponding to the processors. A graph data storage system and an electronic device are further provided.