Graph Compression Structure for Memory Access Optimization

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Conventional graph storage and access techniques experience performance degradation due to memory access overhead, particularly in complex data structures like graphs, where pointer-chasing significantly reduces performance, and existing solutions like distributing graphs across machines or using hardware acceleration are not effective for client and edge systems or do not justify the required resources.

Innovation Solution

A graph compression process that generates a compression information structure by determining protocol-dependent neighbors during pre-processing and storing them, allowing for efficient propagation from a node to its neighbors without pointer-chasing, thereby reducing resource requirements and improving performance.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Productivity

If conventional graph storage techniques are used, then graph data can be stored, but memory access overhead and pointer-chasing significantly reduce performance

Engineering Contradiction:

Improvegraph processing performanceVSAvoidmemory access time

Core Design Contradiction:

ProductivityVSLoss of time

Solution Approach 1:

The patent applies preliminary action by pre-processing the graph data structure before actual processing operations. Specifically, it performs a breadth-first search (BFS) traversal in advance to compute and store distance information, level assignments, and neighbor relationships. This pre-computed information is stored in optimized data structures that eliminate the need for pointer-chasing during runtime, thereby resolving the memory access overhead problem while maintaining graph processing capabilities

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The patent creates compressed copies of the original graph data structure with optimized layouts. It generates alternative representations such as compressed sparse row (CSR) format and level-based neighbor lists that copy only the essential connectivity information needed for processing. These copies eliminate random memory access patterns by organizing data in contiguous memory blocks, thus resolving the pointer-chasing performance issue

Inventive Principle:

Principle #26Copying

2Productivity

If graphs are distributed across multiple machines, then performance can be improved, but communication overhead and system complexity increase

Engineering Contradiction:

Improvegraph processing throughputVSAvoidsystem architecture complexity

Core Design Contradiction:

ProductivityVSDevice complexity

Solution Approach 1:

The patent applies segmentation by dividing the graph processing workload into independent stages that can be executed sequentially or in parallel within a single machine. It segments the graph into level-based partitions based on distance from source nodes, allowing each partition to be processed independently. This segmentation enables high throughput without requiring distributed system complexity, as each segment can be handled by separate processing threads or functional units within one system

Inventive Principle:

Principle #1Segmentation

3Productivity

If hardware acceleration is used, then performance improves, but resource requirements and cost increase

Engineering Contradiction:

Improveprocessing speedVSAvoidhardware resources

Core Design Contradiction:

ProductivityVSQuantity of substance

Solution Approach 1:

The patent replaces the need for specialized hardware acceleration by substituting mechanical/computational complexity with optimized data structures and algorithms. Instead of using GPU clusters or FPGA-based graph processors, it uses carefully designed memory layouts and pre-computed distance labels that enable fast processing on conventional CPUs. This substitution achieves high processing speeds without requiring additional hardware resources beyond standard system memory and processing units

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

4Adaptability or versatility

If complete graph information is stored, then all neighbor relationships are available, but memory usage increases significantly

Engineering Contradiction:

Improvegraph query capabilityVSAvoidmemory consumption

Core Design Contradiction:

Adaptability or versatilityVSVolume of stationary object

Solution Approach 1:

The patent extracts only the essential information needed for common graph operations and discards redundant data. It extracts distance labels, level assignments, and immediate neighbor relationships while eliminating the need to store complete adjacency matrices or full pointer chains. This extraction maintains adaptability for distance-based queries, shortest path computations, and neighborhood explorations while reducing memory consumption by storing only the minimal necessary graph information

Inventive Principle:

Principle #2Taking out (Extraction)

Data Source

PatentUS10817567B2Techniques for information graph compression

Publication Date: 2020.10.27 INTEL CORP

AI summary

Techniques and apparatus for providing graph compression structures for graph information are described. In one embodiment, for example, an apparatus may include at least one memory, at least one processing circuitry, and logic, coupled to the at least one processing circuitry, to access graph information comprising a plurality of nodes, define a unique index for each of the plurality of nodes, determine whether each of the plurality of nodes has at least one neighbor node, and generate a graph compression structure comprising an entry for each of the plurality of nodes having at least one neighbor node and an adjacency list comprising an array of neighbor nodes of each entry.