Graph-Based Memory Storage Compiler for GPU Data Locality

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Deep learning and other software applications face delays due to the need to load and store data from different memory locations during parallel computations, leading to inefficiencies in GPU processing.

Innovation Solution

A compiler generates kernels based on graphs to determine optimal memory locations for data storage, using graph generators and memory constraints calculators to identify consecutive memory locations for vectorized data sets, reducing the need for frequent loading and storing.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Productivity

If data is loaded from and stored to different locations in memory for parallel computations, then computations can be performed in parallel using GPUs, but loading and storing causes delays in execution

Engineering Contradiction:

Improveparallel computation throughputVSAvoidmemory access delay

Core Design Contradiction:

ProductivityVSLoss of time

Solution Approach 1:

The patent merges data that is accessed together during parallel computations into the same memory location or adjacent memory locations. By combining data that has spatial and temporal locality into contiguous memory regions, the system reduces the number of separate memory access operations required, thereby decreasing memory access delays while maintaining parallel computation throughput

Inventive Principle:

Principle #5Merging (Combining)

Solution Approach 2:

The patent performs preliminary analysis of computation graphs to identify data access patterns and dependencies before execution. By pre-determining which data should be stored together in memory based on their usage patterns in the computation graph, the system prepares optimal memory layouts in advance, eliminating the need for frequent data movement during actual computation execution

Inventive Principle:

Principle #10Preliminary action

2Adaptability or versatility

If data is stored in different memory locations for each parallel computation, then data can be accessed by multiple processors, but the number of memory locations increases leading to inefficiency

Engineering Contradiction:

Improvedata accessibilityVSAvoidmemory location management

Core Design Contradiction:

Adaptability or versatilityVSDevice complexity

Solution Approach 1:

The patent creates memory layouts where data is organized to serve multiple computational operations simultaneously. By storing data in a unified memory structure that can be accessed by multiple processors for different operations, the system maintains high data accessibility while reducing the total number of memory locations required, thereby simplifying memory management complexity

Inventive Principle:

Principle #6Universality (Multi-functionality)

Data Source

PatentUS20230244391A1Graph-based memory storage

Publication Date: 2023.08.03 NVIDIA CORP

AI summary

Apparatuses, systems, and techniques to cause information to be stored in one or more memory locations based, at least in part, on one or more graphs. In at least one embodiment, a compiler analyzes one or more graphs to determine one or more sets of data items to be stored in one or more consecutive memory locations.