GPGPU Buffer Allocation via Access Boundary Alignment

Solution Overview

Problem

Current methods for allocating device buffers on General Purpose Graphics Processing Units (GPGPU) for objects with metadata result in inefficient memory access patterns, leading to increased execution time due to inappropriate memory alignment, which causes unnecessary memory transactions.

Innovation Solution

A method is implemented to analyze the program and determine the set of elements accessed during execution, allowing for optimized placement of objects in device buffers to minimize memory accesses by aligning with memory transaction boundaries, such as 128-byte granularity, and generating load and store instructions accordingly.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Manufacturing precision

If objects with metadata are copied to GPU using standard 128-byte alignment, then memory alignment is achieved, but the number of memory transactions increases due to misalignment between metadata and data access patterns

Engineering Contradiction:

Improvememory alignment precisionVSAvoidexecution speed

Core Design Contradiction:

Manufacturing precisionVSProductivity

Solution Approach 1:

The system performs preliminary analysis of the program to determine which elements of an object will be accessed during execution, then uses this information to pre-calculate the optimal starting offset for the object in device buffer memory. This preliminary action ensures that when the object is copied to the GPU, its metadata and data are positioned to align with memory transaction boundaries, preventing the need for additional memory transactions during execution.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The invention changes the memory allocation parameter by allowing objects to start at non-standard offsets within 128-byte aligned boundaries. Instead of forcing objects to start at fixed 128-byte boundaries, the system calculates optimal offsets within these boundaries based on the access pattern analysis, thereby aligning the actual data access patterns with memory transaction boundaries while maintaining overall 128-byte alignment structure.

Inventive Principle:

Principle #35Parameter changes

2Device complexity

If unappropriated memory allocation is used for objects with metadata, then memory simplicity is maintained, but the number of memory transactions increases

Engineering Contradiction:

Improvememory allocation complexityVSAvoidexecution speed

Core Design Contradiction:

Device complexityVSProductivity

Solution Approach 1:

The system performs preliminary analysis of the program to determine which elements of an object will be accessed during execution, then uses this information to pre-calculate the optimal starting offset for the object in device buffer memory. This preliminary action ensures that when the object is copied to the GPU, its metadata and data are positioned to align with memory transaction boundaries, preventing the need for additional memory transactions during execution.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The invention changes the memory allocation parameter by allowing objects to start at non-standard offsets within 128-byte aligned boundaries. Instead of forcing objects to start at fixed 128-byte boundaries, the system calculates optimal offsets within these boundaries based on the access pattern analysis, thereby aligning the actual data access patterns with memory transaction boundaries while maintaining overall 128-byte alignment structure.

Inventive Principle:

Principle #35Parameter changes

3Ease of operation

If metadata is placed at offset 0-15 for float arrays, then standard memory layout is maintained, but memory access efficiency decreases due to boundary misalignment

Engineering Contradiction:

Improvememory layout standardizationVSAvoidmemory access efficiency

Core Design Contradiction:

Ease of operationVSProductivity

Solution Approach 1:

The system applies local quality by allowing different parts of the memory layout to have different alignment characteristics. Specifically, while the overall object allocation maintains 128-byte alignment, the internal offset of metadata within the object can be adjusted to create optimal alignment for the actual data access patterns. This local adjustment enables the data portion to align with memory transaction boundaries even when metadata is placed at non-standard offsets.

Inventive Principle:

Principle #3Local quality

Solution Approach 2:

The invention changes the memory allocation parameter by allowing objects to start at non-standard offsets within 128-byte aligned boundaries. Instead of forcing objects to start at fixed 128-byte boundaries, the system calculates optimal offsets within these boundaries based on the access pattern analysis, thereby aligning the actual data access patterns with memory transaction boundaries while maintaining overall 128-byte alignment structure.

Inventive Principle:

Principle #35Parameter changes

Data Source

PatentUS11398004B2Allocating device buffer on GPGPU for an object with metadata using access boundary alignment

Publication Date: 2022.07.26 INTERNATIONAL BUSINESS MACHINE CORPORATION

AI summary

A method is provided for buffer allocation on a graphics processing unit. The method includes analyzing, by the graphics processing unit, a program to be executed on the graphics processing unit to determine, for an object in the program, a set of elements in the object that are designated to be accessed during an execution of the program. The method further includes allocating, by the graphics processing unit, a placement of the object in a device buffer on the graphics processing unit based on the set of elements to minimize a number of memory accesses during the execution of the program.