Private Memory Mode Warp Access Using Scalar Addresses

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing GPU architectures face inefficiencies in loading and storing data for multiple threads due to the need for vector operations and vector address buses, which complicates data addressing for concurrent thread execution.

Innovation Solution

A method and apparatus that utilize scalar memory addresses to efficiently load and store data for multiple threads by generating a scalar memory address based on thread block dimension information and addressing parameters, allowing data to be loaded or stored from consecutive memory addresses using a memory interface and memory unit.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Adaptability or versatility

If vector operations and vector address bus are used to load and store data for multiple threads, then data can be addressed for concurrent thread execution, but the device complexity and operation complexity increase

Engineering Contradiction:

Improvedata addressing capabilityVSAvoidaddressing system complexity

Core Design Contradiction:

Adaptability or versatilityVSDevice complexity

Solution Approach 1:

The patent segments the addressing task by separating the base address generation (scalar operation) from the thread-specific address calculation. The scalar memory address provides the base, while thread identifiers and block dimension information are used to calculate individual thread addresses, dividing the complex vector addressing into manageable scalar components.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent introduces an intermediary addressing mode that acts as a mediator between scalar and vector addressing. The scalar memory address serves as an intermediary base address, which is then combined with thread-specific offsets to generate complete thread addresses, avoiding the need for full vector address bus operations.

Inventive Principle:

Principle #24Intermediary (Mediator)

2Adaptability or versatility

If vector operations are used to prepare data addresses for multiple threads, then concurrent thread execution is supported, but the productivity and efficiency of data loading decreases

Engineering Contradiction:

Improvemulti-thread data accessVSAvoiddata loading efficiency

Core Design Contradiction:

Adaptability or versatilityVSProductivity

Solution Approach 1:

The system enables self-service addressing where each thread can calculate its own address offset from the scalar base address using its thread identifier and block dimension information. This eliminates the need for centralized vector operation preparation, allowing threads to independently access their data efficiently.

Inventive Principle:

Principle #25Self-service

Solution Approach 2:

The scalar memory address is prepared in advance as a base address before thread execution begins. This preliminary action provides a ready-to-use reference point that threads can quickly combine with their individual offsets, eliminating the need for time-consuming vector operations during actual data access.

Inventive Principle:

Principle #10Preliminary action

3Adaptability or versatility

If vector address bus is used for data addressing, then multiple thread data addresses can be generated, but the ease of operation and simplicity of the addressing process deteriorates

Engineering Contradiction:

Improvemulti-thread addressingVSAvoidaddressing process simplicity

Core Design Contradiction:

Adaptability or versatilityVSEase of operation

Solution Approach 1:

Instead of using complex vector operations to generate multiple addresses simultaneously, the patent inverts the approach by using a single scalar address combined with simple thread-specific calculations. This reversal simplifies the operation from complex parallel vector addressing to simple sequential scalar-based addressing.

Inventive Principle:

Principle #13The other way round (Inversion)

Data Source

PatentUS20250291710A1Private memory mode sequential memory access in multi-threaded computing

Publication Date: 2025.09.18 AZURENGINE TECH ZHUHAI INC

AI summary

Processors, systems and methods are provided for thread level parallel processing where threads in a warp are concurrently executed. A method may include sending thread block dimension information and addressing parameters stored in scalar registers of a sequencer by the sequencer to a memory port, generating a scalar memory address by the memory port and based on the thread block dimension information and the addressing parameters, and loading K words of data for a warp to a vector register from K consecutive memory addresses starting at the scalar memory address by a memory interface and a memory unit. The scalar memory address may point to a storage location of a piece of data for a first thread of the warp. And K may be a warp size representing a number of threads to be concurrently processed by executing a vector instruction.