Shared Register Mapping for GPU Thread Data

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

The performance of machine learning on a GPU is bottlenecked by unnecessary memory allocation to data objects that are no longer needed, leading to inefficient use of resources and performance degradation.

Innovation Solution

A method and apparatus that determine shared data usage among threads, map shared data addresses to shared registers, and load shared data into these registers, reducing redundant data loading and optimizing memory usage.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Productivity

If shared data is loaded into separate registers for each thread, then each thread can access its data independently, but redundant data loading occurs and memory resources are wasted

Engineering Contradiction:
Improvethread execution efficiencyVSAvoidredundant data loading
Core Design Contradiction:
ProductivityVSLoss of energy

Solution Approach 1:

Multiple threads share a single register containing common data instead of each thread having its own copy. The register is marked as shared and accessible by multiple threads, eliminating redundant data loading from global memory and reducing memory bandwidth consumption.

Inventive Principle:
Principle #5Merging (Combining)

Solution Approach 2:

A single register serves multiple threads simultaneously by being marked as a shared register. This multi-functional register can be accessed by any thread in the thread block that requires the same data, making the register resource universal rather than dedicated to a single thread.

Inventive Principle:
Principle #6Universality (Multi-functionality)

2Speed

If more registers are allocated to store shared data for each thread, then data access speed improves, but the total number of registers increases and resource utilization decreases

Engineering Contradiction:
Improvedata access speedVSAvoidregister allocation complexity
Core Design Contradiction:
SpeedVSDevice complexity

Solution Approach 1:

Multiple threads share a single register containing common data instead of each thread having its own copy. The register is marked as shared and accessible by multiple threads, eliminating redundant data loading from global memory and reducing memory bandwidth consumption.

Inventive Principle:
Principle #5Merging (Combining)

Solution Approach 2:

An address-to-register mapping table acts as an intermediary to manage shared register allocation. When a thread needs data, the mapping table checks if the data is already loaded into a shared register, avoiding redundant loads and managing register resources efficiently across multiple threads.

Inventive Principle:
Principle #24Intermediary (Mediator)

3Quantity of substance

If shared data is kept in global memory for all threads, then memory capacity is sufficient, but access speed decreases due to memory bandwidth limitations

Engineering Contradiction:
Improvedata storage capacityVSAvoiddata access speed
Core Design Contradiction:
Quantity of substanceVSSpeed

Solution Approach 1:

Data is segmented into two categories: shared data that fits in registers and general data that remains in global memory. The shared data portion is loaded into shared registers for fast access by multiple threads, while other data remains in global memory, optimizing the balance between capacity and speed.

Inventive Principle:
Principle #1Segmentation

Solution Approach 2:

Shared data is pre-loaded from global memory into shared registers before thread execution begins. This preliminary action moves frequently accessed data closer to the processing units, enabling faster access during thread execution without compromising global memory capacity for other data.

Inventive Principle:
Principle #10Preliminary action

4Loss of energy

If an address-to-register mapping table is maintained to track shared data, then redundant loading is reduced, but the complexity of address management increases

Engineering Contradiction:
Improveredundant memory accessVSAvoidaddress mapping management
Core Design Contradiction:
Loss of energyVSDevice complexity

Solution Approach 1:

An address-to-register mapping table acts as an intermediary to manage shared register allocation. When a thread needs data, the mapping table checks if the data is already loaded into a shared register, avoiding redundant loads and managing register resources efficiently across multiple threads.

Inventive Principle:
Principle #24Intermediary (Mediator)

Data Source

PatentUS12298901B2Apparatus and method with register sharing
Publication Date: 2025.05.13 SAMSUNG ELECTRONICS CO LTD
  • US12298901B2 patent drawing
  • US12298901B2 patent drawing
  • US12298901B2 patent drawing

AI summary

An apparatus and method with register sharing are provided. In one general aspect, a method of operating a processing apparatus includes determining whether there is shared data that is used by each of threads in a plurality of threads sharing a shared memory, based on an instruction that has been decoded, based on determining whether there is shared data that is used by each of the threads in the plurality of threads, determining whether an address of the shared data corresponding to each of the threads in the plurality of threads is stored in an address-to-register mapping table, based on a result of either the determining whether the address is stored in the address-to-register mapping table, mapping the address of the shared data to a shared register corresponding to the shared data, and loading the shared data based on the shared register.