Asynchronous Data Movement API for Nonblocking Thread Processing

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Existing data movement operations in parallel computing platforms consume significant time, power, and computing resources, necessitating improved efficiency.

Innovation Solution

Implementing an application programming interface (API) that utilizes manual transaction accounting for asynchronous data movement operations, allowing threads to perform other operations while data is being moved, using asynchronous data movement hardware and synchronization hardware to track and synchronize data movement efficiently.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If synchronous data movement operations are used, then data transfer reliability is ensured, but computing resources and time are significantly consumed

Engineering Contradiction:
Improvedata transfer reliabilityVSAvoidcomputing resource efficiency
Core Design Contradiction:
ReliabilityVSProductivity

Solution Approach 1:

The patent segments data movement operations into independent transfer tasks that can be executed asynchronously. Each transfer operation is divided into discrete units that can be tracked and managed separately, allowing the system to initiate multiple data transfers without waiting for each to complete before starting the next one.

Inventive Principle:
Principle #1Segmentation

Solution Approach 2:

The patent implements preliminary action by pre-configuring transfer descriptors and setting up data movement operations before they are actually executed. Transfer descriptors contain all necessary information (source, destination, size, control parameters) prepared in advance, enabling the hardware to execute transfers autonomously without continuous software intervention.

Inventive Principle:
Principle #10Preliminary action

2Reliability

If threads wait for data movement completion, then data integrity is maintained, but processing time increases significantly

Engineering Contradiction:
Improvedata integrityVSAvoidprocessing time
Core Design Contradiction:
ReliabilityVSLoss of time

Solution Approach 1:

The patent introduces transfer descriptors as intermediary structures that mediate between software threads and hardware data movement operations. These descriptors act as contracts that define the transfer parameters and enable the hardware to execute transfers independently, allowing threads to continue processing without blocking on transfer completion.

Inventive Principle:
Principle #24Intermediary (Mediator)

Solution Approach 2:

The patent replaces the traditional synchronous waiting mechanism with an asynchronous event-driven approach. Instead of threads mechanically blocking and waiting for transfer completion, the system uses hardware-generated completion events and status flags that notify threads when transfers are done, allowing non-blocking processing.

Inventive Principle:
Principle #28Mechanics substitution (Replace mechanical system)

3Productivity

If manual transaction accounting is implemented, then resource consumption is reduced, but system complexity increases

Engineering Contradiction:
Improveresource consumption efficiencyVSAvoidsystem complexity
Core Design Contradiction:
ProductivityVSDevice complexity

Solution Approach 1:

The patent implements self-service by enabling the data movement hardware to automatically track and account for its own transactions. The hardware autonomously manages transfer state, updates completion status, and generates event notifications without requiring external software management, thereby reducing the accounting overhead on the CPU while maintaining efficiency.

Inventive Principle:
Principle #25Self-service

Data Source

PatentUS12450683B2Application programming interface to provide information
Publication Date: 2025.10.21 NVIDIA CORP
  • US12450683B2 patent drawing
  • US12450683B2 patent drawing
  • US12450683B2 patent drawing

AI summary

Apparatuses, systems, and techniques to cause information to be provided. In at least one embodiment, one or more circuits are to perform an application programming interface (API) to cause an amount of information to be accessed as a result of one or more memory transactions to be provided to one or more users.