Work Unit Stack Executors for Async DPU Concurrency

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Traditional CPU programming models face inefficiencies in handling large numbers of concurrent threads due to overhead costs such as context switching and data locking, while DPU programming models, although improving this, introduce challenges in composability and programmer productivity.

Innovation Solution

Adapting asynchronous functions to DPU programming using a WU stack executor, where async functions are executed in chunks between await points, retaining their natural form and leveraging a WU stack to manage execution, thus avoiding thread overheads and data locking.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Productivity

If traditional CPU multi-threading is used to handle concurrent activities, then throughput and resource utilization improve, but context switching overhead and data locking overhead increase

Engineering Contradiction:

ImprovethroughputVSAvoidcontext switching overhead

Core Design Contradiction:

ProductivityVSDevice complexity

Solution Approach 1:

The patent segments concurrent activities into discrete work units that are pushed onto a stack, allowing the DPU to process them in a simplified manner without traditional thread overhead. Each work unit represents a discrete task that can be independently managed and executed.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent introduces a work unit stack as an intermediary data structure between the producer DPU and consumer DPUs. This stack mediates the transfer and management of concurrent activities, eliminating the need for complex thread synchronization and context switching mechanisms.

Inventive Principle:

Principle #24Intermediary (Mediator)

2Use of energy by moving object

If traditional CPU multi-threading is used to handle concurrent activities, then resource utilization improves, but data locking overhead increases

Engineering Contradiction:

Improveresource utilizationVSAvoiddata locking overhead

Core Design Contradiction:

Use of energy by moving objectVSDevice complexity

Solution Approach 1:

The work unit stack operates in a self-service manner where work units are autonomously pushed onto the stack by producer DPUs and pulled off by consumer DPUs without requiring external synchronization or locking mechanisms. Each DPU independently manages its own work unit processing.

Inventive Principle:

Principle #25Self-service

3Device complexity

If DPU programming model is used to execute concurrent activities, then thread overhead is avoided, but composability and programmer productivity suffer

Engineering Contradiction:

Improvethread overheadVSAvoidprogrammer productivity

Core Design Contradiction:

Device complexityVSEase of operation

Solution Approach 1:

The work unit stack serves multiple functions: it acts as a queue for work units, a stack for LIFO processing, and a synchronization mechanism all in one. This universal data structure simplifies the programming model while maintaining efficient concurrent execution.

Inventive Principle:

Principle #6Universality (Multi-functionality)

Solution Approach 2:

The patent uses stack copying techniques where work unit stacks can be duplicated and transferred between DPUs, allowing efficient task distribution and load balancing without complex inter-process communication mechanisms.

Inventive Principle:

Principle #26Copying

Data Source

PatentUS20260003644A1Asynchronous function executors utilizing work unit stacks

Publication Date: 2026.01.01 MICROSOFT TECHNOLOGY LICENSING LLC

AI summary

Implementations for executing asynchronous functions using a work unit stack executor on a data processing unit are provided. One aspect includes a computing system for executing asynchronous functions using a work unit (WU) stack executor, the computing system comprising a data processing unit including a plurality of programmable processing cores configured to execute an asynchronous function by performing a call to the asynchronous function, creating a future corresponding to the asynchronous function, creating a WU stack, creating the WU stack executor on the WU stack to execute the future and sending a WU to start the WU stack executor.