In-Place Memory Copy for Remote Data Transfer in Accelerators

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

In heterogeneous compute environments, the existing method of direct memory copy for remote data transfer incurs latency and allocates large temporary buffers, which can consume valuable memory resources from the accelerator's compute kernel.

Innovation Solution

Implementing a hardware access control mechanism in the accelerator's memory management unit to allow in-place memory copy by gating write transactions into the accelerator memory without using a staging or temporary buffer, and defining a state machine to manage the availability of accelerator pages for memory copy and compute kernel use.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Speed

If direct memory copy is used for remote data transfer, then data transfer speed is improved, but latency increases and temporary buffers consume valuable memory resources

Engineering Contradiction:

Improvedata transfer speedVSAvoidtransfer latency

Core Design Contradiction:

SpeedVSLoss of time

Solution Approach 1:

The patent extracts and eliminates the temporary buffer allocation step from the data transfer process. By implementing in-place memory copy, the system removes the intermediate staging buffer that traditionally consumed memory resources and added latency, allowing direct writing of transferred data to the final destination memory location.

Inventive Principle:

Principle #2Taking out (Extraction)

Solution Approach 2:

The patent merges the data transfer operation with the memory allocation operation. Instead of allocating temporary buffers separately and then copying data to them, the system combines these operations by directly copying data to the final destination memory location, reducing the number of memory operations and associated latency.

Inventive Principle:

Principle #5Merging (Combining)

2Reliability

If temporary buffers are allocated for memory copy, then data integrity is ensured, but memory resource utilization deteriorates

Engineering Contradiction:

Improvedata integrityVSAvoidmemory resource utilization

Core Design Contradiction:

ReliabilityVSQuantity of substance

Solution Approach 1:

The patent removes the temporary buffer allocation mechanism from the system. By implementing in-place memory copy, it extracts the intermediate storage step that consumed memory resources, allowing data to be transferred directly to its final destination without occupying additional memory buffers.

Inventive Principle:

Principle #2Taking out (Extraction)

Solution Approach 2:

The patent changes the memory management parameters by implementing atomic in-place copy operations. This approach modifies how memory is accessed and managed during data transfer, using hardware-supported atomic operations to ensure data integrity without requiring additional buffer allocations.

Inventive Principle:

Principle #35Parameter changes

3Quantity of substance

If in-place memory copy is implemented, then memory resource utilization is improved, but hardware access control complexity increases

Engineering Contradiction:

Improvememory resource utilizationVSAvoidhardware access control mechanism

Core Design Contradiction:

Quantity of substanceVSDevice complexity

Solution Approach 1:

The patent introduces a hardware access control mechanism as an intermediary layer between the data transfer operation and the memory system. This mediator implements atomicity and consistency controls for in-place memory copy operations, managing the complexity of concurrent access and ensuring data integrity without requiring software intervention.

Inventive Principle:

Principle #24Intermediary (Mediator)

Solution Approach 2:

The patent implements self-service memory management through hardware-supported atomic operations. The in-place memory copy mechanism automatically handles consistency and concurrency control through hardware enforcement, eliminating the need for software-based locking or synchronization mechanisms.

Inventive Principle:

Principle #25Self-service

4Reliability

If hardware access control mechanism is added, then data corruption is prevented, but device complexity increases

Engineering Contradiction:

Improvedata corruption preventionVSAvoidmemory management unit complexity

Core Design Contradiction:

ReliabilityVSDevice complexity

Solution Approach 1:

Inventive Principle:

Principle #24Intermediary (Mediator)

Solution Approach 2:

The patent replaces software-based data protection mechanisms with hardware-enforced atomic operations. By substituting mechanical/software control with hardware-level atomicity and consistency controls, the system prevents data corruption through more efficient and reliable hardware-supported operations.

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

Data Source

PatentUS12299286B2In-place memory copy during remote data transfer in heterogeneous compute environment

Publication Date: 2025.05.13 INTEL CORP

AI summary

An apparatus to facilitate in-place memory copy during remote data transfer in a heterogeneous compute environment is disclosed. The apparatus includes a processor to receive data via a network interface card (NIC) of a hardware accelerator device; identify a destination address of memory of the hardware accelerator device to write the data; determine that access control bits of the destination address in page tables maintained by a memory management unit (MMU) indicate that memory pages of the destination address are both registered and free; write the data to the memory pages of the destination address; and update the access control bits for memory pages of the destination address to indicate that the memory pages are restricted, wherein setting the access control bits to restricted prevents the NIC and a compute kernel of the hardware accelerator device from accessing the memory pages.