Tensor Memory Accelerator Copy Engines for Asynchronous Block Transfers

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing graphics processors and accelerator devices face inefficiencies in handling large and complex data movements, which hinder the performance of arithmetic operations and specialized tasks such as tensor data processing.

Innovation Solution

Implementing a tensor data movement accelerator with a copy engine that performs asynchronous block copies of tensor data, offloading data movement tasks and optimizing data transfers for improved performance.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Speed

If processing elements perform data movements, then data transfer capability is improved, but arithmetic operation performance deteriorates due to resource allocation

Engineering Contradiction:

Improvedata transfer capabilityVSAvoidarithmetic operation performance

Core Design Contradiction:

SpeedVSProductivity

Solution Approach 1:

The patent segments the data transfer function from the processing elements by introducing a dedicated copy engine. This separation allows processing elements to focus exclusively on arithmetic operations while the copy engine handles data movement tasks, resolving the resource allocation conflict between data transfer and computation.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The copy engine acts as an intermediary component between memory and processing elements. It mediates data transfer operations, enabling efficient data movement without burdening the processing elements, thus allowing them to maintain high arithmetic operation performance.

Inventive Principle:

Principle #24Intermediary (Mediator)

2Reliability

If dedicated hardware units perform synchronous block copies, then data transfer reliability is improved, but processing throughput deteriorates due to synchronization overhead

Engineering Contradiction:

Improvedata transfer reliabilityVSAvoidprocessing throughput

Core Design Contradiction:

ReliabilityVSProductivity

Solution Approach 1:

The copy engine implements asynchronous periodic data transfer operations, allowing data movement to proceed independently of the main processing timeline. This eliminates synchronization overhead while maintaining reliable data transfer through structured periodic operations.

Inventive Principle:

Principle #19Periodic action

Solution Approach 2:

The asynchronous copy engine enables continuous data transfer operations that do not interrupt the processing flow. Data movement occurs continuously in the background, maintaining reliability without causing synchronization pauses that would reduce processing throughput.

Inventive Principle:

Principle #20Continuity of useful action

Data Source

PatentEP4617860A1Tensor memory accelerator enhancements

Publication Date: 2025.09.17 INTEL CORP

Figure 1
Figure 2A
Figure 2B~2C

AI summary

One embodiment provides a graphics processor comprising a memory interface and a graphics core cluster including a plurality of graphics cores and tensor processing circuitry. The tensor processing circuitry includes a local memory, a tensor accelerator coupled with the local memory, the tensor accelerator configured to perform a matrix multiply and accumulate operation, and a tensor data movement accelerator configured to asynchronously transfer tensor data between a global memory coupled to the memory interface and the local memory. The tensor data movement accelerator includes circuitry configured to translate the tensor data from a first tensor format to a second tensor format.