Asynchronous Low-Precision Compute Offload for ML Processors

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing data processing systems face challenges in efficiently accelerating low-precision computations, particularly in machine learning workloads, due to high circuit area and power costs associated with implementing remote accelerators and synchronous processing techniques.

Innovation Solution

An apparatus with decoding and processing circuitry that offloads low-precision computation tasks asynchronously to extension processing circuitry via a separate interface, reducing configuration overhead and circuit area/power costs by utilizing a dedicated hardware pipeline.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Speed

If remote accelerators are used to accelerate low-precision computations, then computation speed is improved, but circuit area and power consumption increase

Engineering Contradiction:

Improvecomputation speedVSAvoidcircuit area

Core Design Contradiction:

SpeedVSArea of stationary object

Solution Approach 1:

The extension processing circuitry is merged with the host data processing apparatus to form an integrated system. The extension task offload interface is combined with the existing memory system interface, eliminating the need for separate remote accelerator interfaces and reducing overall circuit area while maintaining acceleration capabilities.

Inventive Principle:

Principle #5Merging (Combining)

Solution Approach 2:

The extension processing circuitry is designed to handle multiple types of low-precision computation tasks including dot products, outer products, and other machine learning operations. This multi-functional design replaces the need for specialized remote accelerators for each operation type, reducing total circuit area while providing comprehensive acceleration.

Inventive Principle:

Principle #6Universality (Multi-functionality)

2Speed

If remote accelerators are used to accelerate low-precision computations, then computation speed is improved, but power consumption increases

Engineering Contradiction:

Improvecomputation speedVSAvoidpower consumption

Core Design Contradiction:

SpeedVSUse of energy by stationary object

Solution Approach 1:

The extension processing circuitry shares power supply infrastructure and control logic with the host data processing apparatus. By merging power management resources and using the existing memory interface for task offloading, the system reduces overall power consumption compared to separate remote accelerator implementations.

Inventive Principle:

Principle #5Merging (Combining)

3Device complexity

If synchronous processing techniques are used, then task coordination is simplified, but configuration overhead and processing time increase

Engineering Contradiction:

Improvetask coordination complexityVSAvoidconfiguration overhead time

Core Design Contradiction:

Device complexityVSLoss of time

Solution Approach 1:

The system dynamically selects between synchronous and asynchronous processing modes based on task characteristics. The extension processing circuitry can operate in asynchronous mode for independent computation tasks, allowing the host apparatus to continue processing without waiting for extension task completion, thereby reducing configuration overhead and improving throughput.

Inventive Principle:

Principle #15Dynamics

4Productivity

If a separate interface for extension tasks is implemented, then task offloading efficiency is improved, but device complexity increases

Engineering Contradiction:

Improvetask offloading efficiencyVSAvoidinterface complexity

Core Design Contradiction:

ProductivityVSDevice complexity

Solution Approach 1:

The extension task offload interface is merged with the memory system interface, allowing task offloading to share existing physical and logical infrastructure. This integration provides efficient task offloading capabilities while avoiding the complexity of completely separate interface implementations, as the same interface hardware handles both memory access and extension task submission.

Inventive Principle:

Principle #5Merging (Combining)

Data Source

PatentUS12561145B1Low-precision computation

Publication Date: 2026.02.24 ARM LTD

AI summary

An apparatus comprises decoding circuitry configured to decode instructions; processing circuitry configured to perform data processing operations in response to the instructions decoded by the decoding circuitry; extension processing circuitry configured to perform a low-precision computation extension task asynchronously with respect to other data processing operations performed by the processing circuitry, the low-precision computation extension task comprising processing one or more sets of data elements for which at least one of the sets of data elements comprises low-precision data represented in a low-precision number format with lower precision than a single-precision floating-point format; and an extension task offload interface separate from an interface by which the processing circuitry issues a memory system request to a memory system, wherein the extension task offload interface is responsive to at least one task offloading instruction decoded by the decoding circuitry to offload the low-precision computation extension task to the extension processing circuitry.