Direct Addressing of Edge Accelerators via Global Memory

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Current edge computing systems face latency and overhead issues when accessing accelerated functions hosted on different accelerator devices, as they often require CPU intervention to determine the appropriate device, leading to increased processing time and costs.

Innovation Solution

Assigning direct addresses to each accelerator device in the edge network, allowing requests to specify the target device directly, thereby reducing the need for CPU mediation and lowering latency by using various addressing schemes such as global memory or network addresses.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Ease of operation

If CPU intervention is used to determine the appropriate accelerator device, then serviceability and control are improved, but latency and processing time increase

Engineering Contradiction:

ImproveserviceabilityVSAvoidlatency

Core Design Contradiction:

Ease of operationVSLoss of time

Solution Approach 1:

The patent introduces a global memory address space as an intermediary layer between the edge device and accelerator devices. This address space acts as a mediator that enables direct addressing of accelerator devices without requiring CPU intervention to determine the target device, thus reducing latency while maintaining serviceability through the structured address space.

Inventive Principle:

Principle #24Intermediary (Mediator)

Solution Approach 2:

The patent enables accelerator devices to be directly addressable through the global memory address space, allowing the system to self-route requests without CPU mediation. The edge device can directly specify the target accelerator device through address encoding, making the system self-sufficient in routing decisions and eliminating the time-consuming CPU intervention step.

Inventive Principle:

Principle #25Self-service

2Adaptability or versatility

If CPU mediation is used to access accelerator devices, then resource management is improved, but processing speed and efficiency decrease

Engineering Contradiction:

Improveresource managementVSAvoidprocessing speed

Core Design Contradiction:

Adaptability or versatilityVSProductivity

Solution Approach 1:

The patent replaces the mechanical CPU mediation process with a direct memory-addressing mechanism. Instead of using CPU instructions to determine and route to accelerator devices, the system uses direct address specification through the global memory address space, substituting a software-based routing mechanism with a hardware-efficient addressing scheme that maintains resource management capabilities while dramatically improving processing speed.

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

3Loss of time

If direct addressing is implemented without CPU mediation, then latency is reduced, but system complexity increases

Engineering Contradiction:

ImprovelatencyVSAvoidaddressing scheme complexity

Core Design Contradiction:

Loss of timeVSDevice complexity

Solution Approach 1:

The patent creates a universal global memory address space that serves multiple functions: it provides direct addressing of accelerator devices, maintains resource management capabilities, and enables both CPU-mediated and direct addressing modes. This multi-functional address space reduces the need for separate addressing mechanisms, thereby reducing overall system complexity while achieving low-latency direct addressing.

Inventive Principle:

Principle #6Universality (Multi-functionality)

Data Source

PatentUS11617227B2Technologies for providing hardware resources as a service with direct resource addressability

Publication Date: 2023.03.28 INTEL CORP

AI summary

Technologies for providing hardware resources as a service with direct resource addressability are disclosed. According to one embodiment of the present disclosure, a device receives a request to access a destination accelerator device in an edge network, the request specifying a destination address assigned to the destination accelerator device. The device determines, as a function of the destination address, a location of the destination accelerator device and sends the request to the destination accelerator device.