Infrastructure Processing Unit Offloads RPC Latency via RDMA

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing RPC protocols, such as gRPC, suffer from high average latencies and wide latency distributions due to their software-focused implementations, which do not leverage domain-specific hardware like Infrastructure Processors (IPUs), leading to inefficiencies in cloud-based microservices environments.

Innovation Solution

A hardware engine is coupled with RDMA to support RPC operations, allowing for efficient network usage by making decisions closer to the network and decoupling interface-related decisions like workload distribution and load balancing from application logic, using components like IPUs, DPUs, or smartNICs.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Adaptability or versatility

If software-focused RPC protocols are used, then implementation flexibility is improved, but latency increases and performance deteriorates

Engineering Contradiction:

Improveimplementation flexibilityVSAvoidlatency

Core Design Contradiction:

Adaptability or versatilityVSLoss of time

Solution Approach 1:

The patent introduces an IPU as an intermediary device between the CPU and network infrastructure. The IPU offloads RPC protocol processing from the software layer to dedicated hardware circuits, achieving both low latency through hardware acceleration and flexibility through programmable logic that can be configured for different RPC protocols and workloads.

Inventive Principle:

Principle #24Intermediary (Mediator)

Solution Approach 2:

The patent replaces software-based RPC processing (mechanical system) with hardware-based processing on the IPU. This substitution uses dedicated circuits and logic gates to execute RPC operations directly in hardware, eliminating software interpretation overhead and reducing latency while maintaining protocol flexibility through programmable logic elements.

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

2Ease of operation

If software-focused RPC protocols are used, then ease of operation is improved, but productivity deteriorates

Engineering Contradiction:

Improveease of operationVSAvoidperformance

Core Design Contradiction:

Ease of operationVSProductivity

Solution Approach 1:

The IPU performs self-service by autonomously handling RPC protocol processing, load balancing decisions, and network packet manipulation without requiring CPU intervention. The programmable logic on the IPU automatically adapts to different RPC workloads and protocols, maintaining ease of operation while achieving high performance through hardware-accelerated processing.

Inventive Principle:

Principle #25Self-service

3Device complexity

If domain-specific hardware like IPU is not leveraged, then device complexity is reduced, but latency increases

Engineering Contradiction:

Improvedevice complexityVSAvoidlatency

Core Design Contradiction:

Device complexityVSLoss of time

Solution Approach 1:

The patent segments the computing system into distinct functional units: CPU for high-level application logic, IPU for RPC protocol processing and network operations, and specialized circuits for specific tasks. This segmentation allows each component to be optimized for its specific function, reducing overall latency while keeping individual components relatively simple and manageable.

Inventive Principle:

Principle #1Segmentation

Data Source

PatentUS20230342214A1Methods and apparatus for a remote processing acceleration engine

Publication Date: 2023.10.26 INTEL CORP

AI summary

Systems, apparatus, articles of manufacture, and methods are disclosed for a remote processing acceleration engine. Disclosed is an infrastructure processing unit (IPU) comprising an offload engine driver to access a remote procedure call (RPC) from business logic circuitry, network interface circuitry, and RPC offload circuitry to select a destination to perform an operation associated with the RPC call, the destination selected based on an ability of the destination to perform the operation using remote direct memory access (RDMA), and cause communication of the operation to the destination via the network interface circuitry.