PCIe Bridge Interrupt Address Translation for Multi-CPU Initialization

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Initializing interrupts in a PCIe-based Data Center interconnect is challenging when extending PCIe technology for use between multiple independent CPUs, as the CPU programming the interrupt may not be aware of the proper address, leading to initialization failures.

Innovation Solution

A management I/O device controller is used to map and register target interrupt register addresses across multiple worker computing resources, enabling proper interrupt initialization and delivery within a PCIe fabric, utilizing a translating interrupt register table to coordinate interrupt handling across the network.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If PCIe is extended for use as a unified interconnect technology between components of a data center, then cost efficiency and power consumption are improved, but interrupt initialization becomes problematic because the CPU programming the interrupt at the device may not be aware of the proper address

Engineering Contradiction:
Improveinterrupt initialization reliabilityVSAvoidaddress mapping complexity
Core Design Contradiction:
ReliabilityVSDevice complexity

Solution Approach 1:

The patent introduces a bridge device as an intermediary between the CPU and I/O devices in the PCIe fabric. The bridge maintains a translation table that maps device-specific interrupt addresses to CPU-visible interrupt addresses. When an I/O device generates an interrupt, the bridge translates the address through the translation table and delivers the interrupt to the correct CPU location, resolving the address awareness problem without requiring CPU knowledge of device-specific addresses.

Inventive Principle:
Principle #24Intermediary (Mediator)

Solution Approach 2:

The bridge device creates a copy of the device's interrupt register address space in its translation table. This copied address mapping allows the CPU to interact with a simplified address space while the bridge handles the actual address translation to the device-specific address space, enabling interrupt initialization without CPU awareness of the proper device addresses.

Inventive Principle:
Principle #26Copying

2Reliability

If a translation table is introduced to map interrupt addresses, then interrupt delivery reliability is improved, but the complexity of the address mapping system increases

Engineering Contradiction:
Improveinterrupt delivery reliabilityVSAvoidaddress mapping system complexity
Core Design Contradiction:
ReliabilityVSDevice complexity

Solution Approach 1:

The bridge device autonomously maintains and updates the translation table based on configuration information received during system initialization. The bridge automatically performs address translation without requiring CPU intervention or complex software management, reducing the operational complexity despite the presence of the translation mechanism. The system configures itself through standard PCIe configuration mechanisms.

Inventive Principle:
Principle #25Self-service

3Speed

If PCIe is used for CPU-to-CPU communication in data centers, then bandwidth and latency are improved, but interrupt initialization fails when CPUs are not aware of proper interrupt addresses

Engineering Contradiction:
ImproveCPU-to-CPU communication speedVSAvoidinterrupt initialization success
Core Design Contradiction:
SpeedVSReliability

Solution Approach 1:

The bridge acts as a mediator that enables fast PCIe-based CPU-to-CPU communication while simultaneously handling interrupt address translation. The bridge receives interrupts from I/O devices, translates addresses through its translation table, and delivers them to the target CPU, maintaining both the high speed benefits of PCIe and reliable interrupt initialization without requiring CPU awareness of device addresses.

Inventive Principle:
Principle #24Intermediary (Mediator)

Data Source

PatentEP3033681B1Method and apparatus for delivering MSI-x interrupts through non-transparent bridges to computing resources in PCI-express clusters
Publication Date: 2020.01.15 HUAWEI TECH CO LTD
  • EP3033681B1 patent drawingFigure 1
  • EP3033681B1 patent drawingFigure 2
  • EP3033681B1 patent drawingFigure 3

AI summary

An apparatus for initialization. The apparatus includes a management I/O device controller for managing initialization of a plurality of I/O devices coupled to a PCI-Express (PCIe) fabric. The management I/O device controller is configured for receiving a request to register a target interrupt register address of a first worker computing resource, wherein the target interrupt register address is associated with a first interrupt generated by a first I/O device coupled to the PCIe fabric. A mapping module of the management I/O device controller is configured for mapping the target interrupt register address to a mapped interrupt register address of a domain in which the first I/O device resides. A translating interrupt register table includes a plurality of mapped interrupt register addresses in the domain that is associated with a plurality of target interrupt register addresses of a plurality of worker computing resources.