PCIe NTB Posted Message Queue Scalability
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The existing Peripheral Component Interconnect Express (PCIe) Non-Transparent Bridge (NTB) Look-Up Table (R-LUT) is limited in memory space, preventing the deployment of large-scale processor resource pools in disaggregated data center architectures, as it can only support a small number of simultaneously connected processors.
Innovation Solution
Configuring processors with receive, transmit, and completion queues to manage posted messages exclusively, bypassing the NTB R-LUT and allowing for scalability by analyzing message content and placing them in appropriate queues for data transfers, thereby avoiding the limitations of the R-LUT.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If the PCIe NTB R-LUT is used to manage processor connections, then the system can support simultaneous processor connections, but the memory space limitation of the R-LUT prevents large-scale deployment (only supports small number of processors)
Solution Approach 1:
The patent extracts the message management function from the hardware R-LUT and relocates it to software-level queue structures. Posted messages are placed in transmit queues and processed by receive queues at the software level, completely bypassing the R-LUT. This extraction removes the memory space constraint of the hardware table while maintaining connection management functionality, enabling support for up to 64,000 processors.
Solution Approach 2:
The patent introduces queue structures (transmit queues and receive queues) as intermediary components between processors and the NTB. These queues act as mediators that buffer and manage posted messages without requiring R-LUT lookup. The intermediary queue mechanism decouples the message routing function from the limited R-LUT, allowing scalable processor connections.
2Adaptability or versatility
If posted messages are managed at hardware level via R-LUT, then message routing is automated, but scalability is limited by R-LUT memory capacity
Solution Approach 1:
The patent transitions message management from hardware dimension (R-LUT) to software dimension (queue structures). By moving the management layer from hardware to software, the system gains adaptability and scalability while maintaining automated processing through software-based queue management. This dimensional shift allows support for 64,000 processors without sacrificing automation.
3Reliability
If non-posted messages are used for data transfer, then message acknowledgment is ensured, but the additional message overhead increases communication complexity
Solution Approach 1:
The patent implements a self-service mechanism where the receiving processor autonomously manages posted messages by placing them in its transmit queue and processing them through its receive queue. This self-service approach eliminates the need for separate acknowledgment messages, as the receipt and processing of posted messages inherently provides the acknowledgment function, reducing message overhead while maintaining reliability.
Data Source
AI summary
A method of communicating data over a Peripheral Component Interconnect Express (PCIe) Non-Transparent Bridge (NTB) comprising transmitting a first posted write message to a remote processor via the NTB, wherein the first posted write message indicates an intent to transfer data to the remote processor, and receiving a second posted write message in response to the first posted write message, wherein the second posted write message indicates a destination address list for the data. Also disclosed is a method of communicating data over a PCIe NTB comprising transmitting a first posted write message to a remote processor via the NTB, wherein the first posted write message comprises a request to read data, and receiving a data transfer message comprising at least some of the data requested by the first posted write message.