PCIe Memory Mapping via Non-Transparent Bridges
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing memory mapping systems for PCIe buses, such as Non-Transparent Bridges (NTBs), are limited by the number of Base Address Registers (BARs), which restricts the number of addresses that can be mapped, hindering efficient inter-host communication and resource utilization.
Innovation Solution
A memory mapping method and system that configures extended memory addresses on a management host and servers, using multiple non-transparent bridges to map these addresses across the PCIe bus, allowing for global memory address mapping and sharing of virtual functions with only one BAR per NTB function, thereby maximizing PCIe bus bandwidth.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If traditional NTB memory mapping is used with limited BARs, then the structure is simple, but the number of mappable addresses is restricted
Solution Approach 1:
The patent introduces extended memory addresses beyond the traditional 32-bit address space, utilizing 64-bit addressing capability. This dimensional expansion allows the system to map a vastly larger number of addresses without increasing the number of BARs, thereby resolving the contradiction between quantity of mappable addresses and device complexity
Solution Approach 2:
The patent implements a hierarchical memory mapping structure where extended memory addresses are mapped through multiple levels of NTB bridges. The first NTB maps extended addresses to intermediate addresses, and the second NTB maps intermediate addresses to final physical addresses. This nested mapping approach enables comprehensive address coverage while maintaining manageable complexity at each mapping level
2Quantity of substance
If multiple BARs are used to increase address mapping capacity, then more addresses can be mapped, but PCIe bus bandwidth is reduced
Solution Approach 1:
The patent makes each BAR universal by enabling it to handle extended memory address mapping through the multi-level NTB bridge structure. Instead of requiring separate BARs for different address ranges, a single BAR can effectively serve multiple address spaces through the extended addressing mechanism, thereby increasing mappable addresses without consuming additional PCIe bandwidth
Solution Approach 2:
By transitioning from 32-bit to 64-bit extended memory addressing, the system expands the addressable space exponentially without requiring proportional increases in BAR count. This dimensional change allows the same hardware resources to support a much larger address space, maintaining high PCIe bandwidth utilization while dramatically increasing the number of mappable addresses
3Productivity
If extended memory addressing is implemented, then inter-host communication efficiency improves, but the configuration complexity increases
Solution Approach 1:
The patent introduces intermediate address spaces that serve as mediators between extended memory addresses and physical addresses. The first NTB bridge creates an intermediate mapping layer that simplifies the configuration process by breaking down the complex extended address mapping into manageable segments, thereby improving communication efficiency while controlling configuration complexity
Solution Approach 2:
The memory mapping system is segmented into multiple NTB bridges, each handling specific portions of the address mapping. This segmentation allows the complex task of extended address mapping to be divided into simpler, manageable segments, reducing the overall configuration complexity while enabling efficient inter-host communication through coordinated mapping operations
Data Source
AI summary
A memory mapping method for coupling a plurality of servers with a PCI express bus is disclosed. The method comprises: configuring an extended memory address on a management host having a memory address; mapping the extended memory address of the management host corresponding to each of the servers to memory addresses of each of the servers respectively by a plurality of non-transparent bridges of the PCI express bus; configuring an extended memory address on each of the servers; and mapping the extended memory address of each of the servers to the memory address and the extended memory address of the management host by the non-transparent bridges, the extended memory address of each of the servers corresponding to the servers and the management host.