Extra-band communication method and device of PCIe Switch
By integrating an MCTP bridge module and dynamic routing table into the PCIe Switch, the compatibility issues between MCTP devices are resolved, enabling comprehensive management and efficient transmission between devices, reducing hardware costs and design complexity, and adapting to complex device deployments.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- BEIJING SHUDU INFORMATION TECH CO LTD
- Filing Date
- 2026-04-17
- Publication Date
- 2026-05-15
AI Technical Summary
In the existing technology, MCTP over PCIe and MCTP over SMBus devices cannot interact directly, which makes it impossible for the management unit to effectively manage devices that only support SMBus communication, affecting the comprehensiveness and effectiveness of out-of-band management. At the same time, the addition of a bridging chip increases hardware cost and design complexity.
The PCIe Switch incorporates an MCTP bridge module and a dynamic routing table to enable bidirectional conversion and routing of MCTP over PCIe and MCTP over SMBus messages. The built-in controller handles message format conversion and routing management, and dynamically refreshes routing information to adapt to device access and removal.
It achieves compatibility and comprehensive management among MCTP devices, reduces hardware costs and design complexity, improves transmission efficiency and stability, and adapts to different device access and removal scenarios.
Smart Images
Figure CN122053473A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to an out-of-band communication method and apparatus for a PCIe switch, belonging to the field of communication technology for switching chips. Background Technology
[0002] In electronic devices such as computer systems and server systems based on PCIe Switch architecture, out-of-band management is a key link to ensure stable system operation. Through out-of-band management units (such as baseboard management controllers, BMCs), functions such as status monitoring, fault diagnosis, and firmware upgrades of various components and devices in the system can be realized.
[0003] Currently, MCTP is the mainstream communication protocol for out-of-band management, supporting multiple transmission media such as PCIe and SMBus (System Management Bus). However, existing technologies suffer from significant compatibility issues: some MCTP devices only support SMBus communication (MCTP over SMBus), while the management unit in the system (such as the BMC) only supports MCTP communication over PCIe (MCTP over PCIe). Because MCTP messages corresponding to the two communication media cannot directly interact, the management unit cannot effectively manage MCTP devices that only support SMBus communication. For example, it cannot collect status parameters such as device temperature and voltage, nor can it issue control commands, severely impacting the comprehensiveness and effectiveness of out-of-band management.
[0004] To address the aforementioned compatibility issues, existing solutions often employ the method of adding an additional independent MCTP bridging chip to achieve message format conversion between MCTP Over SMBus and MCTP Over PCIe.
[0005] However, this solution has many shortcomings: on the one hand, adding extra chips will increase the hardware cost of the system and increase the space occupied by the PCB board; on the other hand, the connection between the independent bridging chip and the PCIe Switch and management unit requires additional wiring and protocol adaptation, which increases the complexity of system design and poses risks such as signal interference and increased transmission delay.
[0006] Therefore, there is an urgent need for a PCIe Switch out-of-band communication solution that is highly integrated, low-cost, and highly compatible. Summary of the Invention
[0007] To address the shortcomings of existing technologies, this invention provides an out-of-band communication method and apparatus for PCIe Switches, aiming to solve the problem that MCTP interaction cannot be directly achieved between the management unit (supporting MCTP Over PCIe) and devices that only support MCTP Over SMBus, while reducing system hardware costs and design complexity.
[0008] The specific technical solution is as follows:
[0009] In a first aspect, an out-of-band communication device for a PCIe switch includes:
[0010] PCIe Switch, built-in controller, MCTP bridge module, and dynamic routing table;
[0011] The PCIe Switch has an upstream port and multiple downstream ports. The upstream port is connected to the management unit via the PCIe bus and is used to transmit MCTP Over PCIe messages. At least one of the downstream ports is connected to the component device via the SMBus bus and is used to transmit MCTP Over SMBus messages.
[0012] The built-in controller is integrated inside the PCIe Switch body, and the MCTP bridge module and dynamic routing table are both integrated into the built-in controller;
[0013] The MCTP bridge module is used to convert the message format between MCTP Over PCIe messages and MCTP Over SMBus messages.
[0014] The dynamic routing table is used to store routing information, which includes the identification information of the management unit and each component device, as well as the corresponding communication medium type and port information. The dynamic routing table supports dynamically refreshing the routing information according to the device access / removal status.
[0015] The built-in controller is used to receive MCTP messages from upstream or downstream ports, determine the target device and corresponding communication medium of the message by querying the dynamic routing table, call the MCTP bridge module to complete the message format conversion, and route the converted message to the target port. If the communication medium of the message is consistent with the communication medium of the target device, the message is directly routed to the target port.
[0016] A further improvement is made to the method of refreshing the dynamic routing table, which includes:
[0017] When a component device accesses / removes a downstream port via SMBus, the built-in controller detects changes in device status through the device enumeration mechanism of SMBus and updates the device identifier, port information, and communication medium type in the dynamic routing table based on the detection results.
[0018] When the management unit connects to / removes an upstream port, the built-in controller updates the corresponding information in the dynamic routing table through the PCIe hot-plug detection mechanism.
[0019] A further improvement is made to the message format conversion process of the MCTP bridge module, which includes:
[0020] When an MCTP Over PCIe message is received from the management unit, the target device identifier in the message is parsed. If the target device is a component device that only supports SMBus, the PCIe header of the MCTP Over PCIe message is stripped, an SMBus header is added, and it is converted into an MCTP Over SMBus message.
[0021] When an MCTP Over SMBus message is received from a component device, the target device identifier in the message is parsed. If the target device is a management unit, the SMBus header of the MCTP Over SMBus message is stripped, a PCIe header is added, and it is converted into an MCTP Over PCIe message.
[0022] In a further improvement, the management unit is a BMC.
[0023] The MCTP bridge module is either a hardware logic circuit or a firmware module embedded in the built-in controller.
[0024] The dynamic routing table is implemented using a content-addressable memory or a linked list structure, supporting fast lookup and dynamic update of entries indexed by device identifier.
[0025] Secondly, an out-of-band communication method for a PCIe switch includes the following steps:
[0026] S1: Initialize the dynamic routing table in the built-in controller, obtain information about the management unit and each component device currently connected to the PCIeSwitch through the device enumeration mechanism, and generate the initial routing table;
[0027] S2: The built-in controller monitors the device access / removal status of each port in real time. When a status change is detected, the routing information in the dynamic routing table is dynamically refreshed.
[0028] S3: The built-in controller receives MCTP messages from the upstream or downstream port;
[0029] S4: Parse the received MCTP message and extract the target device identifier from the message;
[0030] S5: Query the dynamic routing table to determine the communication medium type and target port corresponding to the target device based on the target device identifier;
[0031] S6: Determine whether the communication medium type of the received message is consistent with the communication medium type of the target device; if they are consistent, directly route the received message to the target port; if they are inconsistent, call the MCTP bridge module to complete the message format conversion of the received message, and then route the converted message to the target port.
[0032] A further improvement is made in step S2, where the process of dynamically refreshing the routing information in the dynamic routing table includes:
[0033] If a new component device is detected to access a downstream port via SMBus, the identification information of the device is obtained through SMBus enumeration, and the device identification, communication medium type, and corresponding downstream port information are added to the dynamic routing table.
[0034] If a component device is detected to have been removed from a downstream port of SMBus, the routing information corresponding to that device is deleted from the dynamic routing table.
[0035] If the management unit is detected to be accessing / removing an upstream port, the identification information, communication medium type, and upstream port information of the management unit in the dynamic routing table will be updated accordingly.
[0036] In a further improvement, step S6 includes the following specific process for message format conversion:
[0037] When the received message is an MCTP Over PCIe message and the target device is a component device that supports SMBus, the MCTP bridge module strips the PCIe header, extracts the MCTP payload, adds an SMBus header, and generates an MCTP Over SMBus message.
[0038] When the received message is an MCTP Over SMBus message and the target device is a management unit, the MCTP bridge module strips the SMBus header, extracts the MCTP payload, adds a PCIe header, and generates an MCTP Over PCIe message.
[0039] In a further improvement, in step S6, if the target device is a broadcast address or a multicast address, the built-in controller queries all matching downstream devices according to the dynamic routing table, and after copying and performing necessary transformations on the message, routes it to the corresponding port.
[0040] Thirdly, a PCIe Switch out-of-band communication system includes:
[0041] Management unit, supports MCTP over PCIe communication;
[0042] At least one component device supports MCTP Over SMBus communication; and the out-of-band communication device of the PCIe Switch is connected between the management unit and the component device to enable MCTP message interaction between the two.
[0043] The management unit is a baseboard management controller, and the component device includes at least one of a temperature sensor, a voltage sensor, a fan controller, or a power management unit.
[0044] Fourthly, a server comprising the aforementioned PCIe Switch out-of-band communication system.
[0045] The beneficial effects of this invention are:
[0046] 1. Resolving compatibility issues: By building an MCTP bridge module into the PCIe Switch, bidirectional conversion and routing of MCTP over PCIe and MCTP over SMBus messages are realized. This enables MCTP devices that only support SMBus communication to interact normally with management units that only support MCTP communication on PCIe media, ensuring the comprehensiveness of out-of-band management and enabling out-of-band management operations such as status monitoring and command issuance for various MCTP devices.
[0047] 2. Reduced cost and design complexity: The MCTP bridge module and dynamic routing table are integrated into the built-in controller of the PCIe Switch, eliminating the need for additional independent bridge chips, reducing hardware costs, saving PCB board space, and avoiding additional wiring and protocol adaptation work, thus reducing the complexity of system design, production and debugging.
[0048] 3. High flexibility and scalability: The dynamic routing table supports real-time updating of routing information based on the device access / removal status, and can automatically adapt to the access and removal of different numbers and types of component devices. No manual routing configuration is required, which improves the scalability and compatibility of the system and is suitable for complex and ever-changing device deployment scenarios.
[0049] 4. Improved transmission efficiency and stability: The built-in MCTP bridge module performs message format conversion directly inside the PCIe Switch, reducing latency and signal interference caused by external transmission links, shortening the message transmission path, and improving the stability and data transmission efficiency of out-of-band communication. Attached Figure Description
[0050] Figure 1This is a general structural diagram of the out-of-band communication device of the PCIe Switch;
[0051] Figure 2 This is a diagram of the out-of-band communication process in Example 1. Detailed Implementation
[0052] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.
[0053] Definitions of abbreviations and key terms:
[0054] PCIe Switch: PCIe switch.
[0055] PCIe: Peripheral Component Interconnect Express, is a high-speed serial computer expansion bus standard, primarily used to expand the data throughput of computer system buses and improve device communication speed.
[0056] BMC: Baseboard Management Controller.
[0057] MCTP: Management Component Transport Protocol, is a message transmission protocol used for communication between various management components within a platform such as servers and data center equipment. It is independent of the underlying physical bus and can run on various physical media, such as I2C (Inter-Integrated Circuit Bus), PCIe, and USB (Universal Serial Bus).
[0058] SMBus: System Management Bus.
[0059] PCB: Printed Circuit Board.
[0060] MCTP payload: MCTP protocol data payload.
[0061] GPU: Graphics Processing Unit.
[0062] Example 1
[0063] This embodiment uses an out-of-band temperature acquisition scenario in a server system as an example to explain the specific implementation process of the present invention in detail. In this embodiment, the management unit is a BMC (supporting MCTP over PCIe), and the downstream component device connected to the PCIe Switch is a temperature sensor (supporting only MCTP over SMBus). The BMC (out-of-band management controller) acquires the temperature data from the temperature sensor through the out-of-band communication device of the PCIeSwitch of the present invention, controls the fan speed based on the temperature data, and executes a heat dissipation strategy.
[0064] The out-of-band communication device of the PCIe Switch includes:
[0065] PCIe Switch: Equipped with 1 upstream port (e.g., Figure 1 The module consists of an "upstream PCIe interface module" and two downstream ports (downstream port 1 and downstream port 2). The upstream port is connected to the BMC (out-of-band management controller) via a PCIe bus (PCIe 5.0 bus), and downstream port 1... Figure 1 The "downstream SMBus interface module" is connected to device 1 (temperature sensor) via the SMBus bus, and downstream port 2 ( Figure 1 The "downstream PCIe interface module" in the diagram connects to device 2 (e.g., a GPU or PCIe network card) via the PCIe bus. The overall structure diagram is as follows: Figure 1 As shown;
[0066] Built-in controller: Integrated inside the PCIe switch, responsible for message reception, parsing, routing, and routing table refresh;
[0067] MCTP bridge module: Integrated into the built-in controller, it uses hardware logic circuits to implement MCTP message format conversion, ensuring conversion efficiency.
[0068] according to Figure 1 It can be seen that the core function of the MCTP bridge module is to realize the conversion of MCTP message formats between two different media.
[0069] Dynamic routing table: Integrated into the storage unit of the built-in controller, it uses a linked list structure to store routing information and supports dynamic addition, deletion, modification and query operations on routing information.
[0070] Communication process as follows Figure 2 As shown, it includes the following steps:
[0071] Step 1: During system initialization, the built-in controller detects the BMC's connection to the upstream port via the PCIe hot-plug detection mechanism and detects that device 1 (temperature sensor) is connected to the downstream port 1 via the SMBus enumeration mechanism, generating an initial routing table. The contents of the routing table are shown in Table 1.
[0072] Table 1
[0073]
[0074] Step 2: The BMC needs to acquire temperature data from the temperature sensor and generate an MCTP Over PCIe temperature acquisition message. The message format is: [PCIe header][MCTP payload 1], where the PCIe header contains the source device identifier (BMC_001) and the target device identifier (Sensor_Temp_001), and MCTP payload 1 contains the acquisition command. This MCTP Over PCIe temperature acquisition message is sent to the upstream port of the PCIe Switch via the PCIe bus.
[0075] Step 3: The built-in controller of the PCIe Switch receives the MCTP Over PCIe temperature acquisition message from the upstream port and parses the target device identifier in the message as Sensor_Temp_001.
[0076] Step 4: The built-in controller queries the dynamic routing table to determine that the communication medium type of the target device (Sensor_Temp_001) is SMBus and the corresponding port is downstream port 1.
[0077] Step 5: Since the communication medium type (PCIe) of the received message is inconsistent with the communication medium type (SMBus) of the target device, the built-in controller calls the MCTP bridge module to perform message format conversion: strip the PCIe transmission header from the message, extract MCTP payload 1, add an SMBus transmission header (containing the target device identifier Sensor_Temp_001), and generate an MCTP OverSMBus temperature acquisition message.
[0078] Step 6: The built-in controller sends the converted MCTP Over SMBus temperature acquisition message to the temperature sensor through downstream port 1.
[0079] Step 7: The temperature sensor receives the MCTP Over SMBus temperature acquisition message, parses the acquisition command, acquires its own temperature data (e.g., 38℃), and generates an MCTP Over SMBus response message. The message format is: [SMBus header][MCTP payload 2], where the SMBus header contains the source device identifier (Sensor_Temp_001) and the target device identifier (BMC_001), and MCTP payload 2 contains the temperature data. This MCTP Over SMBus response message is sent to downstream port 1 of the PCIe Switch via SMBus.
[0080] Step 8: The built-in controller receives the MCTP Over SMBus response message from downstream port 1 and parses the target device identifier in the message as BMC_001.
[0081] Step 9: The built-in controller queries the dynamic routing table to determine that the communication medium type of the target device (BMC_001) is PCIe and the corresponding port is the upstream port.
[0082] Step 10: Since the communication medium type (SMBus) of the received message is inconsistent with the communication medium type (PCIe) of the target device, the built-in controller calls the MCTP bridge module to perform message format conversion: strip the SMBus transmission header from the message, extract MCTP payload 2, add a PCIe transmission header (containing the target device identifier BMC_001), and generate an MCTP OverPCIe response message.
[0083] Step 11: The built-in controller sends the converted MCTP Over PCIe response message to the BMC through the upstream port.
[0084] Step 12: The BMC receives the MCTP Over PCIe response message, parses it to obtain the temperature data from the temperature sensor, and completes one out-of-band temperature acquisition.
[0085] If a new SMBus-enabled component device is subsequently connected to downstream port 2, the built-in controller detects the device through the SMBus enumeration mechanism and automatically adds the device identifier, communication medium type (SMBus), and downstream port 2 information to the dynamic routing table. If the temperature sensor is removed, the built-in controller deletes the corresponding entry in the dynamic routing table to ensure the accuracy of the routing.
[0086] The scope of protection of this invention is not limited to the specific embodiments described above. Those skilled in the art can make adaptive adjustments to the apparatus and method of this invention according to actual application scenarios. For example, the number of upstream and downstream ports of the PCIe Switch can be flexibly set according to the system equipment deployment requirements. The downstream port can simultaneously connect multiple SMBus-supporting component devices and multiple PCIe-supporting component devices. The MCTP bridge module can also be implemented using a combination of software logic and hardware circuits. The dynamic routing table can use other data structures such as arrays and hash tables to store routing information. As long as it does not deviate from the core technical solution of this invention, it falls within the scope of protection of this invention.
[0087] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.
Claims
1. An out-of-band communication device for a PCIe switch, characterized in that, include: PCIe Switch, built-in controller, MCTP bridge module, and dynamic routing table; The PCIe Switch has an upstream port and multiple downstream ports. The upstream port is connected to the management unit via the PCIe bus and is used to transmit MCTP Over PCIe messages. At least one of the downstream ports is connected to the component device via the SMBus bus and is used to transmit MCTP Over SMBus messages. The built-in controller is integrated inside the PCIe Switch body, and the MCTP bridge module and dynamic routing table are both integrated into the built-in controller; The MCTP bridge module is used to convert the message format between MCTP Over PCIe messages and MCTP Over SMBus messages. The dynamic routing table is used to store routing information, which includes the identification information of the management unit and each component device, as well as the corresponding communication medium type and port information. The dynamic routing table supports dynamically refreshing the routing information according to the device access / removal status. The built-in controller is used to receive MCTP messages from upstream or downstream ports, determine the target device and corresponding communication medium of the message by querying the dynamic routing table, call the MCTP bridge module to complete the message format conversion, and route the converted message to the target port. If the communication medium of the message is consistent with the communication medium of the target device, the message is directly routed to the target port.
2. The out-of-band communication device for a PCIe switch according to claim 1, characterized in that, The methods for refreshing the dynamic routing table include: When a component device accesses / removes a downstream port via SMBus, the built-in controller detects changes in device status through the device enumeration mechanism of SMBus and updates the device identifier, port information, and communication medium type in the dynamic routing table based on the detection results. When the management unit connects to / removes an upstream port, the built-in controller updates the corresponding information in the dynamic routing table through the PCIe hot-plug detection mechanism.
3. The out-of-band communication device for a PCIe switch according to claim 1, characterized in that, The message format conversion process of the MCTP bridge module includes: When an MCTP Over PCIe message is received from the management unit, the target device identifier in the message is parsed. If the target device is a component device that only supports SMBus, the PCIe header of the MCTP Over PCIe message is stripped, an SMBus header is added, and it is converted into an MCTP Over SMBus message. When an MCTP Over SMBus message is received from a component device, the target device identifier in the message is parsed. If the target device is a management unit, the SMBus header of the MCTP Over SMBus message is stripped, a PCIe header is added, and it is converted into an MCTP Over PCIe message.
4. The out-of-band communication device for a PCIe switch according to claim 1, characterized in that, The management unit is BMC.
5. An out-of-band communication method for a PCIe Switch, applied to the out-of-band communication device of a PCIe Switch as described in any one of claims 1-4, characterized in that, Includes the following steps: S1: Initialize the dynamic routing table in the built-in controller, obtain information about the management unit and each component device of the currently connected PCIe Switch through the device enumeration mechanism, and generate the initial routing table; S2: The built-in controller monitors the device access / removal status of each port in real time. When a status change is detected, the routing information in the dynamic routing table is dynamically refreshed. S3: The built-in controller receives MCTP messages from the upstream or downstream port; S4: Parse the received MCTP message and extract the target device identifier from the message; S5: Query the dynamic routing table to determine the communication medium type and target port corresponding to the target device based on the target device identifier; S6: Determine whether the communication medium type of the received message is consistent with the communication medium type of the target device; if they are consistent, directly route the received message to the target port; if they are inconsistent, call the MCTP bridge module to complete the message format conversion of the received message, and then route the converted message to the target port.
6. The out-of-band communication method for a PCIe Switch according to claim 5, characterized in that: In step S2, the process of dynamically refreshing the routing information in the dynamic routing table includes: If a new component device is detected to access a downstream port via SMBus, the identification information of the device is obtained through SMBus enumeration, and the device identification, communication medium type, and corresponding downstream port information are added to the dynamic routing table. If a component device is detected to have been removed from a downstream port of SMBus, the routing information corresponding to that device is deleted from the dynamic routing table. If the management unit is detected to be accessing / removing an upstream port, the identification information, communication medium type, and upstream port information of the management unit in the dynamic routing table will be updated accordingly.
7. The out-of-band communication method for a PCIe Switch according to claim 5, characterized in that: In step S6, the specific process of message format conversion includes: When the received message is an MCTP Over PCIe message and the target device is a component device that supports SMBus, the MCTP bridge module strips the PCIe header, extracts the MCTP payload, adds an SMBus header, and generates an MCTP Over SMBus message. When the received message is an MCTP Over SMBus message and the target device is a management unit, the MCTP bridge module strips the SMBus header, extracts the MCTP payload, adds a PCIe header, and generates an MCTP Over PCIe message.
8. The out-of-band communication method for a PCIe switch according to claim 5, characterized in that: In step S6, if the target device is a broadcast address or a multicast address, the built-in controller queries all matching downstream devices according to the dynamic routing table, and after copying and converting the message, routes it to the corresponding port.
9. A PCIe Switch out-of-band communication system, characterized in that, include: Management unit, supports MCTP over PCIe communication; At least one component device supports MCTP Over SMBus communication; And an out-of-band communication device for a PCIe Switch as described in any one of claims 1-4, connected between the management unit and the component device, for implementing MCTP message interaction between the two.
10. A server, characterized in that, Includes the PCIe Switch out-of-band communication system as described in claim 9.