MCTP networking method, system and device crossing multiple PCIe RC domains and storage medium

Abstract

The invention discloses an MCTP networking method, system and device crossing multiple PCIe RC domains and a storage medium. The method comprises the steps that in the initialization stage, a unique ID is distributed to each PCIe RC controller, an MCTP routing table stored in a storage device is constructed through a broadcast discovery mechanism, and the routing table maps an MCTP endpoint identifier EID to a unique identifier composed of an RCID and a BDF; in the working stage, the MCTP protocol packet is automatically received to the storage device through the hardware, the software analyzes the target EID and queries the routing table to obtain target routing information and modify the packet header of the protocol packet, and then the hardware sends the modified protocol packet to the target PCIe RC controller. Through software and hardware cooperation and centralized routing table management, efficient and flexible cross-RC-domain MCTP networking is achieved, the problems that a pure hardware routing scheme is large in resource consumption and difficult in time sequence convergence are effectively avoided, and the method has the advantages of being small in resource occupation, high in reliability and easy to achieve.

Description

Technical Field

[0001] This invention belongs to the field of chip design technology, and particularly relates to a method, system, device and storage medium for MCTP networking across multiple PCIe RC domains. Background Technology

[0002] MCTP (Management Component Transport Protocol) is a protocol developed by the DMTF (Distributed Management Task Force) to support communication between platform management subsystems. MCTP is primarily used to provide a unified and flexible management communication mechanism in servers, data centers, and embedded devices.

[0003] MCTP over PCIe VDM (Management Component Transport Protocol over PCI Express) is an implementation of the MCTP protocol over the PCIe (Peripheral Component Interconnect Express) bus. PCIe VDM (Vendor Defined Message) is a type of message (Posted) in the PCIe protocol, allowing device manufacturers to implement custom functions and communications beyond the standard PCIe protocol. It is used for proprietary or exclusive feature extensions for specific hardware vendors and belongs to MessageTLP (Transaction Layer Packet). When MCTP runs on PCIe, it utilizes PCIe VDM as the transport mechanism. The basic process is as follows: 1) VDM messages carry MCTP data: MCTP data packets are encapsulated in PCIe VDM and transmitted over the PCIe link. This allows devices to transmit management information using standard PCIe paths without requiring additional hardware interfaces.

[0004] 2) Packet Structure: The format of MCTP over PCIe VDM messages follows the MCTP protocol standard and is encapsulated in the payload field of PCIe VDM. PCIe VDM provides the transport carrier, while the MCTP protocol defines the higher-level message structure and content.

[0005] 3) Endpoint Device Identification: Each device in PCIe has a unique ID (Bus / Device / FunctionNumber), so communication between devices can be distinguished by these IDs. In a PCIe link, devices can address and transmit management information by sending VDMs.

[0006] A PCIe RC (Root Complex Domain) is a key concept in the PCIe architecture, referring to a logical topology area comprised of a single root complex (RC) and all downstream PCIe devices. In MCTP (Management Component Transport Protocol), an EID (Endpoint ID) is a logical address used to uniquely identify a communication endpoint in an MCTP network. Its core function is to provide routing and addressing capabilities for MCTP messages, similar to an IP address in an IP network.

[0007] Message passing and distribution: The PCIe controller passes MCTP messages as VDM messages to the target device. Vendor-defined fields included in the VDM can be used to specify message type and routing information.

[0008] However, within the same PCIe network, MCTP's EID can be mapped one-to-one with the PCIe network ID (BDF). But when a chip contains multiple PCIe RC controllers, MCTP networking requires consideration of different RC domains. PCIe devices with the same BDF number represent different MCTP devices. The MCTP protocol itself does not consider this situation; that is, the MCTP protocol only considers the networking situation of a single PCIe network. When networking across multiple PCIe networks, it is necessary to consider the one-to-one mapping between EID and each PCIe device in the MCTP network.

[0009] BDF (Bus, Device, Function) is a unique address identifier for a device on the PCIe bus. It consists of the bus number (Bus), device number (Device), and function number (Function), typically represented as BB:DD.F (e.g., 00:1F.3). The operating system and drivers use it to locate a specific device.

[0010] To address the MCTP networking challenge across RC domains, the industry typically employs an on-chip networking approach, where multiple RCs are networked within the chip. This involves building a PCIe ID routing bridge within the chip, enabling MCTP protocol packets to be routed based on their IDs within the chip, ultimately facilitating cross-RC domain communication. However, this approach has significant drawbacks: firstly, the hardware routing bridge consumes substantial on-chip logic and wiring resources, increasing chip area and cost; secondly, since multiple PCIe controllers may be physically located far apart, long-distance signal wiring can lead to timing convergence difficulties, impacting system performance and stability. Summary of the Invention

[0011] The purpose of this invention is to address the aforementioned technical problems in the prior art and to provide a method, system, device, and storage medium for MCTP networking across multiple PCIe RC domains. This aims to solve the problems of high resource consumption and difficulty in timing convergence caused by the use of hardware routing bridges in the prior art. This invention utilizes a hardware-software co-operation approach, leveraging routing tables in storage devices to achieve MCTP protocol packet routing across RC domains, offering advantages such as flexible implementation, low resource consumption, and timing friendliness.

[0012] To solve the above-mentioned technical problems, the present invention adopts the following technical solution: A method for MCTP networking across multiple PCIe RC domains includes the following steps: Step 1: MCTP Network Initialization Phase 1) Assign a unique RC identifier ID to each PCIe RC controller in the SoC chip; 2) Establish an MCTP routing table in the storage device. The MCTP routing table is used to map the MCTP endpoint identifier EID to a unique identifier composed of the PCIe root composite identifier RC_ID and the PCIe network identifier BDF. Step 2, MCTP network normal operation phase 1) Receive MCTP protocol packets via hardware and store them in a storage device; 2) The target EID of the MCTP protocol packet is parsed using software, and the MCTP routing table is queried to obtain the target RC_ID and target BDF corresponding to the target EID; 3) Modify the header information of the MCTP protocol packet based on the target EID using software; 4) The modified MCTP protocol packet is sent to the target PCIe RC controller corresponding to the target RC_ID via hardware, so that the PCIe RC controller can send the MCTP protocol packet to the MCTP target device under the target PCIe network.

[0013] Furthermore, the detailed steps for establishing the MCTP routing table in the storage device in step one are as follows: a. The SoC chip sends MCTP protocol broadcast commands to each PCIe RC controller and waits for MCTP devices in each PCIe network to respond to the broadcast commands; b. After receiving the broadcast command from the PCIe RC controller, the MCTP device sends a MCTP protocol packet in response to the broadcast to the SoC chip. After receiving the packet, the SoC chip stores the MCTP protocol packet in its storage device. c. Parse each MCTP response packet to obtain the EID of the MCTP device that sent the response packet and the RC_ID and BDF of its PCIe RC domain; d. Based on the parsing results, construct the mapping relationship between EID, RC_ID and BDF, and form an MCTP routing table stored in the storage device.

[0014] Furthermore, the detailed steps for hardware receiving MCTP protocol packets in step two are as follows: a. The software pre-sets storage information in the storage device for storing the storage area to be received MCTP protocol packets; b. The MCTP controller loads the storage information. After loading, the MCTP controller waits for the MCTP protocol packet to be input into the SoC chip. After input, the MCTP controller stores the received MCTP protocol packet into the corresponding storage area according to the storage information and notifies the software to process it.

[0015] Furthermore, the detailed steps for sending the modified MCTP protocol packet via hardware in step two are as follows: a. Based on the storage address and packet size of the modified MCTP protocol packet in the storage device, the software generates MCTP packet sending information that the MCTP controller can read; b. Read the MCTP packet transmission information through the MCTP controller, and read the modified MCTP protocol packet from the storage device according to the MCTP packet transmission information; c. The MCTP controller sends the read MCTP protocol packets to the PCIe RC controller corresponding to the target PCIe network.

[0016] A multi-PCIe RC domain MCTP networking system, characterized in that: it includes... Storage devices are used to store MCTP protocol packets and MCTP routing tables; At least one PCIe RC controller, each PCIe RC controller is used to connect to and control a PCIe network; The MCTP controller communicates with the storage device and the PCIe RC controller to receive and send MCTP protocol packets. The CPU, connected to the storage device and MCTP controller, is configured to run software drivers.

[0017] Furthermore, the system is integrated into a SoC chip, which has a bus for connecting storage devices, PCIe RC controllers, MCTP controllers, and the CPU.

[0018] Furthermore, the storage device is located inside the SoC chip and is either a register or static random access memory (SRAM).

[0019] Furthermore, the storage device is located outside the SoC chip and is either Double Data Rate (DDR) memory or Flash memory.

[0020] An electronic device comprising the aforementioned MCTP networking system spanning multiple PCIe RC domains.

[0021] A computer-readable storage medium storing a computer program thereon, characterized in that: when the computer program is executed by a CPU, it implements the above-described MCTP networking method across multiple PCIe RC domains.

[0022] The present invention, by adopting the above-described technical solution, has the following beneficial effects: This invention utilizes a routing table in a storage device to route MCTP protocol packets across RC domains through a hardware and software co-design, avoiding complex on-chip hardware routing bridge design and significantly saving on-chip logic and wiring resources.

[0023] This invention uses software to dynamically build and maintain routing tables, enabling the system to flexibly adapt to changes in device topology under different RC domains, enhancing the flexibility and scalability of networking, while avoiding the timing convergence problem caused by long-distance cabling. Attached Figure Description

[0024] The present invention will be further described below with reference to the accompanying drawings: Figure 1 This is a schematic diagram of an MCTP networking system spanning multiple PCIe RC domains according to the present invention; Figure 2 This is a flowchart of an MCTP networking method across multiple PCIe RC domains according to the present invention; Figure 3 This is a flowchart of the MCTP network initialization phase in this invention. Detailed Implementation

[0025] like Figures 1 to 2 As shown, this is the present invention. A method for MCTP networking across multiple PCIe RC domains includes the following steps: Step 1: MCTP Network Initialization Phase 1) Assign a unique RC identifier ID to each PCIe RC controller in the SoC chip.

[0026] 2) Establish an MCTP routing table in the storage device. The MCTP routing table is used to map the MCTP endpoint identifier EID to a unique identifier composed of the PCIe root composite identifier RC_ID and the PCIe network identifier BDF.

[0027] a. The SoC chip sends MCTP protocol broadcast commands to each PCIe RC controller and waits for MCTP devices in each PCIe network to respond to the broadcast commands.

[0028] b. After receiving the broadcast command from the PCIe RC controller, the MCTP device sends a MCTP protocol packet in response to the broadcast to the SoC chip. After receiving the packet, the SoC chip stores the MCTP protocol packet in its storage device.

[0029] c. Parse each MCTP response packet to obtain the EID of the MCTP device that sent the response packet and the RC_ID and BDF of its PCIe RC domain.

[0030] d. Based on the parsing results, construct the mapping relationship between EID, RC_ID and BDF, and form an MCTP routing table stored in the storage device.

[0031] Before deploying MCTP networks across multiple PCIe RC domains, an MCTP network initialization process is required, as detailed below. Figure 2 As shown.

[0032] The SoC chip first performs a reset procedure; then, according to the PCIe protocol specification, it waits for the PCIe networks under each PCIe RC controller to complete the connection establishment; next, the firmware running on the SoC chip sends a broadcast command of the MCTP protocol to each PCIe RC controller, the format of which is defined by the MCTP protocol specification; then, the SoC chip waits for the MCTP devices in each PCIe network to respond to the previous broadcast command. After receiving the broadcast command sent by the RC controller, the MCTP device sends a response MCTP protocol packet; the SoC chip saves the response MCTP protocol packet in the storage device, and then the software analyzes the routing information of the response packet, that is, the PCIe Network ID information (BDF) in the MCTP protocol packet header. The software will build a routing table corresponding one-to-one with the MCTP network EID based on the RC ID corresponding to the PCIe network and the ID information parsed from the packet header, and store the routing table in the storage device. After the routing table is built, the MCTP network initialization is complete.

[0033] Table 1 shows an example of an MCTP routing table. Table 1. Example of MCTP routing Step 2, MCTP Network Normal Operation Phase 1) Receive MCTP protocol packets through hardware and store the MCTP protocol packets in the storage device.

[0034] a. The software pre-sets storage information in the storage device for storing the storage area to be received MCTP protocol packets.

[0035] b. The MCTP controller loads the storage information. After loading, the MCTP controller waits for the MCTP protocol packet to be input into the SoC chip. After input, the MCTP controller stores the received MCTP protocol packet into the corresponding storage area according to the storage information and notifies the software to process it.

[0036] 2) The target EID of the MCTP protocol packet is parsed by software, and the MCTP routing table is queried to obtain the target RC_ID and target BDF corresponding to the target EID.

[0037] 3) Modify the header information of the MCTP protocol packet according to the target EID using software.

[0038] 4) The modified MCTP protocol packet is sent to the target PCIe RC controller corresponding to the target RC_ID via hardware, so that the PCIe RC controller can send the MCTP protocol packet to the MCTP target device under the target PCIe network.

[0039] a. Based on the storage address and packet size of the modified MCTP protocol packet in the storage device, the software generates MCTP packet sending information that the MCTP controller can read; b. Read the MCTP packet transmission information through the MCTP controller, and read the modified MCTP protocol packet from the storage device according to the MCTP packet transmission information; c. The MCTP controller sends the read MCTP protocol packets to the PCIe RC controller corresponding to the target PCIe network.

[0040] After the MCTP network is initialized as described above, the MCTP network is completed and then enters the normal operation phase.

[0041] MCTP protocol packets will be routed through the SoC chip and propagated in multiple PCIe networks. The process of the SoC chip forwarding MCTP protocol packets is as follows: Figure 1 As shown.

[0042] After the software completes the routing table setup, the MCTP network initialization is complete. Then, the software sets the storage information for MCTP packets to be received, i.e., the address where the packets to be received will be stored. After the software completes the setup of a series of storage information, it triggers the hardware (i.e., the MCTP controller) to start receiving MCTP protocol packets.

[0043] The MCTP controller will automatically load the stored information until all stored information has been loaded. After loading, the MCTP controller will wait for MCTP protocol packets to be input to the SoC chip. When MCTP traffic is input to the SoC chip, the MCTP controller will store the MCTP protocol packets in the correct storage area according to the previously loaded stored information and notify the software for further processing.

[0044] After receiving a notification from the MCTP controller, the software will read the information of the received MCTP protocol packets from the memory, parse the EID and PCIe Network ID (BDF) of the protocol packet, and then search the routing table. Based on the MCTP network ID information of the target device in the MCTP protocol packet, the software will extract the PCIe RC ID information of the target device and the PCIe Network ID (BDF) under that PCIe network.

[0045] Then modify the header of the MCTP protocol packet, changing the PCIe network ID in the header to the ID (BDF) of the PCIe network where the target device is located.

[0046] After the software modifies the MCTP protocol packet, it begins the MCTP protocol packet sending process, ultimately enabling the MCTP protocol packet stored on the storage device to be received by the MCTP target device.

[0047] The software will first generate MCTP packet sending information that the MCTP controller can read, based on information such as the storage address and packet size of the MCTP protocol packet to be sent.

[0048] This triggers the MCTP controller to begin the transmission process. The MCTP controller first reads the MCTP packet transmission information configured in the software, then reads the MCTP protocol packet on the storage device based on this information, and sends the protocol packet to the PCIe RC controller corresponding to the target PCIe network. The PCIe RC controller will send the MCTP protocol packet to the MCTP target device according to the PCIe protocol specification. Thus, the MCTP protocol packet sent by the MCTP device under a certain PCIe network is successfully sent to the MCTP target device under the target PCIe network.

[0049] In this invention, the above steps are divided into an MCTP network initialization phase and an MCTP network normal operation phase. During the initialization phase, an automated broadcast discovery and response mechanism enables the system to dynamically identify and register all MCTP devices within the RC domain, constructing a globally unified routing view. This avoids the cumbersome manual configuration of device information required in traditional solutions, improving networking efficiency and accuracy, and laying a solid foundation for subsequent cross-domain communication. During the normal operation phase, a refined division of labor between hardware and software enables efficient and accurate forwarding of MCTP traffic across RC domains. The hardware handles high-frequency, real-time packet transmission, reception, and storage operations, ensuring high throughput and low latency in data processing; the software handles complex but highly flexible routing query and packet modification logic, jointly ensuring the reliability of cross-domain communication and achieving an optimal balance between performance and flexibility.

[0050] like Figure 3 As shown, an MCTP networking system spanning multiple PCIe RC domains includes... The storage device is used to store MCTP protocol packets and MCTP routing tables. In this invention, the storage device is not limited to static random access memory (SRAM), registers, or other media inside the SoC chip, but can also be double data rate memory (DDR), flash memory, or other media outside the SoC chip. Any storage medium that can be accessed by both the CPU and the MCTP controller at the same time is acceptable.

[0051] At least one PCIe RC controller, each PCIe RC controller is used to connect to and control a PCIe network, receive MCTP protocol packets sent by the downstream PCIe network into the SoC chip, and send MCTP protocol packets sent from inside the SoC chip to the downstream PCIe network.

[0052] The MCTP controller, controlled by the CPU, communicates with the storage device and the PCIe RC controller. It is responsible for forwarding MCTP protocol packets received by the PCIe RC controller to the storage device, and sending MCTP protocol packets to be sent out to the PCIe RC controller.

[0053] The CPU, connected to the storage device and MCTP controller, is configured to run software drivers.

[0054] The bus is a general-purpose device inside the SoC chip, responsible for interconnecting the components in the SoC chip, namely the connection between the aforementioned storage devices, PCIe RC controller, MCTP controller and CPU, and the aforementioned components are integrated into the SoC chip.

[0055] In this invention, the PCIe network includes PCIe network links, PCIe switches, MCTP devices, etc. Each PCIe network has a corresponding PCIe RC controller, and the MCTP device refers to a PCIe device in the PCIe network that supports the MCTP protocol.

[0056] The routing method of MCTP protocol packets in a single PCIe network is determined by the PCIe protocol specification. However, routing between multiple PCIe networks cannot be performed using the routing method specified by the PCIe protocol specification, and the SoC chip is needed to forward the protocol packets.

[0057] The present invention also discloses an electronic device, including the above-mentioned MCTP networking system spanning multiple PCIe RC domains.

[0058] And a computer-readable storage medium storing a computer program that, when executed by a CPU, implements the above-described MCTP networking method across multiple PCIe RC domains.

[0059] The abbreviations and Chinese translations of the terms used in this invention are shown in the table below; English abbreviations Chinese MCTP Management Component Transport Protocol SoC System on Chip (SoC) – A chip that integrates key components of a computer or other electronic system (such as CPU, GPU, memory controller, I / O interface, DSP, etc.) onto a single chip. Commonly found in mobile phones, tablets, embedded devices, and modern PC processors. PCIe Peripheral Component Interconnect Express (PCI) is a high-speed serial computer expansion bus standard used to connect high-speed devices on the motherboard (such as graphics cards, SSDs, network cards, etc.). It replaced the older PCI and AGP standards. BDF Bus, Device, Function - A unique address identifier for a device on the PCIe bus. It consists of the bus number (Bus), device number (Device), and function number (Function), typically represented as BB:DD.F (e.g., 00:1F.3). The operating system and drivers use it to locate a specific device. RC Root Complex - The top component of the PCIe hierarchy. It connects the CPU / memory subsystem (SoC) to the PCIe switch and endpoint devices (such as graphics cards and SSDs). It is equivalent to the "root" of the PCIe tree. SRAM Static Random-Access Memory (SRAM) is a high-speed, low-latency volatile memory (data is lost when power is off). It retains data without requiring refresh circuitry (as long as power is on), and is faster than DRAM but more expensive and has lower density. It is commonly used in CPU caches, register files, and buffer memories. VDM Vendor-Defined Message (TLP) - A PCIe packet type that allows device manufacturers to implement custom functionality and communication outside of the standard PCIe protocol. Used for proprietary or exclusive feature extensions for specific hardware vendors. Table 2. Glossary of Terms and Their Chinese Translations The above are merely specific embodiments of the present invention, but the technical features of the present invention are not limited thereto. Any simple changes, equivalent substitutions, or modifications made based on the present invention to solve essentially the same technical problems and achieve essentially the same technical effects are all covered within the protection scope of the present invention.

Claims

1. A method for MCTP networking across multiple PCIe RC domains, characterized in that... Includes the following steps: Step 1, MCTP Network Initialization Phase 1) Assign a unique RC identifier ID to each PCIe RC controller in the SoC chip; 2) Establish an MCTP routing table in the storage device. The MCTP routing table is used to map the MCTP endpoint identifier (EID) to a unique identifier composed of the PCIe root composite identifier (RC_ID) and the PCIe network identifier (BDF). Step 2, MCTP network normal operation phase: 1) Receive MCTP protocol packets through hardware and store the MCTP protocol packets in the storage device; 2) Parse the target EID of the MCTP protocol packet through software and query the MCTP routing table to obtain the target RC_ID and target BDF corresponding to the target EID; 3) Modify the packet header information of the MCTP protocol packet according to the target EID through software; 4) Send the modified MCTP protocol packet to the target PCIe RC controller corresponding to the target RC_ID through hardware, so that the PCIe RC controller can send the MCTP protocol packet to the MCTP target device under the target PCIe network.

2. The MCTP networking method across multiple PCIe RC domains according to claim 1, characterized in that: In step one, the detailed steps for establishing the MCTP routing table in the storage device are as follows: a) The SoC chip sends an MCTP protocol broadcast command to each PCIe RC controller and waits for the MCTP device in each PCIe network to respond to the broadcast command; b. After receiving the broadcast command from the PCIe RC controller, the MCTP device sends a response MCTP protocol packet to the SoC chip. The SoC chip receives the response MCTP protocol packet and stores it in the storage device. c. Parse each MCTP response packet to obtain the EID of the MCTP device that sent the response packet and the RC_ID and BDF of its PCIe RC domain. d. Based on the parsing results, construct the mapping relationship between the EID and the RC_ID and BDF, and form the MCTP routing table, which is stored in the storage device.

3. The MCTP networking method across multiple PCIe RC domains according to claim 1, characterized in that: The detailed steps for hardware receiving MCTP protocol packets in step two are as follows: a) The software presets storage information in the storage device for storing the storage area to be received MCTP protocol packets; b) The MCTP controller loads the storage information. After loading, the MCTP controller waits for the MCTP protocol packets to be input into the SoC chip. After input, the MCTP controller stores the received MCTP protocol packets into the corresponding storage area according to the storage information and notifies the software to process them.

4. The MCTP networking method across multiple PCIe RC domains according to claim 1, characterized in that: The detailed steps for sending the modified MCTP protocol packet via hardware in step two are as follows: a) The software generates MCTP packet sending information that the MCTP controller can read, based on the storage address and packet size of the modified MCTP protocol packet in the storage device. b. Read the MCTP packet transmission information through the MCTP controller, and read the modified MCTP protocol packet from the storage device according to the MCTP packet transmission information; c. The MCTP controller sends the read MCTP protocol packets to the PCIe RC controller corresponding to the target PCIe network.

5. An MCTP networking system spanning multiple PCIe RC domains, characterized in that: Includes storage devices for storing MCTP protocol packets and MCTP routing tables; At least one PCIe RC controller, each of which is used to connect to and control a PCIe network; An MCTP controller, communicatively connected to the storage device and the PCIe RC controller, is used to receive and send MCTP protocol packets; The CPU is connected to the storage device and the MCTP controller and is configured to run software drivers.

6. The MCTP networking system across multiple PCIe RC domains according to claim 5, characterized in that: The system is integrated into a SoC chip, which has a bus for connecting the storage device, the PCIe RC controller, the MCTP controller, and the CPU.

7. The MCTP networking system across multiple PCIe RC domains according to claim 6, characterized in that: The storage device is located inside the SoC chip and is either a register or a static random access memory (SRAM).

8. The MCTP networking system across multiple PCIe RC domains according to claim 6, characterized in that: The storage device is located outside the SoC chip and is either Double Data Rate (DDR) memory or Flash memory.

9. An electronic device, characterized in that: This includes the MCTP networking system spanning multiple PCIe RC domains as described in any one of claims 5-8.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by the CPU, it implements the MCTP networking method across multiple PCIe RC domains as described in any one of claims 1-4.

Images