PCIe device bandwidth management device, method and server

Abstract

This application discloses a PCIe device bandwidth management device, method, and server, relating to the field of server technology. These methods utilize communication circuits to chain control modules, CPUs, and PCIe devices, reducing redundant connection lines in traditional star topologies, lowering hardware design complexity and cost, and improving system integration and reliability. The control module uses the communication circuits to uniformly set the PCIe device data transmission channels to a first level, rapidly scans and identifies the channel status of each CPU root node, and determines a bandwidth setting data table. This allows for dynamic adaptation to the bandwidth requirements of different devices, avoiding resource waste or conflicts.

Description

Technical Field

[0001] The present application relates to the field of server technology, and in particular to a PCIe device bandwidth management device, method, and server. Background Art

[0002] In modern server systems, motherboards are typically equipped with PCIe x8 and PCIe x16 slots, providing users with flexible expansion capabilities. This design allows for the insertion of expansion cards of varying specifications, greatly improving the compatibility and scalability of hardware configurations.

[0003] However, a significant technical limitation of current server architectures is the CPU's fixed maximum bandwidth allocation mechanism for PCIe x8 and x16 slots. When expansion cards with a small number of lanes are inserted, the system fails to effectively identify and dynamically adapt bandwidth, and resources are still allocated based on the slot's maximum bandwidth. This results in low PCIe resource utilization, increases the CPU's resource scheduling burden, and causes performance loss. Summary of the Invention

[0004] The present application provides a PCIe device bandwidth management apparatus, method, and server to at least solve the problem of how to accurately identify and allocate PCIe device bandwidth in the related art.

[0005] The present application provides a PCIe device bandwidth management device, comprising: a communication circuit, a control module, and a basic input / output system, wherein the communication circuit is used to connect the control module, a CPU, and a PCIe device in series; the control module is used to set the data transmission channel of each PCIe device to a first level through the communication circuit, and determine a bandwidth setting data table based on the levels of all data transmission channels of the PCIe root node of each CPU; and the basic input / output system is used to allocate bandwidth to each PCIe device based on the bandwidth setting data table.

[0006] The present application also provides a PCIe device bandwidth management method, which is applied to the control module of the PCIe device bandwidth management device of the first aspect and any optional embodiment thereof, and the method includes: using a communication circuit to configure both the CPU and the PCIe device to enter a test mode; setting the data transmission channel of each PCIe device to a first level through the communication circuit; and determining a bandwidth setting data table based on the levels of all data transmission channels of the PCIe root node of each CPU, where the bandwidth setting data table is used to determine the PCIe device bandwidth.

[0007] The present application also provides a server, comprising: the above PCIe device bandwidth management device.

[0008] The present application also provides an electronic device, comprising: a memory for storing a computer program; and a processor for implementing the steps of any of the above-mentioned PCIe device bandwidth management methods when executing the computer program.

[0009] The present application also provides a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, the steps of any of the above-mentioned PCIe device bandwidth management methods are implemented.

[0010] The present application also provides a computer program product, including a computer program, which implements the steps of any of the above-mentioned PCIe device bandwidth management methods when executed by a processor.

[0011] Through this application, the control module, CPU and PCIe devices are chained together using communication circuits, which reduces the redundant connection lines in the traditional star topology, reduces the complexity and cost of hardware design, and improves the system integration and reliability. At the same time, when adding or removing PCIe devices, only the connection nodes on the link need to be adjusted, without the need for large-scale changes to the system architecture, which facilitates hardware upgrades and maintenance.

[0012] This application uniformly sets the PCIe device data transmission channel to the first level through the communication circuit, quickly scans and identifies the channel status of each CPU root node, determines the bandwidth setting data table, and dynamically adapts to the bandwidth requirements of different devices to avoid resource waste or conflicts. At the same time, it forces the unified level state to shield the electrical differences between different devices, ensure the consistency of data transmission protocols, and improve the overall compatibility and stability of the system.

[0013] Through this application, based on the bandwidth setting data table generated by the control module, the transmission resources of PCIe devices are accurately allocated, the performance requirements of high-load devices (such as GPUs and high-speed storage) are prioritized, and bottlenecks caused by bandwidth competition are avoided. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] In order to more clearly illustrate the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0015] Figure 1 A diagram showing the composition of a PCIe device bandwidth management device provided in an embodiment of the present application;

[0016] Figure 2 A structural diagram of a communication circuit provided in an embodiment of the present application;

[0017] Figure 3A specific circuit structure diagram of the switch circuit provided in an embodiment of the present application;

[0018] Figure 4 A structural diagram of a hot-swap circuit provided in an embodiment of the present application;

[0019] Figure 5 A flowchart of a PCIe device bandwidth management method provided in an embodiment of the present application;

[0020] Figure 6 A flowchart of another PCIe device bandwidth management method provided in an embodiment of the present application;

[0021] Figure 7 A diagram illustrating the composition of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0022] The following will be combined with the accompanying drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0023] It should be noted that, in the description of this application, the terms "comprises," "includes," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. The terms "first," "second," etc., in this application are used to distinguish similar objects, and are not used to describe a particular order or sequence.

[0024] In order to enable those skilled in the art to better understand the present application, the present application is further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0025] In conjunction with the specific application environment architecture or specific hardware architecture on which the execution of the PCIe device bandwidth management method depends, the specific application environment architecture or specific hardware architecture is described herein.

[0026] In related technology, PCIe (Peripheral Component Interconnect Express) is a high-speed serial computer expansion card standard primarily used to connect CPUs and various expansion cards, such as graphics cards, sound cards, and network adapters. PCIe represents a fundamental shift in bus architecture, primarily in two key areas: the transition from a parallel bus to a serial bus; and the adoption of point-to-point interconnection. This shift transforms the bus, which connects devices below the bridge in the original parallel bus architecture, into a link. A link can consist of one or more lanes, each consisting of two pairs of differential signal lines forming a simplex serial transmission channel. There are no dedicated data, address, control, or clock lines, and transactions on the bus are organized into packets for transmission. Another breakthrough in PCIe's traditional bus architecture is its point-to-point interconnection approach. Each device is connected by an independent link with dedicated bandwidth, effectively increasing transmission rates. The PCIe bus architecture treats each link as a bus, maintaining compatibility with the traditional PCI bus in terms of address space, configuration mechanisms, and software. Each PCIe device occupies one bus (link), resulting in a high number of bridges and buses in PCIe-based computers. Traditional PCI devices can also run on the PCIe platform. A PCIe to PCI (or PCI-X) bridge can be used to bring out the PCI (or PCI-X) bus. Traditional PCI devices can be connected to this bus and integrated into the PCI Express structure. 11th Generation Intel Core CPUs and motherboards added support for PCIe 4.0, and now, the 12th Generation platform offers PCIe 5.0 and is fully backward compatible with 4.0 and 3.0 devices. 12th Generation Intel Core CPUs offer up to 16 CPU PCIe 5.0 lanes and up to 4 CPU PCIe 4.0 lanes, while 11th Generation Intel Core CPUs (such as the Intel Core i9-11900K) offer up to 20 CPU PCIe 4.0 lanes.

[0027] The bandwidth of PCIe is closely related to its version and the number of lanes used for data transmission. The following is a detailed description of the different versions of PCIe and their bandwidth:

[0028] PCIe 1.0: The line bit rate is 2.5Gb / s, using 8 / 10 encoding, so the bandwidth of PCIe 1.0x1 is 250MB / s. PCIe 2.0: The line bit rate is doubled to 5Gb / s, using 8 / 10 encoding, so the bandwidth of PCIe 2.0x1 is 500MB / s. PCIe 3.0: The line bit rate is 8Gb / s, using 128 / 130 encoding, so the bandwidth of PCIe 3.0x1 is 1GB / s. PCIe 4.0: The line bit rate is increased to 16GT / s. PCIe 5.0: The line bit rate can reach up to 32GT / s.

[0029] In addition, PCIe bandwidth is also related to the number of lanes. For example, the theoretical maximum bandwidth of a PCIe x1 slot is 1GB / s, which is suitable for devices with low data transmission requirements; the theoretical maximum bandwidth of a PCIe x4 slot is 4GB / s, which is suitable for devices with medium bandwidth requirements; the theoretical maximum bandwidth of a PCIe x8 slot is 8GB / s, which is suitable for devices with higher data transmission speed requirements; and the PCIe x16 slot has the highest theoretical maximum bandwidth of 16GB / s and is widely used to connect high-performance graphics cards.

[0030] The PICe interface on the server motherboard generally uses PCIe x8 slots and PCIe x16 slots to facilitate the subsequent insertion of different types of expansion cards. Even if the gold finger on the expansion card is x2 or x4, it can still be inserted into the PCIe x8 slot or PCIex16 slot. However, the bandwidth currently provided by the CPU directly to the PCIe x8 slot and PCIe x16 slot is the maximum bandwidth, resulting in ineffective recognition when expansion cards with a small number of lanes are inserted. The CPU also allocates the maximum bandwidth, resulting in a waste of PCIe resources and a loss of CPU function. For example, if two PCIE X8 devices are inserted into two PCIE X16 slots, it is impossible to support two X8 devices in one X16 slot; automatic recognition and bandwidth allocation of PCIe cannot be achieved.

[0031] Based on the above problems, the current methods used by PCIe to automatically identify and allocate bandwidth are: first, using the PCA9555GPIO expansion chip instead of GPIO to dynamically configure the bandwidth; second, using the resistance control current or voltage detection method on the physical detection circuit board to determine the required bandwidth.

[0032] Both of the above methods require adding a bandwidth allocation table to the non-matching detection program, extending chip detection or physical resistance detection. After obtaining the change data, it is necessary to match the values with the corresponding Lane number in the pre-set bandwidth allocation table, and then the CPU analyzes the corresponding bandwidth for allocation. In the early stage, it is necessary to count the data of different expansion cards and simulate the table. The types and models of corresponding expansion cards are fixed. When the expansion card data that is not in the table is plugged in, there will be recognition errors. The table needs to be matched when used, and the overall recognition and allocation of bandwidth are inefficient.

[0033] Bandwidth is also allocated based on the pin level of the PCIe device. Using PRSNT2# can only allocate the required bandwidth to a device when there is already a device in the X16 port. Alternatively, using PRSNT2# can only distinguish the bandwidth of X8 and X16. Both of these methods also result in incomplete and inaccurate allocation.

[0034] There are many server models and many customer customization requirements, which leads to the complexity of the diversity of server CPU PCIe port connection methods. The connection between the CPU PCIe port and the device and riser card on different motherboards is not the same. The PCIe port allocation within the riser card itself is also diverse. Each model and each customized shipping configuration BIOS needs to be individually developed and adapted based on the PCIe port topology of the motherboard and the PCIe port topology of the several riser cards paired with the motherboard. As a result, the development of server firmware for a specific configuration is very cumbersome and affects the efficiency of customized development and mass production of server products.

[0035] Based on this, the present application provides a PCIe device bandwidth management device, such as Figure 1 As shown, it includes: a communication circuit based on the Joint Test Action Group (JTAG) interface, a control module, and a basic input and output system.

[0036] The communication circuit is used to connect the control module, CPU and PCIe device in series.

[0037] The control module is used to set the data transmission channel of each PCIe device to the first level through the communication circuit, and to set the bandwidth data table based on the levels of all data transmission channels of the PCIe root node of each CPU.

[0038] Optionally, the control module is a baseboard management controller (BMC) in the server. A baseboard management controller (BMC) is a dedicated controller used to monitor and manage servers. In layman's terms, a BMC is an independent system under the host server system. This independent system has its own processor and memory. Even if the host hardware or operating system crashes or is shut down, the host system can still be managed by the BMC system. It is similar to a backdoor on the server and is used to manage the server. Generally, when a new server is put on the shelf, one way to do it is to modify the BMC-related network configuration, configure the IP address and gateway, connect the network cable through the out-of-band management port to achieve communication, and then use IPMI to achieve remote management of the server, such as console redirection (KVM) and other operations, and then perform remote management and system installation.

[0039] The BMC system mainly has the following functions:

[0040] (1) Equipment information management: Record detailed information about the server, including model, manufacturer, date, production and technical information of each component, chassis information, motherboard information, etc., as well as BMC information itself, such as server host name, IP address, BMC firmware version, etc.

[0041] (2) Server status monitoring and management: Detect the temperature, voltage, and other health status of each server component (such as CPU, memory, hard disk, fan, chassis, etc.). At the same time, adjust the fan speed in real time according to the conditions of each temperature collection point to ensure that the server does not overheat and control the overall power consumption within a reasonable range. If any abnormality occurs in a single board component, the BMC will promptly report the information to the upper-level network management through various industry-standard protocols such as SNMP protocol, SMTP protocol, and Redfish protocol.

[0042] (3) Remote control and management of servers: including server power on / off, restart, maintenance, firmware update, system installation and other operations.

[0043] (4) Maintenance management: including log management, user management, BIOS management, alarm management, etc.

[0044] The BMC is typically viewed as a small operating system or dedicated management system independent of the server's compute nodes, enjoying greater authority than the host server. This enables the BMC to manage and monitor business nodes in real time, ensuring the separation of business and management nodes, thereby preventing interference from business nodes on management nodes.

[0045] In a broad sense, BMC is a SoC system that consists of two layers: the BMC chip and the BMC firmware. It does not rely on other hardware (such as the CPU, memory, hard disk, etc.) or software (such as the BIOS, OS, CPLD, etc.) on the system, but exists as a completely independent system.

[0046] Optionally, the first capacitor is at a high level, and the BMC sets all lanes of the PCIe port of each PCIe device to a high level in sequence through the communication circuit. Therefore, after the PCIe device is inserted, the lane corresponding to the root node of the CPU to which the PCIe device is attached is also at a high level. Therefore, the control module can obtain the number of lanes of the PCIe port of the attached PCIe device by detecting the high-level lane corresponding to the root node of each CPU, thereby determining the bandwidth setting data table.

[0047] Optionally, if capacitors are provided on lanes, when the BMC sequentially sets all lanes of each PCIe device's PCIe port to a high level via the communication circuit, the lane level does not jump directly to a high level, but instead has a rising edge. Therefore, the control module can determine the number of lanes of the PCIe port of the attached PCIe device by detecting the lane level change corresponding to each CPU's root node, thereby determining the bandwidth setting data table. The level change is not limited to rising or falling edges.

[0048] The Basic Input / Output System (BIOS) is a set of programs stored in memory. It stores the computer's most important basic input and output programs, post-boot self-test programs, and system startup programs. It is used to allocate bandwidth to each PCIe device based on the bandwidth setting data table. The memory can be RAM or ROM, without limitation.

[0049] Specifically, after determining the bandwidth setting data table, the control module stores the bandwidth setting data table in the memory of the server during the server power-on self-test, and the basic input and output system allocates bandwidth to each root node of the CPU.

[0050] In an optional embodiment, the server includes at least one CPU, and the communication circuit includes:

[0051] The JTAG interface of the baseboard management controller, the JTAG interface of the CPU, and the JTAG interface of the PCIe device, where each JTAG interface includes a data input port (TDI), a data output port (TDO), a mode select port (TMS), and a clock port (TCK).

[0052] Specifically, the data output port of the baseboard management controller is connected to the data input port of the first-level CPU through the communication circuit, the clock port of the baseboard management controller is connected to the clock port of each CPU and the clock port of each PCIe device, and the mode selection port of the baseboard management controller is connected to the mode selection port of each CPU and the mode selection port of each PCIe device; the data output port of the first-level CPU is connected to the data input port of the next-level CPU, and the data output port of the last-level CPU is connected to the data input port of the first-level PCIe device; the data output port of the first-level PCIe device is connected to the data input port of the next-level PCIe device, and the data output port of the last-level PCIe device is connected to the data input port of the baseboard management controller.

[0053] Specifically, the JTAG interface was originally used for chip testing. The basic principle of the JTAG interface is to define a test access port (TAP) within a device to test internal nodes using dedicated JTAG test tools. JTAG testing allows multiple devices to be connected in series via the JTAG interface, forming a JTAG chain, allowing each device to be tested individually. Today, the JTAG interface is also commonly used for in-system programming (ISP) to program devices such as Flash memory.

[0054] JTAG programming is an in-circuit programming method. In traditional production processes, the chip is pre-programmed before being mounted on the board. This simplified process involves first mounting the device on the circuit board and then programming it using JTAG, greatly speeding up the project. The JTAG interface can program all components within the DSP chip.

[0055] Chips with JTAG ports have the following JTAG pin definitions:

[0056] TCK——test clock input;

[0057] TDI - test data input, data is input into the JTAG port through TDI;

[0058] TDO - test data output, data is output from the JTAG port through TDO;

[0059] TMS - Test mode selection, TMS is used to set the JTAG port to a specific test mode.

[0060] Optional pin TRST - test reset, input pin, low level is valid.

[0061] There are many types of chips that contain JTAG ports, such as CPU, DSP, CPLD, etc.

[0062] JTAG has a state machine inside, called the TAP controller. The TAP controller's state machine changes state based on the TCK test clock input and TMS test mode selection, enabling data and instruction input.

[0063] Boundary-scan registers in JTAG chips: The JTAG standard defines a serial shift register. Each cell in the register is assigned to a corresponding pin on the IC chip. Each individual cell is called a Boundary-Scan Cell (BSC). This series of BSCs forms the JTAG loop within the IC. All Boundary-Scan Registers (BSRs) are activated during JTAG testing, while these pins maintain normal IC functionality.

[0064] Specifically, BMC uses the boundary scan test function of the JTAG interface to complete the test. JTAG supports boundary scan testing (Boundary Scan), a technology used to detect soldering and connection problems on circuit boards. By placing a shift register (called a boundary scan cell) between each pin of the IC and the internal logic, the state of each pin can be controlled and observed. JTAG interface: The standard JTAG interface includes four signal lines: TMS (mode select), TCK (clock), TDI (data input), and TDO (data output). JTAG test mode: The boundary scan cell switches between functional mode and test mode, such as Figure 2 As shown in the figure, the BMC acts as the JTAG host end, and the CPU and various PCIe components act as the JTAG slave ends.

[0065] For example, Figure 2 Taking the two CPUs and two PCIe devices shown in the figure as an example, the BMC connects multiple CPUs and multiple PCIe devices in series through the JTAG communication circuit. Figure 1Taking the two CPUs (CPU0 and CPU1) and two PCIe devices (PCIe1 and PCIe2) shown in the figure as an example, the BMC's data output port TDO is connected to the CPU0's data input port TDI, which is then connected to the CPU1's data input port TDI. The CPU1's data output port TDO is connected to the PCIe1's data input port TDI, which is then connected to the PCIe2's data input port TDI. The PCIe2's data output port TDO is then connected back to the BMC's data input port TDI, forming a series JTAG communication circuit. The BMC's mode select port TMS is connected to the mode select ports TMS of CPU0, CPU1, PCIe1, and PCIe2. The BMC's clock port TCK is connected to the clock ports TCK of CPU0, CPU1, PCIe1, and PCIe2.

[0066] In an optional embodiment, the PCIe device bandwidth management device also includes: multiple switching circuits, one switching circuit corresponds to the JTAG interface of a PCIe device, wherein the switching circuit is used to disconnect the data input port and data output port of the JTAG interface of the PCIe device when the PCIe device is inserted; and short-circuit the data input port and data output port of the JTAG interface of the PCIe device when the PCIe device is not inserted.

[0067] In an optional embodiment, as Figure 3 As shown, the switch circuit includes: an analog switch and an isolation sub-circuit, wherein,

[0068] an analog switch, wherein a first end of the analog switch is connected to a data input port of a JTAG interface of a PCIe device, a second end of the analog switch is connected to a data output port of the JTAG interface of the PCIe device, a power supply end of the analog switch is connected to a power supply voltage, an enable end of the analog switch is connected to an output end of the isolation subcircuit, and the analog switch is configured to disconnect the data input port and the data output port based on an enable signal;

[0069] The isolation sub-circuit has a power supply end connected to the power supply voltage and an input end connected to the PRSNT2 pin of the PCIe interface of the PCIe device. It is used to output an enable signal to the analog switch when the PCIe device is inserted.

[0070] Specifically, the PCIe specification specifies the following pinouts for the PCIe device side: PRSNT1# (pin 1) on the A side; PRSNT2# (pin 17) on the B side of an x1 bandwidth device; PRSNT2# (pin 31) on the B side of an x4 bandwidth device; PRSNT2# (pin 48) on the B side of an x8 bandwidth device; and PRSNT2# (pin 81) on the B side of an x16 bandwidth device. PRSNT1# and PRSNT2# on the PCIe device side are connected together. On the slot side, PRSNT1# is grounded, and PRSNT2# is pulled high by a pull-up resistor. When a device is inserted into the slot, PRSNT2# is connected to PRSNT1#, pulling PRSNT2# low by ground. A transition from high to low indicates card insertion.

[0071] Optionally, the PCIe device hot plug level detection circuit is as follows: Figure 4 As shown in the figure, the gold fingers corresponding to PRSNT1# and PRSNT2# are of different lengths from other signals. When the other signals of the PCIe device are connected, the PRSNT1# and PRSNT2# of the PCIe device are connected together and therefore grounded and pulled low. The transition of PRSNT2# from high to low indicates that the PCIe device is plugged in. Similarly, it can also detect when the PCIe device is unplugged.

[0072] Specifically, based on the above hot-plug principle, when no PCIe device is inserted into the standard slot, the TDI and TDO signals of the slot are connected using an analog switch. When the PCIe device is inserted, TDI and TDO are disconnected, and the level status signal of PRSNT2 is transmitted to the BMC through the analog switch.

[0073] In an optional embodiment, as Figure 4 As shown, the isolation sub-circuit includes: a first IGBT tube D1, a first resistor R1, a second resistor R2 and a third resistor R3, wherein,

[0074] A first IGBT tube D1 has a gate connected to the first end of the second resistor R2 and the first end of the third resistor R3, an emitter connected to the first end of the first resistor R1 and the enable end of the analog switch, and a collector grounded;

[0075] A first resistor R1, a second end of which is connected to a power supply voltage;

[0076] A second resistor R2, a second end of which is connected to the power supply voltage;

[0077] A second end of the third resistor R3 is connected to the PRSNT2 pin of the PCIe interface of the PCIe device.

[0078] Optionally, the first IGBT tube D1 is a PNP-type IGBT tube, which can achieve electrical isolation.

[0079] Specifically, when the PCIe device is not inserted, PRSNT2 is suspended, the first IGBT tube D1 is closed, and the analog switch connects TDI and TDO. When the PCIe device is inserted, PRSNT2 is grounded, the first IGBT tube D1 is disconnected, and the analog switch disconnects TDI and TDO.

[0080] This application also provides a PCIe device bandwidth management method, which is applied to the control module of the PCIe device bandwidth management device. The method is described in detail in conjunction with the execution process of the PCIe device bandwidth management method. Figure 5 As shown, the method includes:

[0081] Step S1: Utilizing the communication circuit, the CPU and the PCIe device are configured to enter a test mode.

[0082] Step S2: Setting the data transmission channel of each PCIe device to a first level through the communication circuit.

[0083] Step S3: Based on the levels of all data transmission channels of the PCIe root node of each CPU, a bandwidth setting data table is determined, where the bandwidth setting data table is used to determine the PCIe device bandwidth.

[0084] Specifically, the control module sets all lanes of the PCIe port of each PCIe device to a high level in sequence through the communication circuit. Therefore, after the PCIe device is inserted, the lane corresponding to the root node of the CPU to which the PCIe device is attached is also a high level or the level changes. Therefore, the control module can obtain the number of lanes of the PCIe port of the attached PCIe device by detecting the high-level lane corresponding to the root node of each CPU, thereby determining the bandwidth setting data table.

[0085] In an optional implementation, the process of determining and allocating PCIe device bandwidth includes:

[0086] Read the levels of all data transmission channels of the PCIe root node of each CPU.

[0087] The position of the root node corresponding to the data transmission channel with the first level is detected, and the data transmission channel position of the PCIe root node of the CPU to which each PCIe device is mounted is recorded to obtain a bandwidth setting data table.

[0088] The bandwidth setting data table includes the number of all data transmission channels corresponding to the PCIe root node of each CPU at the first level.

[0089] Specifically, the BMC first uses the communication circuit to configure both the CPU and PCIe devices into test mode, sets all lanes of the PCIe port of each PCIe device to high levels in turn, then reads all PCIe root nodes (RootPorts) of each CPU, checks which RootPort corresponds to a high lane, records which CPU's RootPort corresponds to which data transmission channel (lane) of each PCIe device's PCIe port, and generates a data table in a lightweight data exchange format (JavaScript Object Notation, json), namely the bandwidth setting data table, and records the json data table to the Electrically Erasable Programmable Read-Only Memory (EEPROM).

[0090] In an optional implementation, when a capacitor is connected in series to a data transmission channel of a PCIe device, a process of determining and allocating bandwidth of the PCIe device includes:

[0091] Read the levels of all data transmission channels of the PCIe root node of each CPU; detect the position of the root node corresponding to the data transmission channel with level change, and record the position of the data transmission channel of the PCIe root node of the CPU to which each PCIe device is mounted, to obtain a bandwidth setting data table; the bandwidth setting data table includes the number of all data transmission channels corresponding to the PCIe root node of each CPU with a first level.

[0092] In an optional embodiment, before configuring the CPU and the PCIe device to enter the test mode using the communication circuit, the method further includes:

[0093] After the server mainboard is powered on, it checks whether it is the first time the server is powered on, whether the power supply has been disconnected, and whether the server has stored the bandwidth setting data table. If it is the first time the server is powered on, the power supply has been disconnected, and the server has stored the bandwidth setting data table, the CPU and PCIe devices are configured to enter the test mode using the communication circuit.

[0094] Optionally, if any one of the conditions in the server first power-on, power supply disconnection, and server storage bandwidth setting data table is not met, the current function program is exited.

[0095] In an optional embodiment, the process of detecting whether it is the first time to start the server and whether the server has been disconnected from the power supply includes:

[0096] Connect to the server's management network and access the server through the network tool of the intelligent platform management interface; based on the IP address in the server, obtain server information by viewing the server's documentation or using the management port search tool on the server motherboard. The server information includes whether it is the first time to start up and whether the power supply has been disconnected; based on the server information, determine whether it is the first time to start up and whether the server has been disconnected from the power supply.

[0097] Specifically, the BMC first checks whether it is the first time the server is powered on and whether the server has been disconnected from the AC220V power supply. The specific steps include:

[0098] (1) Connect to the server's management network through the Intelligent Platform Management Interface (IPMI) network tool. IPMI allows remote access to the server's BMC and the execution of operations.

[0099] (2) Determine the server's BMC IP address by viewing the server's documentation or using the management port lookup tool on the server's motherboard.

[0100] (3) Use various IPMI tools to connect to the BMC and obtain server information. A commonly used tool is ipmitool. Use ipmitool commands in the command line to perform various operations.

[0101] (4) After connecting to the BMC, use ipmitool to send various commands to obtain server information. For example, use the command "ipmitool power" to check whether the server has been turned on for the first time and whether the server has been disconnected from the AC220V power supply.

[0102] In an optional implementation, the process of detecting whether the server stores the bandwidth setting data table includes:

[0103] Check the server's replaceable unit information; determine whether the replaceable unit information includes a bandwidth setting data table.

[0104] Specifically, the BMC uses the "ipmitool fru" command via the IPMI interface to view the server's Field Replaceable Unit (FRU) information, including detailed descriptions of hardware components. These are typically replaceable components on the server, such as the motherboard, power supply, and fan. If a FRU is present, it can be quickly and easily replaced.

[0105] In typical server products, FRU information is burned into non-volatile memory (such as EEPROM). This information is referred to as VPD (Vital Product Data), or simply FRU data, such as the bandwidth configuration data table. Server motherboards typically have an EEPROM that stores the motherboard's FRU information (such as the manufacturer or product number). This information is accessed by the BMC, which also reads the JSON data table information stored in the EEPROM.

[0106] In an optional embodiment, after determining the bandwidth setting data table, it also includes: setting all CPUs and PCIe devices to enter working mode through the communication circuit and continuing to start up; during the power-on inspection process, the bandwidth setting data table is stored in the preset memory of the server.

[0107] Specifically, if Figure 6 As shown, the BMC sets all CPUs and PCIe devices to work mode, continues to boot, and passes the JSON data table to the basic input and output system, that is, the BIOS, during the BIOS POST (power-on inspection). The BIOS stores the JSON data table in the non-volatile random access memory area of BIOS FLASH, that is, the NVRAM area. In case the BMC does not pass the data table to the BIOS, the BIOS can use its local NVRAM data to initialize the PCIe bandwidth. After that, the BIOS allocates the bandwidth to each RootPort of the CPU.

[0108] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus the necessary general hardware platform, and of course it can also be implemented by hardware, but in many cases the former is a better implementation method.

[0109] The present application also provides a server, including: a PCIe device bandwidth management device. The PCIe device bandwidth management device includes: a communication circuit, a control module and a basic input and output system, wherein:

[0110] A communication circuit, which is used to connect the control module, the CPU and the PCIe device in series;

[0111] a control module configured to set the data transmission channel of each PCIe device to a first level through a communication circuit, and to configure a bandwidth setting data table based on the levels of all data transmission channels of the PCIe root node of each CPU;

[0112] The basic input and output system is used to allocate bandwidth to each PCIe device based on the bandwidth setting data table.

[0113] The embodiment of the present application also provides an electronic device, such as Figure 7 As shown, it includes a memory 10 and a processor 20, wherein the memory 10 stores a computer program, and the processor 20 is configured to run the computer program to execute the steps in any of the above-mentioned PCIe device bandwidth management method embodiments.

[0114] An embodiment of the present application further provides a computer-readable storage medium, in which a computer program is stored. The computer program is configured to execute the steps of any of the above-mentioned PCIe device bandwidth management method embodiments when running.

[0115] In an exemplary embodiment, the computer-readable storage medium may include, but is not limited to, various media that can store computer programs, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk, or an optical disk.

[0116] An embodiment of the present application further provides a computer program product, which includes a computer program. When the computer program is executed by a processor, the steps in any of the above-mentioned PCIe device bandwidth management method embodiments are implemented.

[0117] An embodiment of the present application further provides another computer program product, including a non-volatile computer-readable storage medium, wherein the non-volatile computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of any of the above-mentioned PCIe device bandwidth management method embodiments are implemented.

[0118] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the above description has generally described the components and steps of each example according to their functions. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0119] The above is a detailed introduction to a PCIe device bandwidth management device, method, and server provided by the present application. This article uses specific examples to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method and core ideas of the present application. It should be pointed out that for ordinary technicians in this technical field, without departing from the principles of the present application, several improvements and modifications can be made to the present application, and these improvements and modifications also fall within the scope of protection of the claims of the present application.

Claims

1. A PCIe device bandwidth management device, characterized in that: include: Communication circuit, control module and basic input and output system, the control module is a baseboard management controller in the server, wherein, A communication circuit, which is used to connect the control module, the CPU and the PCIe device in series; a control module configured to set the data transmission channel of each PCIe device to a first level through the communication circuit, and determine a bandwidth setting data table based on the levels of all data transmission channels of the PCIe root node of each CPU; a basic input / output system, configured to allocate bandwidth to each PCIe device based on the bandwidth setting data table; The server includes at least one CPU, and the communication circuit includes: a JTAG interface of a baseboard management controller, a JTAG interface of the CPU, and a JTAG interface of a PCIe device; The baseboard management controller is tested using the boundary scan test function of the JTAG interface.

2. The PCIe device bandwidth management device according to claim 1, wherein: The first level is a high level.

3. The PCIe device bandwidth management device according to claim 1, wherein: Each JTAG interface includes a data input port, a data output port, a mode selection port, and a clock port; The data output port of the baseboard management controller is connected to the data input port of the first-level CPU through a communication circuit, the clock port of the baseboard management controller is connected to the clock port of each CPU and the clock port of each PCIe device, and the mode selection port of the baseboard management controller is connected to the mode selection port of each CPU and the mode selection port of each PCIe device; The data output port of the first-level CPU is connected to the data input port of the next-level CPU, and the data output port of the last-level CPU is connected to the data input port of the first-level PCIe device; The data output port of the first-level PCIe device is connected to the data input port of the next-level PCIe device, and the data output port of the last-level PCIe device is connected to the data input port of the baseboard management controller.

4. The PCIe device bandwidth management device according to claim 1, wherein: Also includes: Multiple switch circuits, one switch circuit corresponds to a JTAG interface of a PCIe device, wherein, The switch circuit is used to disconnect the data input port and data output port of the JTAG interface of the PCIe device when the PCIe device is inserted; and to short-circuit the data input port and data output port of the JTAG interface of the PCIe device when the PCIe device is not inserted.

5. The PCIe device bandwidth management device according to claim 4, characterized in that: The switch circuit includes: an analog switch and an isolation sub-circuit, wherein: an analog switch, wherein a first end of the analog switch is connected to a data input port of a JTAG interface of a PCIe device, a second end of the analog switch is connected to a data output port of the JTAG interface of the PCIe device, a power supply end of the analog switch is connected to a power supply voltage, an enable end of the analog switch is connected to an output end of the isolation subcircuit, and the analog switch is configured to disconnect the data input port and the data output port based on an enable signal; The isolation sub-circuit has a power supply terminal connected to the power supply voltage and an input terminal connected to the PRSNT2 pin of the PCIe interface of the PCIe device. It is used to output an enable signal to the analog switch when the PCIe device is inserted.

6. The PCIe device bandwidth management device according to claim 5, characterized in that: The isolation sub-circuit includes: a first IGBT tube, a first resistor, a second resistor and a third resistor, wherein: A first IGBT tube, whose gate is connected to the first end of the second resistor and the first end of the third resistor, whose emitter is connected to the first end of the first resistor and the enable end of the analog switch, and whose collector is grounded; a first resistor, a second end of which is connected to a power supply voltage; a second resistor, a second end of which is connected to the supply voltage; A second end of the third resistor is connected to the PRSNT2 pin of the PCIe interface of the PCIe device.

7. The PCIe device bandwidth management device according to claim 6, characterized in that: The first IGBT tube is a PNP type IGBT tube.

8. A PCIe device bandwidth management method, characterized in that: The method is applied to the control module of the PCIe device bandwidth management device according to any one of claims 1 to 7, and the method includes: Using the communication circuit, both the CPU and PCIe devices are configured to enter test mode; Setting the data transmission channel of each PCIe device to a first level through the communication circuit; Based on the levels of all data transmission channels of the PCIe root node of each CPU, a bandwidth setting data table is determined, where the bandwidth setting data table is used to determine the PCIe device bandwidth.

9. The PCIe device bandwidth management method according to claim 8, wherein: The process of determining and allocating PCIe device bandwidth includes: Read the levels of all data transmission channels of the PCIe root node of each CPU; Detecting the position of the root node corresponding to the data transmission channel with the first level, and recording the data transmission channel position of the PCIe root node of the CPU to which each PCIe device is mounted, to obtain a bandwidth setting data table; The bandwidth setting data table includes the number of all data transmission channels corresponding to the PCIe root node of each CPU at the first level.

10. The PCIe device bandwidth management method according to claim 8, wherein: When a capacitor is connected to the data transmission channel of a PCIe device, the process of determining and allocating the PCIe device bandwidth includes: Read the levels of all data transmission channels of the PCIe root node of each CPU; Detect the location of the root node corresponding to the data transmission channel with level change, and record the data transmission channel location of the PCIe root node of the CPU to which each PCIe device is mounted, to obtain a bandwidth setting data table; The bandwidth setting data table includes the number of all data transmission channels corresponding to the PCIe root node of each CPU at the first level.

11. The PCIe device bandwidth management method according to claim 8, wherein: Before configuring the CPU and PCIe device to enter the test mode using the communication circuit, the following steps are also included: After the server motherboard is powered on, check whether it is the first time to start up, whether the server has been disconnected from the power supply, and whether the server has stored the bandwidth setting data table; If the server is powered on for the first time and has been disconnected from the power supply, the server stores a bandwidth setting data table, and then the CPU and PCIe devices are configured to enter the test mode using the communication circuit.

12. The PCIe device bandwidth management method according to claim 11, wherein: The process of checking whether this is the first power-on and whether the server has been disconnected from the power supply includes: Connect to the server's management network through the network tool of the intelligent platform management interface and access the server; Based on the IP address in the server, obtain server information by viewing the server documentation or using the management port search tool on the server motherboard. The server information includes whether it is the first time to start the server and whether the power supply has been disconnected; Based on the server information, determine whether it is the first time to start the server and whether the server has been disconnected from the power supply.

13. The PCIe device bandwidth management method according to claim 11, wherein: The process of detecting whether the server stores the bandwidth setting data table includes: View the server's replaceable unit information; It is determined whether the replaceable unit information includes a bandwidth setting data table.

14. The PCIe device bandwidth management method according to claim 8, wherein: Also includes: If any of the conditions in the server's first power-on, power-off, and server storage bandwidth setting data table are not met, the current function program will be exited.

15. The PCIe device bandwidth management method according to claim 8, wherein: After determining the bandwidth setting data table, it also includes: Through the communication circuit, set all CPUs and PCIe devices to enter working mode and continue to boot up; During the power-on inspection process, the bandwidth setting data table is stored in the preset memory of the server.

16. A server, characterized in that: include: The PCIe device bandwidth management device according to any one of claims 1 to 7.

17. An electronic device, characterized in that: include: memory for storing computer programs; A processor, configured to implement the steps of the PCIe device bandwidth management method according to any one of claims 8 to 15 when executing the computer program.

18. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, wherein when the computer program is executed by a processor, the steps of the PCIe device bandwidth management method according to any one of claims 8 to 15 are implemented.

19. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the PCIe device bandwidth management method according to any one of claims 8 to 15 are implemented.

Images