PCIe equipment bandwidth management device and method and server

Abstract

The invention discloses a PCIe device bandwidth management device, a PCIe device bandwidth management method and a server, and relates to the technical field of servers, a communication circuit is used for connecting a control module, a CPU and a PCIe device in a chained mode, redundant connecting lines in a traditional star topology are reduced, the hardware design complexity and cost are reduced, and the system integration degree and reliability are improved. The control module uniformly sets PCIe device data transmission channels to be the first level through the communication circuit, rapidly scans and identifies the channel state of each CPU root node, determines the bandwidth setting data table, can dynamically adapt to the bandwidth requirements of different devices, and avoids resource waste or conflicts.

Description

Technical Field

[0001] The present application relates to the field of server technology, and in particular to a PCIe device bandwidth management device, method and server. Background Art

[0002] In modern server systems, motherboards are usually equipped with PCIe x8 and PCIe x16 slots, providing users with flexible expansion capabilities. This design allows expansion cards of different specifications to be inserted and used, greatly improving the compatibility and scalability of hardware configurations.

[0003] However, the current server architecture has an obvious technical limitation: the CPU bandwidth allocation mechanism for PCIe x8 and x16 slots is a fixed maximum bandwidth mode. When an expansion card with a small number of lanes is inserted, the system cannot effectively identify and dynamically adapt the bandwidth, and still allocates resources according to the maximum bandwidth of the slot, resulting in low PCIe resource utilization, while increasing the CPU resource scheduling burden and causing functional loss. Summary of the invention

[0004] The present application provides a PCIe device bandwidth management apparatus, method and server to at least solve the problem of how to accurately identify and allocate PCIe device bandwidth in the related art.

[0005] The present application provides a PCIe device bandwidth management device, comprising: a communication circuit, a control module and a basic input-output system, wherein the communication circuit is used to connect the control module, a CPU and a PCIe device in series; the control module is used to set the data transmission channel of each PCIe device to a first level through the communication circuit, and determine a bandwidth setting data table based on the levels of all data transmission channels of the PCIe root node of each CPU; the basic input-output system is used to allocate bandwidth to each PCIe device based on the bandwidth setting data table.

[0006] The present application also provides a PCIe device bandwidth management method, which is applied to the control module of the PCIe device bandwidth management device of the first aspect and any optional implementation manner thereof, and the method includes: using a communication circuit to configure both the CPU and the PCIe device to enter a test mode; setting the data transmission channel of each PCIe device to a first level through the communication circuit; and determining a bandwidth setting data table based on the levels of all data transmission channels of the PCIe root node of each CPU, wherein the bandwidth setting data table is used to determine the PCIe device bandwidth.

[0007] The present application also provides a server, comprising: the above PCIe device bandwidth management device.

[0008] The present application also provides an electronic device, comprising: a memory for storing a computer program; and a processor for implementing the steps of any of the above-mentioned PCIe device bandwidth management methods when executing the computer program.

[0009] The present application also provides a computer-readable storage medium, in which a computer program is stored, wherein when the computer program is executed by a processor, the steps of any of the above-mentioned PCIe device bandwidth management methods are implemented.

[0010] The present application also provides a computer program product, including a computer program, which implements the steps of any of the above-mentioned PCIe device bandwidth management methods when executed by a processor.

[0011] Through this application, the control module, CPU and PCIe devices are chained together using communication circuits, which reduces redundant connection lines in the traditional star topology, reduces hardware design complexity and cost, and improves system integration and reliability. At the same time, when adding or removing PCIe devices, only the connection nodes on the link need to be adjusted, without the need for large-scale changes to the system architecture, which facilitates hardware upgrades and maintenance.

[0012] Through this application, the PCIe device data transmission channel is uniformly set to the first level through the communication circuit, the channel status of each CPU root node is quickly scanned and identified, and the bandwidth setting data table is determined, which can dynamically adapt to the bandwidth requirements of different devices to avoid resource waste or conflict. At the same time, the level state is forced to be unified, the electrical differences of different devices are shielded, the consistency of data transmission protocols is ensured, and the overall compatibility and stability of the system are improved.

[0013] Through this application, based on the bandwidth setting data table generated by the control module, the transmission resources of the PCIe device are accurately allocated, the performance requirements of high-load devices (such as GPUs and high-speed storage) are prioritized, and bottlenecks caused by bandwidth competition are avoided. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] In order to more clearly illustrate the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0015] Figure 1 A composition diagram of a PCIe device bandwidth management device provided in an embodiment of the present application; Figure 2 A communication circuit structure diagram provided for an embodiment of the present application; Figure 3 A specific circuit structure diagram of a switch circuit provided in an embodiment of the present application; Figure 4 A structural diagram of a hot-swap circuit provided in an embodiment of the present application; Figure 5 A flowchart of a PCIe device bandwidth management method provided in an embodiment of the present application; Figure 6 A flowchart of another PCIe device bandwidth management method provided in an embodiment of the present application; Figure 7 A diagram of the composition of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0016] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0017] It should be noted that, in the description of this application, the terms "include", "comprise" or any other variant thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. The terms "first", "second", etc. in this application are used to distinguish similar objects, and are not used to describe a specific order or sequence.

[0018] In order to enable those skilled in the art to better understand the present application, the present application is further described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0019] In conjunction with the specific application environment architecture or specific hardware architecture on which the execution of the PCIe device bandwidth management method depends, the specific application environment architecture or specific hardware architecture is described herein.

[0020] In the related technology, PCIe (Peripheral Component Interconnect Express) is a high-speed serial computer expansion card standard, mainly used to connect the CPU and various expansion cards, such as graphics cards, sound cards and network adapters. PCIe has taken a fundamental change in the bus structure, which is mainly reflected in two aspects: one is the change from a parallel bus to a serial bus; the other is the use of point-to-point interconnection. The bus under the bridge in the original parallel bus structure is changed into a link. A link can contain one or more channels. Each channel consists of two pairs of differential signal lines to form a dual-simplex serial transmission channel. There are no dedicated data, address, control and clock lines. Various transactions on the bus are organized into information packets for transmission. Another feature of PCIe that breaks through the traditional bus is the use of a point-to-point interconnection method. Each device is connected by an independent link and has exclusive bandwidth. This is an effective solution to improve the transmission rate. The PCIe bus structure regards a link as a bus, thereby maintaining compatibility with the traditional PCI bus in terms of address space, configuration mechanism and software. A PCIe device occupies a bus (link), so in a PCIe-based computer, there are many bridges and buses. Traditional PCI devices can also run on the PCIe platform. Through the PCIe to PCI (or PCI-X) bridge, the PCI (or PCI-X) bus can be brought out. Traditional PCI devices can be integrated into the PCIExpress structure by hanging on such a bus. The 11th generation Intel Core CPU and motherboard added support for PCIe4.0, and now the 12th generation platform provides PCIe 5.0 and is fully backward compatible with 4.0 and 3.0 devices. The 12th generation Intel Core CPU provides up to 16 CPU PCIe 5.0 channels and up to 4 CPU PCIe 4.0 channels, while the 11th generation Intel Core CPU (such as the Intel Core i9-11900K) provides up to 20 CPU PCIe 4.0 channels.

[0021] The bandwidth of PCIe is closely related to its version and the number of lanes for data transmission. The following are the details of different versions of PCIe and their bandwidth: PCIe 1.0: The online bit rate is 2.5Gb / s, using 8 / 10 encoding, so the bandwidth of PCIe 1.0x1 is 250MB / s. PCIe 2.0: The online bit rate doubles to 5Gb / s, using 8 / 10 encoding, so the bandwidth of PCIe 2.0x 1 is 500MB / s. PCIe 3.0: The online bit rate is 8Gb / s, using 128 / 130 encoding, so the bandwidth of PCIe 3.0x1 is 1GB / s. PCIe 4.0: The online bit rate is increased to 16GT / s. PCIe 5.0: The online bit rate can reach up to 32GT / s.

[0022] In addition, the bandwidth of PCIe is also related to the number of Lanes. For example, the theoretical maximum bandwidth of PCIe x1 slot is 1GB / s, which is suitable for devices with low data transmission requirements; the theoretical maximum bandwidth of PCIe x4 slot is 4GB / s, which is suitable for devices with medium bandwidth requirements; the theoretical maximum bandwidth of PCIe x8 slot is 8GB / s, which is suitable for devices with higher data transmission speed requirements; PCIe x16 slot has the highest theoretical maximum bandwidth of 16GB / s, which is widely used to connect high-performance graphics cards.

[0023] The PICe interface on the server motherboard generally uses PCIe x8 slots and PCIe x16 slots to facilitate the later insertion of different types of expansion cards. Even if the gold finger on the expansion card is x2 or x4, it can also be plugged into the PCIe x8 slot or PCIex16 slot. However, the bandwidth directly provided by the CPU to the PCIe x8 slot and PCIe x16 slot is the maximum bandwidth, resulting in the inability to effectively identify the expansion card when a small number of lanes are inserted. The CPU also allocates the maximum bandwidth, resulting in a waste of PCIe resources and a loss of CPU function. For example, two PCIE X8 devices are plugged into two PCIE X16 slots, but it is impossible to support two X8 devices in one X16 slot; PCIe cannot automatically identify and allocate bandwidth.

[0024] Based on the above problems, at present, PCIe automatically identifies and allocates bandwidth in two ways: first, using the PCA9555GPIO expansion chip instead of GPIO to dynamically configure the bandwidth; second, using the resistance control current or voltage detection method on the physical detection circuit board to determine the required bandwidth.

[0025] Both of the above methods require adding a bandwidth allocation table in the non-matching detection program, extending chip detection or physical resistance detection. After obtaining the change data, it is necessary to match the numerical value with the corresponding Lane number in the bandwidth allocation table set in advance, and then the CPU analyzes the corresponding bandwidth for allocation. In the early stage, it is necessary to count the data of different expansion cards to simulate and build a table. The types and models of corresponding expansion cards are fixed. After the expansion card data that is not in the table is plugged in, there is an identification error. The table needs to be matched when used, and the overall identification and allocation of bandwidth are inefficient.

[0026] Bandwidth is also allocated by the pin level of the PCIe device. PRSNT2# can only allocate the required bandwidth to a device when there is an existing device in the X16 Port, or PRSNT2# can only distinguish the bandwidth of X8 and X16. Both of these also result in unclear and inaccurate allocation.

[0027] There are many server models and many customer customization requirements, which leads to the diversity of server CPU PCIe port connection forms. The connection between the CPU PCIe port and the device and Riser card on different motherboards is not the same. The PCIe port allocation inside the Riser card itself is also diverse. Each model and each customized shipping configuration BIOS needs to be developed and adapted separately according to the motherboard PCIe port topology and the PCIe port topology of several Riser cards matched with the motherboard. As a result, the development of server firmware for a specific configuration is very cumbersome and affects the efficiency of customized development and mass production of server products.

[0028] Based on this, the present application provides a PCIe device bandwidth management device, such as Figure 1 As shown, it includes: a communication circuit based on the Joint Test Action Group (JTAG) interface, a control module and a basic input and output system.

[0029] The communication circuit is used to connect the control module, CPU and PCIe device in series.

[0030] The control module is used to set the data transmission channel of each PCIe device to the first level through the communication circuit, and the bandwidth setting data table based on the levels of all data transmission channels of the PCIe root node of each CPU.

[0031] Optionally, the control module is a baseboard management controller (BMC) in the server. The baseboard management controller (BMC) is a dedicated controller for monitoring and managing the server. In layman's terms, the BMC is an independent system under the host server system. This independent system has its own processor and memory. Even if the host hardware or operating system crashes or is shut down, the host system can still be managed by the BMC system. It is similar to a backdoor on the server and is used to manage the server. Generally, a new server is to be put on the shelf. After the server is put on the shelf, one of the ways is to modify the network configuration related to the BMC, configure the IP address and gateway, connect the network cable through the out-of-band management port to achieve communication, and then use IPMI to achieve remote management of the server, such as console redirection (KVM) and other operations, and then perform remote management and install the system.

[0032] The BMC system mainly has the following functions: (1) Equipment information management: Record detailed information about the server, including model, manufacturer, date, production and technical information of each component, chassis information, motherboard information, etc., as well as BMC information itself, such as server host name, IP address, BMC firmware version, etc.

[0033] (2) Server status monitoring and management: Detect the temperature, voltage and other health status of each server component (such as CPU, memory, hard disk, fan, frame, etc.). At the same time, adjust the fan speed in real time according to the conditions of each temperature collection point to ensure that the server does not overheat and control the overall power consumption within a reasonable range. If any abnormality occurs in a single board component, the BMC will report the information to the upper-level network management in a timely manner through various industry-wide specifications such as SNMP protocol, SMTP protocol, and Redfish protocol.

[0034] (3) Remote control management of servers: including server power on / off, restart, maintenance, firmware update, system installation and other operations.

[0035] (4) Maintenance management: including log management, user management, BIOS management, alarm management, etc.

[0036] BMC is usually regarded as a small operating system or dedicated management system independent of the server computing node, which has greater authority than the host server. This enables BMC to manage and monitor business nodes in real time, ensuring the separation of business nodes and management nodes, thereby avoiding interference of business nodes on management nodes.

[0037] In a broad sense, BMC is a SOC system, which is divided into two levels: BMC chip and BMC firmware. It does not rely on other hardware (such as CPU, memory, hard disk, etc.) or software (such as BIOS, OS, CPLD, etc.) on the system, but exists as a completely independent system.

[0038] Optionally, the first capacitor is at a high level, and the BMC sets all lanes of the PCIe port of each PCIe device to a high level in sequence through the communication circuit. Therefore, after the PCIe device is inserted, the lane corresponding to the root node of the CPU to which the PCIe device is mounted is also at a high level. Therefore, the control module can obtain the number of lanes of the PCIe port of the mounted PCIe device by detecting the high-level lane corresponding to the root node of each CPU, thereby determining the bandwidth setting data table.

[0039] Optionally, if a capacitor is set on the lane, when the BMC sequentially sets all lanes of the PCIe port of each PCIe device to a high level through the communication circuit, the level of the lane will not jump directly to a high level, but has a rising edge. Therefore, the control module can obtain the number of lanes of the PCIe port of the mounted PCIe device by detecting the lane with a level change corresponding to the root node of each CPU, thereby determining the bandwidth setting data table. The level change is not limited to a rising edge or a falling edge.

[0040] The Basic Input / Output System (BIOS) is a set of programs that are fixed to the memory. It stores the most important basic input and output programs of the computer, the self-check program after power-on, and the system self-starting program. It is used to allocate bandwidth to each PCIe device based on the bandwidth setting data table. The memory can be RAM memory or ROM memory, which is not limited here.

[0041] Specifically, after determining the bandwidth setting data table, the control module stores the bandwidth setting data table in the memory of the server when the server performs a self-test, and the basic input and output system allocates bandwidth to each root node of the CPU.

[0042] In an optional embodiment, the server includes at least one CPU, and the communication circuit includes: The JTAG interface of the baseboard management controller, the JTAG interface of the CPU and the JTAG interface of the PCIe device, wherein each JTAG interface includes a data input port (TDI), a data output port (TDO), a mode selection port (TMS) and a clock port (TCK).

[0043] Specifically, the data output port of the baseboard management controller is connected to the data input port of the first-level CPU through the communication circuit, the clock port of the baseboard management controller is connected to the clock port of each CPU and the clock port of each PCIe device, and the mode selection port of the baseboard management controller is connected to the mode selection port of each CPU and the mode selection port of each PCIe device; the data output port of the first-level CPU is connected to the data input port of the next-level CPU, and the data output port of the last-level CPU is connected to the data input port of the first-level PCIe device; the data output port of the first-level PCIe device is connected to the data input port of the next-level PCIe device, and the data output port of the last-level PCIe device is connected to the data input port of the baseboard management controller.

[0044] Specifically, the JTAG interface was originally used to test chips. The basic principle of the JTAG interface is to define a test access port (TAP) inside the device to test the internal nodes through a dedicated JTAG test tool. JTAG testing allows multiple devices to be connected in series through the JTAG interface to form a JTAG chain, which can test each device separately. Today, the JTAG interface is also commonly used to implement In-System Programmer (ISP) to program devices such as FLASH.

[0045] JTAG programming is online programming. In the traditional production process, the chip is pre-programmed before being installed on the board. The simplified process is to first fix the device on the circuit board and then use JTAG programming, which greatly speeds up the project progress. The JTAG interface can program all components inside the DSP chip.

[0046] Chips with JTAG ports have the following JTAG pin definitions: TCK——test clock input; TDI——test data input, data is input into the JTAG port through TDI; TDO——Test data output, data is output from the JTAG port through TDO; TMS - Test mode selection, TMS is used to set the JTAG port to a specific test mode.

[0047] Optional pin TRST - test reset, input pin, low level is valid.

[0048] There are many types of chips that contain JTAG ports, such as CPU, DSP, CPLD, etc.

[0049] There is a state machine inside JTAG, called TAP controller. The state machine of TAP controller changes state through TCK test clock input and TMS test mode selection to realize data and instruction input.

[0050] Boundary-scan register of JTAG chip: JTAG standard defines a serial shift register. Each unit of the register is assigned to the corresponding pin of IC chip, and each independent unit is called BSC (Boundary-Scan Cell). This series BSC forms JTAG loop inside IC, and all BSR (Boundary-Scan Register) boundary scan registers are activated by JTAG test, and these pins maintain normal IC function at ordinary times.

[0051] Specifically, BMC uses the boundary scan test function of the JTAG interface to complete the test. JTAG supports boundary scan testing (Boundary Scan), which is a technology used to detect soldering and connection problems on circuit boards. By placing a shift register (called a boundary scan cell) between each pin of the IC and the internal logic, the state of each pin can be controlled and observed. JTAG interface: The standard JTAG interface includes four signal lines: TMS (mode select), TCK (clock), TDI (data input), and TDO (data output). JTAG test mode: The boundary scan cell switches between functional mode and test mode, such as Figure 2 As shown in the figure, the BMC acts as the host end of JTAG, and the CPU and each PCIe component act as the slave end of JTAG.

[0052] For example, Figure 2 Taking the two CPUs and two PCIe devices shown in the figure as an example, the BMC connects multiple CPUs and multiple PCIe devices in series through the JTAG communication circuit. Figure 1 Taking the two CPUs (CPU0 and CPU1) and two PCIe devices (PCIe1 and PCIe2) shown in the figure as an example, the data output port TDO of the BMC is connected to the data input port TDI of CPU0, the data output port TDO of CPU0 is connected to the data input port TDI of CPU1, the data output port TDO of CPU1 is connected to the data input port TDI of PCIe1, the data output port TDO of PCIe1 is connected to the data input port TDI of PCIe2, and the data output port TDO of PCIe2 is connected back to the data input port TDI of the BMC, forming a series circuit structure of the JTAG communication circuit. The mode selection port TMS of the BMC is connected to the mode selection ports TMS of CPU0, CPU1, PCIe1 and PCIe2, and the clock port TCK of the BMC is connected to the clock ports TCK of CPU0, CPU1, PCIe1 and PCIe2.

[0053] In an optional embodiment, the PCIe device bandwidth management device also includes: multiple switching circuits, one switching circuit corresponds to a JTAG interface of a PCIe device, wherein the switching circuit is used to disconnect the data input port and data output port of the JTAG interface of the PCIe device when the PCIe device is inserted; and short-circuit the data input port and data output port of the JTAG interface of the PCIe device when the PCIe device is not inserted.

[0054] In an optional embodiment, if Figure 3 As shown, the switch circuit includes: an analog switch and an isolation subcircuit, wherein: An analog switch, wherein a first end thereof is connected to a data input port of a JTAG interface of a PCIe device, a second end thereof is connected to a data output port of a JTAG interface of a PCIe device, a power supply end thereof is connected to a power supply voltage, an enable end thereof is connected to an output end of an isolation subcircuit, and is used to disconnect a connection between a data input port and a data output port based on an enable signal; The isolation sub-circuit has a power supply end connected to the power supply voltage and an input end connected to the PRSNT2 pin of the PCIe interface of the PCIe device, and is used to output an enable signal to the analog switch when the PCIe device is inserted.

[0055] Specifically, the relevant PCIE specification stipulates the PCIe device side pins, A side 1 pin PRSNT1#, X1 bandwidth device B side 17 pin PRSNT2#, X4 bandwidth device B side 31 pin PRSNT2#, X8 bandwidth device B side 48 pin PRSNT2#, X16 bandwidth device B side 81 pin PRSNT2#. PRSNT1# and PRSNT2# on the PCIe device side are connected together. PRSNT1# on the slot side is grounded, and PRSNT2# is pulled up by a pull-up resistor. After the device is inserted into the slot, PRSNT2# is connected to PRSNT1#, so PRSNT2# is grounded and pulled down. The change of PRSNT2# from high to low means that the card is inserted.

[0056] Optionally, the PCIe device hot plug level detection circuit is as follows: Figure 4 As shown. The gold fingers corresponding to PRSNT1# and PRSNT2# are not equal in length to other signals. When other signals of the PCIe device are connected, the PRSNT1# and PRSNT2# of the PCIe device are connected together and therefore grounded and pulled low. The transition of PRSNT2# from high to low indicates that the PCIe device is plugged in. Similarly, it can also detect that the PCIe device is unplugged.

[0057] Specifically, based on the above hot-plug principle, when no PCIe device is inserted into the standard slot, the TDI and TDO signals of the slot are connected using an analog switch. When the PCIe device is inserted, TDI and TDO are disconnected, and the level status signal of PRSNT2 is transmitted to the BMC through the analog switch.

[0058] In an optional embodiment, if Figure 4 As shown, the isolation sub-circuit includes: a first IGBT tube D1, a first resistor R1, a second resistor R2 and a third resistor R3, wherein: A first IGBT tube D1, whose gate is connected to the first end of the second resistor R2 and the first end of the third resistor R3, whose emitter is connected to the first end of the first resistor R1 and the enable end of the analog switch, and whose collector is grounded; A first resistor R1, a second end of which is connected to a power supply voltage; A second resistor R2, a second end of which is connected to the supply voltage; A second end of the third resistor R3 is connected to the PRSNT2 pin of the PCIe interface of the PCIe device.

[0059] Optionally, the first IGBT tube D1 is a PNP-type IGBT tube, which can achieve electrical isolation.

[0060] Specifically, when the PCIe device is not inserted, PRSNT2 is suspended, the first IGBT tube D1 is closed, and the analog switch connects TDI and TDO. When the PCIe device is inserted, PRSNT2 is grounded, the first IGBT tube D1 is disconnected, and the analog switch disconnects TDI and TDO.

[0061] The present application also provides a PCIe device bandwidth management method, which is applied to a control module of a PCIe device bandwidth management device, and the method is described in detail in conjunction with the execution flow of the PCIe device bandwidth management method. Figure 5 As shown, the method includes: Step S1: Utilizing the communication circuit, the CPU and the PCIe device are configured to enter a test mode.

[0062] Step S2: Setting the data transmission channel of each PCIe device to a first level through the communication circuit.

[0063] Step S3: Based on the levels of all data transmission channels of the PCIe root node of each CPU, a bandwidth setting data table is determined, where the bandwidth setting data table is used to determine the PCIe device bandwidth.

[0064] Specifically, the control module sets all lanes of the PCIe port of each PCIe device to a high level in sequence through the communication circuit. Therefore, after the PCIe device is inserted, the lane corresponding to the root node of the CPU to which the PCIe device is connected is also at a high level or the level changes. Therefore, the control module obtains the number of lanes of the PCIe port of the connected PCIe device by detecting the high-level lane corresponding to the root node of each CPU, thereby determining the bandwidth setting data table.

[0065] In an optional implementation, the process of determining and allocating PCIe device bandwidth includes: Read the level of all data transmission channels of the PCIe root node of each CPU.

[0066] The position of the root node corresponding to the data transmission channel with the first level is detected, and the position of the data transmission channel of the PCIe root node of the CPU to which each PCIe device is mounted is recorded to obtain a bandwidth setting data table.

[0067] The bandwidth setting data table includes the number of all data transmission channels of the first level corresponding to the PCIe root node of each CPU.

[0068] Specifically, the BMC first uses the communication circuit to configure the CPU and PCIe devices into the test mode, sets all lanes of the PCIe port of each PCIe device to high level in turn, and then reads all PCIe root nodes of each CPU, namely RootPort, to check which lane corresponding to which RootPort is high level, and records which data transmission channel (ie lane) corresponding to which RootPort of which CPU the PCIe port of each PCIe device is hung on, and generates a data table in a lightweight data exchange format (JavaScript Object Notation, json) format, namely the bandwidth setting data table, and records the json data table to the Electrically Erasable Programmable Read-Only Memory (EEPROM).

[0069] In an optional implementation manner, when a capacitor is connected in series to a data transmission channel of a PCIe device, a process of determining and allocating a bandwidth of the PCIe device includes: Read the levels of all data transmission channels of the PCIe root node of each CPU; detect the position of the root node corresponding to the data transmission channel with level change, and record the position of the data transmission channel of the PCIe root node of the CPU to which each PCIe device is hung, and obtain a bandwidth setting data table; the bandwidth setting data table contains the number of all data transmission channels whose level is the first level corresponding to the PCIe root node of each CPU.

[0070] In an optional implementation, before using the communication circuit to configure both the CPU and the PCIe device to enter the test mode, the method further includes: After the server mainboard is powered on, it detects whether it is the first time to start up, whether the server has been disconnected from the power supply, and whether the server stores the bandwidth setting data table; if the server is started up for the first time, the power supply has been disconnected, and the server stores the bandwidth setting data table, then the CPU and PCIe devices are configured to enter the test mode using the communication circuit.

[0071] Optionally, if any one of the conditions in the server first power-on, power supply disconnection, and server storage bandwidth setting data table is not met, the current function program is exited.

[0072] In an optional implementation, the process of detecting whether it is the first time to start the server and whether the server has been disconnected from the power supply includes: Connect to the management network of the server through the network tool of the intelligent platform management interface and access the server; based on the IP address in the server, obtain server information by viewing the server documentation or using the management port search tool on the server motherboard. The server information includes whether it is the first startup information and whether the power supply has been disconnected; based on the server information, determine whether it is the first startup and whether the server has been disconnected from the power supply.

[0073] Specifically, the BMC first detects whether it is the first time to start the server and whether the server has been disconnected from the AC220V power supply. The specific steps include: (1) Connect to the server's management network through the network tool of the Intelligent Platform Management Interface (IPMI). IPMI allows remote access to the server's BMC and the execution of operations.

[0074] (2) Determine the server's BMC IP address by viewing the server's documentation or using the management port search tool on the server motherboard.

[0075] (3) Use various IPMI tools to connect to the BMC and obtain server information. A commonly used tool is ipmitool. Use ipmitool commands in the command line to perform various operations.

[0076] (4) After connecting to the BMC, use ipmitool to send various commands to obtain server information. For example, use the command "ipmitool power" to check whether the server is turned on for the first time and whether the server has been disconnected from the AC220V power supply.

[0077] In an optional implementation, the process of detecting whether the server stores the bandwidth setting data table includes: Check the replaceable unit information of the server; determine whether the replaceable unit information includes a bandwidth setting data table.

[0078] Specifically, the BMC uses the command "ipmitool fru" through the IPMI interface to view the FRU (Field Replaceable Unit) information of the server, including detailed descriptions of hardware components. Generally, these are some replaceable components used on the server. For example, the motherboard, power supply, fan, etc. If it is a FRU, it can be directly and quickly replaced.

[0079] In general server products, the FRU information is burned into non-volatile memory (such as EEPROM), and this information is called VPD (Vital Product Data). Some are directly called FRU data, such as bandwidth setting data table. On the server motherboard, there is usually an EEPROM to store the motherboard's FRU information (version factory, or product number...), which is read through the BMC. At the same time, the BMC will also read the json data table information in the EEPROM.

[0080] In an optional implementation, after determining the bandwidth setting data table, it also includes: setting all CPUs and PCIe devices to enter working mode through the communication circuit and continuing to boot up; during the boot inspection process, storing the bandwidth setting data table in a preset memory of the server.

[0081] Specifically, Figure 6 As shown, the BMC sets all CPUs and PCIe devices to enter working mode, continues to boot up, and passes the json data table to the basic input and output system, i.e., BIOS, during the BIOS POST (power-on inspection). The BIOS stores the json data table in the non-volatile random access memory area of ​​BIOS FLASH, i.e., the NVRAM area, in case the BMC does not pass the data table to the BIOS. The BIOS can use its local NVRAM data to initialize the PCIe bandwidth, and then the BIOS allocates the bandwidth of each RootPort of the CPU.

[0082] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method.

[0083] The present application also provides a server, including: a PCIe device bandwidth management device. The PCIe device bandwidth management device includes: a communication circuit, a control module and a basic input and output system, wherein: A communication circuit, which is used to connect the control module, the CPU and the PCIe device in series; A control module, which is used to set the data transmission channel of each PCIe device to a first level through a communication circuit, and a bandwidth setting data table based on the levels of all data transmission channels of the PCIe root node of each CPU; A basic input / output system for allocating bandwidth to each PCIe device based on a bandwidth setting data table.

[0084] The embodiment of the present application also provides an electronic device, such as Figure 7As shown, it includes a memory 10 and a processor 20, the memory 10 stores a computer program, and the processor 20 is configured to run the computer program to execute the steps in any of the above PCIe device bandwidth management method embodiments.

[0085] An embodiment of the present application further provides a computer-readable storage medium, in which a computer program is stored, wherein the computer program is configured to execute the steps of any of the above-mentioned PCIe device bandwidth management method embodiments when running.

[0086] In an exemplary embodiment, the computer-readable storage medium may include, but is not limited to, various media that can store computer programs, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk.

[0087] An embodiment of the present application further provides a computer program product, the computer program product comprising a computer program, and when the computer program is executed by a processor, the steps in any one of the above-mentioned PCIe device bandwidth management method embodiments are implemented.

[0088] An embodiment of the present application also provides another computer program product, including a non-volatile computer-readable storage medium, the non-volatile computer-readable storage medium storing a computer program, and when the computer program is executed by a processor, the steps in any of the above-mentioned PCIe device bandwidth management method embodiments are implemented.

[0089] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0090] The above is a detailed introduction to a PCIe device bandwidth management device, method and server provided by the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method and core idea of ​​the present application. It should be pointed out that for ordinary technicians in this technical field, without departing from the principles of the present application, several improvements and modifications can be made to the present application, and these improvements and modifications also fall within the scope of protection of the claims of the present application.

Claims

1. A PCIe device bandwidth management device, characterized in that: include: Communication circuit, control module and basic input and output system, wherein: A communication circuit, which is used to connect the control module, the CPU and the PCIe device in series; A control module, which is used to set the data transmission channel of each PCIe device to a first level through the communication circuit, and determine a bandwidth setting data table based on the levels of all data transmission channels of the PCIe root node of each CPU; The basic input and output system is used to allocate bandwidth to each PCIe device based on the bandwidth setting data table.

2. The PCIe device bandwidth management device according to claim 1, characterized in that: The control module is a baseboard management controller in the server.

3. The PCIe device bandwidth management device according to claim 1, characterized in that: The first level is a high level.

4. The PCIe device bandwidth management device according to claim 2, characterized in that: The server includes at least one CPU, and the communication circuit includes: The JTAG interface of the baseboard management controller, the JTAG interface of the CPU and the JTAG interface of the PCIe device, wherein each JTAG interface includes a data input port, a data output port, a mode selection port and a clock port; The data output port of the baseboard management controller is connected to the data input port of the first-level CPU through a communication circuit, the clock port of the baseboard management controller is connected to the clock port of each CPU and the clock port of each PCIe device, and the mode selection port of the baseboard management controller is connected to the mode selection port of each CPU and the mode selection port of each PCIe device; The data output port of the first-level CPU is connected to the data input port of the next-level CPU, and the data output port of the last-level CPU is connected to the data input port of the first-level PCIe device; The data output port of the first-level PCIe device is connected to the data input port of the next-level PCIe device, and the data output port of the last-level PCIe device is connected to the data input port of the baseboard management controller.

5. The PCIe device bandwidth management device according to claim 1, characterized in that: Also includes: Multiple switch circuits, one switch circuit corresponds to a JTAG interface of a PCIe device, wherein: The switch circuit is used to disconnect the data input port and data output port of the JTAG interface of the PCIe device when the PCIe device is inserted; and to short-circuit the data input port and data output port of the JTAG interface of the PCIe device when the PCIe device is not inserted.

6. The PCIe device bandwidth management device according to claim 5, characterized in that: The switch circuit includes: an analog switch and an isolation subcircuit, wherein: an analog switch, wherein a first end of the analog switch is connected to a data input port of a JTAG interface of a PCIe device, a second end of the analog switch is connected to a data output port of a JTAG interface of a PCIe device, a power supply end of the analog switch is connected to a power supply voltage, an enable end of the analog switch is connected to an output end of the isolation subcircuit, and the analog switch is used to disconnect the data input port and the data output port based on an enable signal; The isolation sub-circuit has a power supply end connected to the power supply voltage and an input end connected to the PRSNT2 pin of the PCIe interface of the PCIe device, and is used to output an enable signal to the analog switch when the PCIe device is inserted.

7. The PCIe device bandwidth management device according to claim 6, characterized in that: The isolation subcircuit includes: a first IGBT tube, a first resistor, a second resistor and a third resistor, wherein: A first IGBT tube, whose gate is connected to the first end of the second resistor and the first end of the third resistor, whose emitter is connected to the first end of the first resistor and the enable end of the analog switch, and whose collector is grounded; A first resistor, a second end of which is connected to a supply voltage; A second resistor, a second end of which is connected to the supply voltage; A second end of the third resistor is connected to the PRSNT2 pin of the PCIe interface of the PCIe device.

8. The PCIe device bandwidth management device according to claim 7, characterized in that: The first IGBT tube is a PNP type IGBT tube.

9. A PCIe device bandwidth management method, characterized in that: The method is applied to the control module of the PCIe device bandwidth management device according to any one of claims 1 to 8, and the method comprises: Using the communication circuit, both the CPU and the PCIe device are configured to enter a test mode; Setting the data transmission channel of each PCIe device to a first level through the communication circuit; Based on the levels of all data transmission channels of the PCIe root node of each CPU, a bandwidth setting data table is determined, and the bandwidth setting data table is used to determine the PCIe device bandwidth.

10. The PCIe device bandwidth management method according to claim 9, characterized in that: The process of determining and allocating PCIe device bandwidth includes: Read the level of all data transmission channels of the PCIe root node of each CPU; Detecting the position of the root node corresponding to the data transmission channel with the first level, and recording the data transmission channel position of the PCIe root node of the CPU to which each PCIe device is mounted, to obtain a bandwidth setting data table; The bandwidth setting data table includes the number of all data transmission channels at the first level corresponding to the PCIe root node of each CPU.

11. The PCIe device bandwidth management method according to claim 9, characterized in that: When a capacitor is connected in series to the data transmission channel of a PCIe device, the process of determining and allocating the bandwidth of the PCIe device includes: Read the level of all data transmission channels of the PCIe root node of each CPU; Detect the position of the root node corresponding to the data transmission channel with level change, and record the data transmission channel position of the PCIe root node of the CPU to which each PCIe device is mounted, to obtain a bandwidth setting data table; The bandwidth setting data table includes the number of all data transmission channels at the first level corresponding to the PCIe root node of each CPU.

12. The PCIe device bandwidth management method according to claim 9, characterized in that: Before configuring the CPU and the PCIe device to enter the test mode using the communication circuit, the method further includes: After the server mainboard is powered on, check whether it is the first time to start the server, whether the server has been disconnected from the power supply, and whether the server has stored the bandwidth setting data table; If the server is powered on for the first time and the power supply has been disconnected, the server stores a bandwidth setting data table, and then the CPU and PCIe devices are configured to enter the test mode using the communication circuit.

13. The PCIe device bandwidth management method according to claim 12, characterized in that: The process of detecting whether it is the first time to start the server and whether the server has been disconnected from the power supply includes: Connect to the server's management network through the network tool of the intelligent platform management interface and access the server; Based on the IP address in the server, obtain server information by viewing the server document or using the management port search tool on the server motherboard, the server information includes whether it is the first time to start the computer and whether the power supply has been disconnected; According to the server information, it is determined whether it is the first time to start the server and whether the server has been disconnected from the power supply.

14. The PCIe device bandwidth management method according to claim 12, characterized in that: The process of detecting whether the server stores the bandwidth setting data table includes: View the server's replaceable unit information; It is determined whether the replaceable unit information includes a bandwidth setting data table.

15. The PCIe device bandwidth management method according to claim 9, characterized in that: Also includes: If any of the conditions in the server's first power-on, power-off, and server storage bandwidth setting data table are not met, the current function program will be exited.

16. The PCIe device bandwidth management method according to claim 9, characterized in that: After determining the bandwidth setting data table, it also includes: Through the communication circuit, all CPUs and PCIe devices are set to enter the working mode and continue to boot up; During the power-on inspection process, the bandwidth setting data table is stored in the preset memory of the server.

17. A server, characterized in that: include: A PCIe device bandwidth management device as described in any one of claims 1-8.

18. An electronic device, characterized in that: include: Memory for storing computer programs; A processor, configured to implement the steps of the PCIe device bandwidth management method as claimed in any one of claims 9 to 16 when executing the computer program.

19. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, wherein the computer program, when executed by a processor, implements the steps of the PCIe device bandwidth management method according to any one of claims 9 to 16.

20. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the PCIe device bandwidth management method according to any one of claims 9 to 16 are implemented.

Images