Communication control method, computer program product, and basic input / output system

Abstract

The application discloses a communication control method, a computer program product and a basic input / output system, relates to the technical field of servers, and comprises the following steps: acquiring first maximum payloads of target links in the case that a mainboard of a server is reset; grouping the target links; virtual switches of the target links in the same link group are connected to different ports of the same physical switch; determining a first maximum payload with the minimum value from the first maximum payloads of the link groups; and updating second maximum payloads of target devices according to the first maximum payloads with the minimum values of the link groups, so that the problem that target devices connected by different virtual switches need to be deployed in the same way and cause resource waste and redundancy can be solved, correct communication between the target devices across different target links can be realized, the deployment flexibility is high, the resource waste and device redundancy are reduced, and the hardware cost is optimized.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of server, and particularly relates to a communication control method, a computer program product and a basic input output system. BACKGROUND

[0002] In a server, the maximum payload size (MPS) on each target link based on the PCIe (Peripheral Component Interconnect Express) protocol is usually determined by each target device on the target link, and the maximum payload size defines the maximum size of the payload in a data packet. The maximum payload size of each target link is independent. When east-west data transmission is performed, if the maximum payload sizes of two target links are different, the encapsulation formats of data packets are different, and finally, communication abnormalities occur.

[0003] In related technologies, to ensure that the maximum payload sizes of each target link in the same physical switch are the same, target devices connected by different virtual switches need to be deployed completely the same (for example, the network cards and hard disks on the A virtual machine and the B virtual machine need to be completely the same). This method causes waste of resources and excessive redundancy, and leads to an increase in cost. SUMMARY

[0004] The present application provides a communication control method, a computer program product and a basic input output system to at least solve the problem that related technologies require each target device connected by different virtual switches to be completely the same, causing waste of resources and excessive redundancy.

[0005] The present application provides a communication control method applied to a basic input output system, and the method comprises the following steps.

[0006] In a case where a motherboard of the server is reset, a first maximum payload size of each target link is acquired; a plurality of target devices in each target link communicate with each other through a target protocol; and the target device comprises a virtual switch.

[0007] Each target link is grouped to obtain at least one link group; and the virtual switches of each target link in the same link group are connected to different ports of the same physical switch.

[0008] For each link group, a first maximum payload size with the smallest value is determined from the first maximum payload sizes of each target link included in the link group.

[0009] The second maximum payloads of the target devices in the link groups are updated according to the first maximum payloads with the smallest values of the corresponding link groups, so that the target links in the link groups communicate based on the first maximum payloads with the smallest values.

[0010] The application further provides a computer program product applied to a basic input / output system, comprising:

[0011] The first maximum payloads of the target links are acquired in the case that the motherboard of the server is reset; the target devices in the target links communicate with each other through a target protocol; and the target devices comprise virtual switches.

[0012] The first processing module is configured to group the target links to obtain at least one link group; and the virtual switches of the target links in the same link group are connected to different ports of the same physical switch.

[0013] The second processing module is configured to determine the first maximum payload with the smallest value from the first maximum payloads of the target links included in each link group.

[0014] The third processing module is configured to update the second maximum payloads of the target devices in the link groups according to the first maximum payloads with the smallest values of the corresponding link groups, so that the target links in the link groups communicate based on the first maximum payloads with the smallest values.

[0015] The application further provides a basic input / output system, which comprises the computer program product provided above.

[0016] The application further provides an electronic device, which comprises a memory configured to store a computer program and a processor configured to execute the computer program to implement the steps of any of the communication control methods.

[0017] The application further provides a non-volatile computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the steps of any of the communication control methods.

[0018] The application further provides another computer program product, which comprises a computer program, and the computer program is executed by a processor to implement the steps of any of the communication control methods.

[0019] By the present application, in the case of resetting the mainboard of the server, the first maximum payload of each target link is obtained; the target devices in each target link communicate with each other through a target protocol; the target devices include virtual switches; each target link is grouped to obtain at least one link group; the virtual switches of each target link in the same link group are connected to different ports of the same physical switch; for each link group, the first maximum payload with the minimum value is determined from the first maximum payloads of each target link included in the link group; and the second maximum payload of each target device in the link group is updated according to the first maximum payload with the minimum value corresponding to each link group, so that each target device connected by the virtual switches in each target link can communicate based on the first maximum payload with the minimum value, which can solve the problem of wasting resources and excessive redundancy caused by deploying each target device connected by different virtual switches to be exactly the same, and realize that each target link can communicate based on the same maximum payload (i.e. the first maximum payload with the minimum value) regardless of whether the target devices connected by the virtual switches in each target link are the same, which has high deployment flexibility, reduces resource waste and device redundancy, and optimizes hardware cost.

[0020] In addition, the entire communication control process is automatically performed without manual modification by technical personnel, and is flexible and convenient to operate.

[0021] In addition, although the first maximum payload with the minimum value is the minimum value among the maximum payloads of each target link, it is also the maximum payload that meets the communication requirements of all target devices in the link group, which can reduce the number of transaction layer data packets in subsequent communication and improve the utilization rate of target link bandwidth. BRIEF DESCRIPTION OF DRAWINGS

[0022] In order to more clearly illustrate the embodiments of the present application, the drawings needed in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0023] Figure 1 A hardware connection relationship schematic diagram in a server system provided by an embodiment of the present application;

[0024] Figure 2 One of the flowcharts of a communication control method provided by an embodiment of the present application;

[0025] Figure 3 A link relationship schematic diagram of each target device of a server provided by an embodiment of the present application;

[0026] Figure 4 Flowchart II of a communication control method according to an embodiment of the present application is provided.

[0027] Figure 5 Structure diagram of a computer program product according to an embodiment of the present application is provided. DETAILED DESCRIPTION

[0028] The technical solutions in the embodiments of the present application will be clearly and completely described with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art without creative work fall within the protection scope of the present application.

[0029] It should be noted that, in the description of the present application, the terms “comprise”, “contain” or any other variants thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device comprising a series of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such process, method, article or device. The terms “first”, “second” and the like in the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence.

[0030] In order for those skilled in the art to better understand the technical solutions of the present application, the present application will be further described in detail below with reference to the drawings and specific embodiments.

[0031] In combination with the specific application environment architecture or specific hardware architecture on which the execution of the communication control method depends, the specific application environment architecture or specific hardware architecture is described herein.

[0032] Referring to Figure 1 The present application provides a hardware connection relationship diagram in a server system, which can be an artificial intelligence (AI) server, including two central processing units (CPUs), namely central processing unit 0 and central processing unit 1. Both central processing unit 0 and central processing unit 1 have a slot (SLOT) for connecting a network card. The slot of central processing unit 0 is slot 0, and the slot of central processing unit 1 is slot 9.

[0033] Central processing unit 0 is connected to virtual switch A1, virtual switch A2, virtual switch B1 and virtual switch B2 through interfaces P1-1, P1-2, P1-3 and P1-4, respectively.

[0034] The central processing unit 1 is connected to the virtual switch C1, the virtual switch C2, the virtual switch D1 and the virtual switch D2 through the interfaces P2-1, P2-2, P2-3 and P2-4 respectively.

[0035] The virtual switch A1 and the virtual switch A2 are connected to different interfaces of the same physical switch A (not shown in the figure). The virtual switch B1 and the virtual switch B2 are connected to different interfaces of the same physical switch B (not shown in the figure). The virtual switch C1 and the virtual switch C2 are connected to different interfaces of the same physical switch C (not shown in the figure). The virtual switch D1 and the virtual switch D2 are connected to different interfaces of the same physical switch D (not shown in the figure).

[0036] Each virtual switch has a corresponding slot, a non-volatile memory express (NVMe) interface and a graphics processing unit (GPU). The NVMe interface is hereinafter referred to as the NVMe interface, and the NVMe interface is used to connect a solid state drive (SSD). The slot is used to deploy a standard Ethernet card or an IB card for the expansion of a GPU cluster between servers.

[0037] The interface slot 1 and the NVMe interface 1 to be used by the virtual switch A1 are connected through the interface 1 and the GPU 1;

[0038] The interface to be used by the virtual switch A2 includes the slot 2 and the NVMe interface 2, which are connected through the interface 2 and the GPU 2;

[0039] The interface to be used by the virtual switch B1 includes the slot 3 and the NVMe interface 3, which are connected through the interface 3 and the GPU 3;

[0040] The interface to be used by the virtual switch B2 includes the slot 4 and the NVMe interface 4, which are connected through the interface 4 and the GPU 4;

[0041] The interface to be used by the virtual switch C1 includes the slot 5 and the NVMe interface, which are connected through the interface 5 and the GPU 5;

[0042] The interface to be used by the virtual switch C2 includes the slot 6 and the NVMe interface 6, which are connected through the interface 6 and the GPU 6;

[0043] The interface to be used by the virtual switch D1 includes the slot 7 and the NVMe interface 7, which are connected through the interface 7 and the GPU 7;

[0044] The interface to be used by the virtual switch D2 includes the slot 8 and the NVMe interface 8, which are connected through the interface 8 and the GPU 8.

[0045] In the server infrastructure, the north-south data and the east-west data in the field of data communication are data used to describe different data flow directions and communication relationships, wherein,

[0046] The north-south data generally refers to the interactive traffic between the user and the system of the AI server (hereinafter referred to as the AI system), that is, the transmission of data between the AI system and the external user, external system or other non-AI internal components. For example, the user sends a request to the application program of the AI system for reasoning, or the AI system returns the processing result to the user, and such data flow direction is referred to as north-south traffic. In some architectures, it can also refer to the communication between the AI system and the external network or external platform, which is responsible for providing the services of the AI system to the outside world and allowing the AI system to access the services of the outside world.

[0047] The east-west data mainly refers to the communication traffic between the internal components of the AI system, such as the data transmission and interaction between the graphic processing units, the graphic processing units and the Ethernet or InfiniBand network, the servers or different microservices in the data center. For example, when a multi-modal large language model (LLM) processes text, audio, image and other data, the data synchronization and transmission between different processing modules belong to east-west communication. In the process of AI training and reasoning, a large amount of data parallel communication and collective communication are involved, such as the use of broadcast, all-to-all and all-reduce operations, which are also part of the east-west communication.

[0048] Figure 1 In the figure, the traffic between the central processing unit 0---> virtual switch A1---> graphic processing unit 1 is north-south data. Similarly, the reverse direction of the central processing unit 0<--- virtual switch A1<--- graphic processing unit 1 is also north-south data (not shown in the figure), and the traffic between the graphic processing unit 3---> virtual switch B1---> virtual switch B2---> graphic processing unit 4 is east-west data. In addition, the reverse direction of the graphic processing unit 3<--- virtual switch B1<--- virtual switch B2<--- graphic processing unit 4 is also east-west data (not shown in the figure).

[0049] In the server infrastructure, the transmission of data in the north-south direction or the east-west direction is basically through the PCIe bus transmission, and the transaction layer packet (TLP) is the basic unit for data transmission in the PCIe protocol stack.

[0050] In the PCIe protocol, the maximum payload of a target link is determined by the devices in the target link jointly, and the maximum payload in the target register of the target device specifies the maximum value of the payload of the TLP data packet sent by the target device.

[0051] The specific process of maximum payload negotiation by the target link is as follows:

[0052] (1) Initialization and capability announcement: After the server motherboard is powered on and reset, the target device will be initialized. Each target device will set a target register in its configuration space to announce the maximum payload it supports. For example, the Device Capabilities register is used to indicate the MPS size it can support, such as 128 bytes, 256 bytes, 512 bytes, etc.

[0053] (2) Link training phase: The target devices in the target link perform link initialization and configuration by sending specific training sequences. In this process, the target devices exchange their capability information, including the maximum payload, and they will select a value that both devices support based on the maximum payload announced by the other party. This value is the maximum payload of the target link.

[0054] (3) Determine the maximum payload of the target link: The target devices in the target link determine the maximum payload they jointly support, which will be used for subsequent data transmission. In the data transmission process, the size of the payload of the transaction layer data packet will not exceed the maximum payload of the target link negotiated.

[0055] The maximum payload negotiation mechanism has the feature of downward compatibility. If a new target device supports a larger maximum payload, and the old device connected to it supports a smaller maximum payload, the new device will adjust to the smaller maximum payload supported by the old device for communication. For example, a new graphics card supports a maximum payload of 4096 bytes, while the PCIe slot on the motherboard only supports a maximum payload of 2048 bytes. The graphics card will adjust its maximum payload to 2048 bytes to achieve compatible communication with the motherboard.

[0056] The application scenarios of the maximum payload mainly include the following aspects:

[0057] Improve transmission efficiency: A larger maximum payload can divide data into fewer transaction layer data packets during transmission, thereby improving the utilization rate of the target link bandwidth. For example, for large data transmission, if the maximum payload is set small, the data will be sent in multiple transaction layer data packets, increasing the transmission time and overhead. Setting a larger maximum payload allows data to be transmitted in fewer transaction layer data packets, improving efficiency.

[0058] Optimizing system performance: In systems such as servers, multiple target devices may share a physical link. Properly setting the maximum payload prevents a single device from using excessively large transaction layer packets, which could affect the performance of other devices. This helps balance communication bandwidth across devices in the system and improve overall performance.

[0059] The negotiation of the maximum payload is a sub-process of PCIe training. Figure 1 As shown, two CPUs are connected to a total of eight virtual switches. Therefore, PCIe training is performed separately on eight target links (also known as PCIe branches). Due to the varying types and number of devices attached downstream of the virtual switches, the negotiated maximum payload is independent and likely different for each of the eight target links.

[0060] The maximum payload of each target link is independent, and each target link is in the north-south direction. Therefore, there will be no inconsistency in the maximum payload during north-south data transmission. However, during east-west data transmission, that is, data transmission between target links, if the maximum payloads between the target links are different, then the data encapsulation formats of the transaction layer data packets of the two links will be different, which will lead to communication abnormalities and generate PCIe malformed TLP errors. Malformed TLP errors are an unrecoverable error type among PCIE errors and can cause serious failures such as server downtime or restart.

[0061] To ensure that the maximum payload of each target link is the same, related technologies require that target devices connected to different virtual switches be evenly distributed and deployed identically. This results in waste of resources and excessive redundancy, leading to increased costs and significant limitations.

[0062] However, in actual applications, different virtual switches may require different numbers and types of target devices due to business needs. Some other related technologies are set through manual modification. According to the principle of backward compatibility, the maximum payload negotiated by each target link is manually checked, and then the maximum payload of the central processing unit Rootport corresponding to the target link with a larger maximum payload is manually set to a smaller one. The manual modification method is relatively inflexible and requires the participation of professional technicians every time, which cannot meet the requirements of mass production.

[0063] To solve the above problems, an embodiment of the present application provides a communication control method, and the method is described in detail in conjunction with the execution process of the communication control method.

[0064] The execution subject of the communication control method provided in the embodiments of the present application can be a target firmware in a server, which can specifically be a Basic Input / Output System (BIOS).

[0065] BIOS is a firmware in a computer system, which is located in a chip on the mainboard of the computer, and is responsible for hardware initialization, hardware self-checking (POST) and booting of the operating system when the computer starts. It is the first step of computer startup and a bridge between the operating system and hardware.

[0066] In addition, the target firmware can also be other programs that can execute the communication control method, such as Unified Extensible Firmware Interface (UEFI), which is not limited.

[0067] Referring to Figure 2 The embodiments of the present application provide one of the flowcharts of the communication control method, which includes steps 210 to 240.

[0068] Step 210, in the case of resetting the mainboard of the server, obtaining the first maximum payload of each target link; the target devices in each target link communicate with each other through a target protocol; the target devices include a virtual switch.

[0069] The server of the embodiments of the present application is a computer system for supporting and running AI model training and inference tasks.

[0070] The AI model can be a large language model, such as ChatGPT, Deepseek, etc.

[0071] Unlike traditional servers, servers are usually equipped with high-performance computing resources, such as powerful central processors, graphics processors and other acceleration hardware, for processing large amounts of data and complex computing tasks.

[0072] The mainboard reset (also known as hardware reset or hardware restart) of the server refers to reinitializing and restoring the server mainboard to its initial state through hardware means, which is usually used to solve problems in the system or to reload the configuration of the mainboard.

[0073] BIOS performs firmware loading work after the server is powered on and reset, and runs the BIOS code in the memory after the target firmware is loaded into the memory.

[0074] Target link refers to a PCIe link, each target link includes a central processing unit, a virtual switch and target devices connected to the virtual switch, and the central processing unit and the virtual switch also belong to devices supporting target protocols.

[0075] The maximum payload of the target link in the embodiment of the present application is referred to as the first maximum payload.

[0076] Target devices refer to various hardware components connected to the server motherboard through the PCIe bus, such as central processing units, graphics processing units, network interface cards, solid state disks, acceleration cards, and storage controller cards.

[0077] After the motherboard of the server is reset, PCIe initialization is started, and the target devices will perform maximum payload advertisement. The maximum payload of the target devices is referred to as the second maximum payload. Each target device will set related target registers in its configuration space, and in the initialization process, the target devices will advertise the second maximum payload supported by the target registers through specific training sequences (order set sequences).

[0078] In actual applications, for example, the target register Device Capabilities register indicates that it can support the second maximum payload, such as 128 bytes, 256 bytes, 512 bytes, etc.

[0079] Target link refers to the connection path between target devices and the motherboard (such as central processing units or other devices), which is established by the PCIe bus through one or more PCIe slots (SLOT). Target link is used to transmit data from one device to another device through physical and electrical interfaces, and is usually used for high-bandwidth, high-speed data transmission.

[0080] Virtual switch (also referred to as PCIe virtual switch) is one of the core components in the virtualization environment, which complies with the PCIe protocol and provides network connection and data forwarding between virtual machines and between virtual machines and physical networks.

[0081] In the virtualization platform (such as VMware, Hyper-V, KVM), the virtual switch will usually be connected to the physical switch through physical ports. These physical ports become the “bridge” between the virtual switch and the physical network.

[0082] In the embodiment of the present application, each target device connected to the same virtual switch belongs to the same target link.

[0083] The central processing unit comprises a plurality of target ports, each target port being connected to a virtual switch, each port corresponding to a target link, and each target device on the target link sending the second maximum payload in its target register to other target devices on the target link during initialization, and updating the second maximum payload in its target register to the second maximum payload of other target devices if the second maximum payload of other target devices is less than the second maximum payload in its target register.

[0084] After receiving the second maximum payload sent by all other target devices on the target link, each target device determines that the negotiation is completed, and the second maximum payload stored in its target register is the first maximum payload of the target link.

[0085] The central processing unit creates a correspondence between the port identifier of each port and the first maximum payload of the target link corresponding to the port, and the BIOS can obtain the first maximum payload of each target link by looking up the mapping relationship.

[0086] Step 220, grouping each target link to obtain at least one link group; the virtual switches of each target link in the same link group are connected to different ports of the same physical switch.

[0087] It can be understood that the communication between target devices usually follows the "nearest principle", and the communication across physical switches will bring higher communication cost. In order to reduce the data transmission cost and improve the data transmission efficiency, when there is no other target device connected to the required resource on the virtual switch connected to the target device, the target device will obtain the required resource from other target devices connected to the "adjacent" virtual machines of the virtual switch connected to the target device. The "adjacent" virtual machines and the virtual switches connected to the target device are usually connected to different ports of the same physical switch, thereby reducing the communication cost.

[0088] If the first maximum payloads of the two target links where the two virtual switches connected to the same physical switch are located are different, the data packet data encapsulation formats of the two target links will be different when east-west data interaction is performed, which will finally cause PCIe communication exception and PCIe malformed TLP error. Malformed TLP error is an unrecoverable error type in PCIE error, which will cause serious failures such as server downtime or restart.

[0089] In order to avoid the limitations of related technologies and save resources, the embodiment of the present application groups the target links. The target links of the virtual switches connected to the same physical switch belong to a link group. A common first maximum payload needs to be determined for each link group so that each target device in the link group can communicate based on the common first maximum payload.

[0090] See also Figure 3 , an embodiment of the present application provides a schematic diagram of the link relationship of each target device of a server, including a central processing unit 0, which is connected to the network card 0 through slot 0 (not shown in the figure), and the central processing unit 0 is connected to the virtual switch A1, virtual switch A2, virtual switch B1, and virtual switch B2 through interfaces P1-1, P1-2, P1-3, and P1-4 respectively.

[0091] Virtual switches A1 and A2 are connected to the same physical switch A (not shown in the figure). The number and type of target devices connected to each virtual switch can be the same or different.

[0092] Virtual switch A1 is connected to network card 1 through slot 1 (not shown in the figure), and is connected to graphics processing unit 1 through interface 1 (not shown in the figure), and has high-speed interface 1.

[0093] Virtual switch A2 is connected to GPU 2 via interface 2 (not shown).

[0094] Virtual switches B1 and B2 belong to the same physical switch B (not shown in the figure).

[0095] The virtual switch B1 is connected to the network card 3 via the slot 3 (not shown in the figure), and is connected to the graphics processing unit 3 via the interface 3 (not shown in the figure), and has a high-speed interface 3.

[0096] Virtual switch B2 is connected to network card 4 through slot 4 (not shown in the figure) and is connected to graphics processing unit 4 through an interface.

[0097] Each virtual switch has a unique target link, for example:

[0098] The target link for virtual switch A1 is: central processing unit 0 ---> virtual switch A1 ---> network card 1, graphics processing unit 1, high-speed interface 1;

[0099] The target link for virtual switch A2 is: CPU 0 ---> virtual switch A1 ---> GPU 2.

[0100] The target link where the virtual switch B1 is located is: central processing unit 0--->virtual switch B1--->network card 3, graphics processing unit 3, high-speed interface 3;

[0101] The target link where the virtual switch B2 is located is: central processing unit 0--->virtual switch B2--->network card 4, graphics processing unit 4.

[0102] The first maximum payload of each target link is a payload determined by the target devices on the target link in common negotiation, and is not greater than the second maximum payload of any target device.

[0103] Referring to Table 1 below, the embodiments of the present application provide the MPS capabilities of the target devices in the virtual switch A1 and the virtual switch A2, and the first maximum payload after negotiation, to distinguish each target link by the identifier of the virtual switch.

[0104]

[0105] Table 1

[0106] According to the maximum payload negotiation mechanism, it needs to be downward compatible. Therefore, the first maximum payload after negotiation of the target link where the virtual switch A1 is located is 256B, and the first maximum payload after negotiation of the target link where the virtual switch A2 is located is 512B.

[0107] Step 230, for each link group, determining the first maximum payload with the minimum value from the first maximum payloads of the target links included in the link group.

[0108] Continuation Figure 3 In the embodiments, the target devices connected by the virtual switch A1 and the virtual switch A2 can perform end-to-end (P2P) communication, i.e., so-called east-west communication.

[0109] As Figure 3 the communication between the graphics processing unit 2 and the network card 1 as marked in the figure, needs to cross the virtual switch A1 and the virtual switch A2, and this application scenario is very common, because the graphics processing unit in a server is used as the core component of calculation, and the number thereof is generally fixed, and the number and type of the network card and the hard disk are configured according to the specific model size and parameter size of the user's training model. Generally, the ratio of the network card and the graphics processing unit is not 1:1. Therefore, there will be the above-mentioned situation that the graphics processing unit: network card: hard disk = 2:1:1, i.e., two graphics processing units share one network card for communication, and in this case, data communication needs to be performed across the virtual switches.

[0110] In Figure 3In the embodiment of the application, the first maximum payload of the target link where the virtual switch A1 is located is 256B after negotiation, and the first maximum payload of the target link where the virtual switch A2 is located is 512B after negotiation. Obviously, the first maximum payloads of the two are different. When the graphics processing unit 2 and the network card 1 directly communicate across the virtual switch, a PCIemalformed TLP error occurs, which further causes a serious problem of server downtime or restart.

[0111] In order to make the first maximum payloads of the target links in the same link group the same, for each link group, the maximum payload with the smallest value in the first maximum payloads of the target links in the link group can be taken as the maximum payload corresponding to the link group. Thus, whether the target devices connected by the virtual switches in the target links are the same or not, the target links where the different virtual switches connected to the same physical switch can communicate based on the same first maximum payload, i.e., based on the first maximum payload with the smallest value.

[0112] In step 240, the second maximum payloads of the target devices in the link group are updated according to the first maximum payload with the smallest value corresponding to the link group, so that the target links in the link group communicate based on the first maximum payload with the smallest value.

[0113] After the first maximum payload with the smallest value of each link group is determined, the second maximum payloads of the target devices in the link group are updated according to the first maximum payload with the smallest value, i.e., in the case where the second maximum payload in the target register of the target device is different from the first maximum payload with the smallest value, the second maximum payload in the target register is updated to the first maximum payload with the smallest value, so that the target links in the link group communicate based on the first maximum payload with the smallest value.

[0114] Continuing Figure 3 In the embodiment of the application, the first maximum payload of the target link where the virtual switch A1 is located is 256B after negotiation, and the first maximum payload of the target link where the virtual switch A2 is located is 512B after negotiation. Obviously, the first maximum payloads of the two are different. When the graphics processing unit 2 and the network card 1 directly communicate across the virtual switch, a PCIemalformed TLP error occurs, which further causes a serious problem of server downtime or restart.

[0115] The communication control method provided in the embodiments of the present application, in the case that the mainboard of the server is reset, acquires the first maximum payload of each target link; the target devices in each target link communicate with each other through a target protocol; the target devices include virtual switches; each target link is grouped to obtain at least one link group; the virtual switches of each target link in the same link group are connected to different ports of the same physical switch; for each link group, the first maximum payload with the minimum value is determined from the first maximum payloads of each target link included in the link group; and the second maximum payload of each target device in the link group is updated according to the first maximum payload with the minimum value corresponding to each link group, so that each target link in the link group communicates based on the first maximum payload with the minimum value, which can solve the problem that different virtual switches connected to each target device need to be deployed in the same way, causing waste of resources and excessive redundancy, and realize that each target link can communicate based on the same maximum payload (i.e., the first maximum payload with the minimum value) regardless of whether the target devices connected by the virtual switches in each target link are the same, has high deployment flexibility, reduces the waste of resources and the device redundancy, and optimizes the hardware cost.

[0116] In addition, the entire communication control process is automatically performed without manual modification by technical personnel, and is flexible and convenient to operate.

[0117] In addition, the first maximum payload with the minimum value, although being the minimum value among the maximum payloads of each target link, is also the maximum payload that meets the communication requirements of all target devices in the link group, which can reduce the number of transaction layer data packets in subsequent communication and improve the utilization rate of the target link bandwidth.

[0118] In some embodiments, updating the second maximum payload of each target device in the link group according to the first maximum payload with the minimum value corresponding to each link group includes:

[0119] For any target device, the second maximum payload of the target device is acquired from a target register of the target device;

[0120] In the case that the second maximum payload in the target register and the first maximum payload with the minimum value are different, the second maximum payload in the target register is updated to the first maximum payload with the minimum value;

[0121] In the case that the second maximum payload in the target register and the first maximum payload with the minimum value are the same, the target register of the target device is skipped.

[0122] The foregoing embodiments have shown that the target device records its second maximum payload in the target register, and in order to enable normal communication within the target link, the second maximum payload in the target register of the target device in each target link is less than the second maximum payload announced by other target devices.

[0123] In order to enable normal communication between each target link, the second maximum payload of each target device in the link group needs to be updated according to the first maximum payload with the smallest value in the link group to which the target link belongs, that is, in a case where it is determined that the second maximum payload in the target register is not the same as the first maximum payload with the smallest value, the second maximum payload in the target register is updated to the first maximum payload with the smallest value; in a case where it is determined that the second maximum payload in the target register is the same as the first maximum payload with the smallest value, the updating of the target register of the target device is skipped.

[0124] Therefore, whether the target devices connected by the virtual switches in each target link are the same or not, the target links in which different virtual switches connected to the same physical switch can communicate based on the same first maximum payload with the smallest value, which improves the flexibility of deployment and reduces the device redundancy within the server.

[0125] In some embodiments, after updating the second maximum payload in the target register to the first maximum payload with the smallest value, the method further comprises:

[0126] Triggering a soft reset instruction through a command line tool of the server to reset the motherboard of the server.

[0127] After the updating of the second maximum payload in the target register is completed in the embodiments of the present application, the updated second maximum payload cannot take effect immediately, and a soft reset instruction (warm reset command) needs to be initiated by the BIOS to reset the motherboard of the server, thereby re-executing the flow steps of the communication control method, that is, executing the steps 110 to 140.

[0128] After updating the second maximum payload in the target register of the target device to the first maximum payload with the smallest value in the embodiments of the present application, resetting the motherboard of the server can ensure that all configurations and updates of the server can take effect, clear potential errors, release resources, and restore the stability and performance of the system.

[0129] In some embodiments, after determining that the second maximum payload in the target register is the same as the first maximum payload with the smallest value, the method further comprises:

[0130] In the case that the second maximum payloads of the target devices are all equal to the first maximum payload with the minimum value of the link groups, a preset boot program is loaded to make the server enter the operating system.

[0131] In the case that the second maximum payloads of the target devices are all equal to the first maximum payload with the minimum value of the link groups, the second maximum payloads of the target devices are all unified, and it is determined that the self-checking is completed, and the preset boot program is loaded to make the server enter the operating system.

[0132] In conclusion, the communication control method of the embodiment of the present application occurs before the operating system of the server is started, and in the case that the second maximum payloads of the target devices are all equal to the first maximum payload with the minimum value of the link groups, the server is controlled to enter the operating system, so that the system startup failure or performance decline caused by hardware incompatibility or configuration error can be avoided, and the stability, reliability and performance of the server system are improved.

[0133] In some embodiments, after the second maximum payloads of the target devices in the link groups are updated according to the first maximum payload with the minimum value of the corresponding link groups, the method further comprises:

[0134] In the case that a new first target device is detected to be accessed, a first target link in which the first target device is located is determined.

[0135] The new first maximum payload of each target device in the first target link is obtained, which is determined again after the first target device is accessed.

[0136] In the case that the new first maximum payload is smaller than the first maximum payload before the first target device is accessed, the second maximum payloads of the target devices in the link group to which the first target link belongs are updated according to the new first maximum payload.

[0137] In actual application, some hot plug events will inevitably exist in the server, such as accessing a new target device to the server or disconnecting (or unplugging) an existing target device of the server.

[0138] It can be understood that when an existing target device is disconnected from the server, the disconnected target device will not affect the maximum payload of the link group to which the target device belongs, and will not affect the communication stability, so as to not cause the change of the first maximum payload with the minimum value of the link group.

[0139] However, when a new first target device is inserted into the server, the second maximum payload of the new first target device can be less than the first maximum payload of the first target link to which the new first target device belongs, in which case, when the new target device and other target devices of the other virtual switches in the group communicate, communication failure can occur.

[0140] Since the original first maximum payload of the first target link is already the minimum value among the second maximum payloads, after the first target device is accessed, the newly negotiated first maximum payload of each target device in the first target link is either equal to or less than the original first maximum payload.

[0141] In the case where the second maximum payload of the first target device is equal to or greater than the first maximum payload of the first target link, the new first maximum payload of the first target link group is the same as the original first maximum payload, but the second maximum payload of the first target device is updated to the original first maximum payload, and the second maximum payloads of the other target devices remain unchanged.

[0142] In the case where the second maximum payload of the first target device is less than the first maximum payload of the first target link, the first maximum payload of the first target link group to which the first target device belongs and the second maximum payloads of the other target devices in the first target link group are affected, and the second maximum payloads of the other target devices need to be updated to the second maximum payload of the first target device. Thus, even if the target devices of the server are expanded, PCIE communication anomalies caused by inconsistent first maximum payloads of target links do not occur, malformed TLP errors caused by target device expansion are avoided, and the reliability of communication is further improved.

[0143] In some embodiments, each target device is configured to split and encapsulate eastbound data to be transmitted based on the second maximum payload in the target register of the target device to obtain at least one transaction layer data packet; and transmit the at least one transaction layer data packet to other target devices.

[0144] The payload of each transaction layer data packet is not greater than the second maximum payload in the target register.

[0145] The foregoing embodiments have explained that the payload size of the transaction layer data packet (TLP) will not exceed the first maximum payload of the smallest value determined by negotiation, so each target device can split the east-west data to be sent based on the second maximum payload in the target register of the target device, and encapsulate each split data through the PCIe protocol to obtain each transaction layer data packet, and send at least one transaction layer data packet to other target devices through the PCIe connection, which can reduce the number of TLP data packets in subsequent communication and improve the utilization rate of the target link bandwidth.

[0146] In some embodiments, each target device includes an accelerator and a network card, and each virtual switch is connected to at least one accelerator and one network card; the accelerators connected to different virtual machines are accessed remotely by the network cards connected to the corresponding virtual machines.

[0147] The target device of the embodiments of the present application includes an accelerator, which includes at least one of a graphics processing unit, a TPU, an FPGA, an NPU, and the like, and the following will be described by taking the graphics processing unit as an example.

[0148] The server of the embodiments of the present application supports the remote direct memory access (RDMA) technology, which allows the opposite network card to directly access the memory of the opposite system.

[0149] Based on the remote direct memory access technology, the network card can directly communicate with the remote graphics processing unit, and the central processing unit participates in the control path, the transmission queue preparation, the control mechanism before and after transmission, and the east-west data can be directly sent from the graphics processing unit display memory to the RDMA network card. The network card supporting the RDMA technology at the opposite end directly transmits the east-west data to the opposite end graphics processing unit display memory after receiving the east-west data.

[0150] In the embodiments of the present application, based on the RDMA technology, one accelerator can directly read and write data from the display memory of another accelerator, which reduces the participation of the central processing unit and the action of moving / copying data, greatly reduces the cross-machine communication delay, and improves the data throughput.

[0151] In some embodiments, each target device sends the second maximum payload in the target register of the target device to other target devices on the target link during initialization, and updates the second maximum payload in the target register of the target device to the second maximum payload of the other target device if it is determined that the second maximum payload of the other target device is smaller than the second maximum payload of the target device.

[0152] The foregoing embodiments have explained this, and will not be described here again.

[0153] It is worth noting that the target device updates the second maximum payload in the target register of itself to the second maximum payload of other target devices in the initialization process, and also in the process of determining the first maximum payload of the target link step by step, which is "actively" occurred, can ensure that the communication between each target device in the target link is based on the same second target maximum payload, meet the payload requirements of all target devices, and also can reduce the number of transaction layer data packets in subsequent communication, and improve the utilization rate of the target link bandwidth.

[0154] In some embodiments, the target link includes a central processing unit, a virtual switch, and a plurality of target devices connected by the virtual switch; the plurality of target devices includes at least part of a graphics processing unit, a network card, and a hard disk.

[0155] The foregoing embodiments have been described, and here will not be repeated.

[0156] Referring to Figure 4 The embodiments of the present application provide a flowchart of a communication control method, which includes the following steps:

[0157] Step 401, power on and reset of the server;

[0158] Step 402, target device initialization and second maximum payload announcement;

[0159] Step 403, the target device in the target link determines the first maximum payload of the target link according to the second maximum payload of itself and other target devices;

[0160] Step 404, the basic input and output system obtains the first maximum payload of each target link in turn;

[0161] Step 405, the basic input and output system groups each target link to obtain at least one link group; the virtual switches of each target link in the same link group are connected to different ports of the same physical switch;

[0162] Step 406, the basic input and output system determines the first maximum payload with the smallest value from the first maximum payload of each target link included in each link group; step 407 or step 408 is executed;

[0163] Step 407, the basic input and output system updates the second maximum payload in the target register of any target device to the first maximum payload with the smallest value in the link group to which the target device belongs, in the case that the second maximum payload in the target register of the target device is different from the first maximum payload with the smallest value in the link group; step 410 is executed;

[0164] Step 408, the basic input / output system performs self-checking in the case that the second maximum payload in the target register of any target device and the first maximum payload with the minimum value in the corresponding link group are the same; and step 409 is performed;

[0165] Step 409, the basic input / output system loads a preset boot program to make the server enter an operating system in the case that the self-checking is successful;

[0166] Step 410, the basic input / output system triggers a soft reset instruction through a command line tool of the server to reset the mainboard of the server in the case that the second maximum payload of each target device is updated; and step 402 is performed.

[0167] The detailed execution process of steps 401 to 410 is described in the foregoing embodiment, which will not be repeated here.

[0168] Through the foregoing description of the embodiments, those skilled in the art can clearly understand that the method according to the foregoing embodiments can be realized by means of software and a necessary general hardware platform, and of course, it can also be realized by hardware, but in many cases, the former is a better embodiment.

[0169] Referring to Figure 5 The embodiment of the present application also provides a structural schematic diagram of a computer program product applied to a basic input / output system, which comprises an acquisition module 510, a first processing module 520, a second processing module 530 and a third processing module 540.

[0170] The acquisition module 510 is configured to acquire the first maximum payload of each target link in the case that the mainboard of the server is reset; the target devices in each target link communicate with each other through a target protocol; and the target devices comprise virtual switches.

[0171] The first processing module 520 is configured to group the target links to obtain at least one link group; and the virtual switches of the target links in the same link group are connected to different ports of the same physical switch.

[0172] The second processing module 530 is configured to determine the first maximum payload with the minimum value from the first maximum payloads of the target links in each link group.

[0173] The third processing module 540 is configured to update the second maximum payload of each target device in the link group according to the first maximum payload with the minimum value corresponding to each link group, so that the target links in the link group communicate with each other based on the first maximum payload with the minimum value.

[0174] The computer program product provided in the embodiments of the present application can obtain the first maximum payload of each target link in the case of resetting the mainboard of the server, the target devices in each target link communicate with each other through a target protocol, the target devices include virtual switches, each target link is grouped to obtain at least one link group, the virtual switches of each target link in the same link group are connected to different ports of the same physical switch, and the first maximum payload with the minimum value is determined from the first maximum payloads of each target link included in each link group. The second maximum payload of each target device in the link group is updated according to the first maximum payload with the minimum value corresponding to each link group, so that each target device connected by the different virtual switches of the different ports of the same physical switch can communicate based on the same maximum payload (i.e., the first maximum payload with the minimum value), which has high deployment flexibility, reduces the waste of resources and the device redundancy, and optimizes the hardware cost.

[0175] In addition, the entire communication control process is automatically performed without manual modification by technical personnel, and is flexible and convenient to operate.

[0176] In addition, the first maximum payload with the minimum value is the minimum value in the maximum payloads of each target link, but is also the maximum payload that meets the communication requirements of all target devices in the link group, which can reduce the number of transaction layer data packets in subsequent communication and improve the utilization rate of the target link bandwidth.

[0177] In some embodiments, the computer program product further includes:

[0178] The fourth processing module is configured to:

[0179] In the case of detecting that a new first target device accesses, the first target link in which the first target device is located is determined.

[0180] The new first maximum payload of each target device in the first target link is obtained after the first target device accesses.

[0181] In the case that the new first maximum payload is smaller than the first maximum payload before the first target device accesses, the second maximum payload of each target device in the link group to which the first target link belongs is updated according to the new first maximum payload.

[0182] In some embodiments, the third processing module 540 is specifically configured to:

[0183] For any one target device, the second maximum payload of the target device is obtained from a target register of the target device;

[0184] In a case where the second maximum payload in the target register and the first maximum payload with the smallest value are determined to be different, the second maximum payload in the target register is updated to the first maximum payload with the smallest value;

[0185] In a case where the second maximum payload in the target register and the first maximum payload with the smallest value are determined to be the same, updating the target register of the target device is skipped.

[0186] In some embodiments, the computer program product further comprises:

[0187] The restart module is configured to trigger a soft restart instruction through a command line tool of the server, so as to reset the mainboard of the server.

[0188] In some embodiments, the computer program product further comprises:

[0189] The loading module is configured to, in a case where the second maximum payload of each target device is determined to be the same as the first maximum payload with the smallest value of the link group to which the target device belongs, load a preset boot program, so that the server enters an operating system.

[0190] Each target device sends the second maximum payload in the target register of the target device to other target devices on the target link to which the target device belongs, and in a case where the obtained second maximum payload of the other target devices is determined to be smaller than the second maximum payload of the target device, the second maximum payload in the target register of the target device is updated to the second maximum payload of the other target devices.

[0191] In some embodiments, each target device is configured to, based on the second maximum payload in the target register of the target device, perform segmentation processing and encapsulation processing on eastbound data to be sent, to obtain at least one transaction layer data packet; and send the at least one transaction layer data packet to other target devices.

[0192] The payload of each transaction layer data packet is not greater than the second maximum payload in the target register.

[0193] In some embodiments, the target link comprises a central processing unit, a virtual switch, and a plurality of target devices connected to the virtual switch; and the plurality of target devices comprise at least part of a graphics processing unit, a network card, and a hard disk.

[0194] The description of the features in the embodiments of the computer program product can refer to the description of the embodiments of the communication control method, which will not be repeated here.

[0195] The embodiments of the present application also provide an electronic device, comprising a memory and a processor, the memory storing a computer program, and the processor being configured to execute the computer program to perform the steps in any of the embodiments of the communication control method.

[0196] The embodiments of the present application also provide a basic input / output system, which comprises the computer program product provided above.

[0197] The embodiments of the present application also provide a non-volatile computer readable storage medium, which stores a computer program, and the computer program is configured to perform the steps in any of the embodiments of the communication control method when executed.

[0198] In an example embodiment, the computer readable storage medium can include, but is not limited to, a U disk, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk, and various media that can store computer programs.

[0199] The embodiments of the present application also provide another computer program product, which comprises a computer program, and the computer program is executed by a processor to perform the steps in any of the embodiments of the communication control method.

[0200] The embodiments of the present application also provide another computer program product, which comprises a non-volatile computer readable storage medium, and the non-volatile computer readable storage medium stores a computer program, and the computer program is executed by a processor to perform the steps in any of the embodiments of the communication control method.

[0201] The skilled person can further realize that the units and algorithm steps of the examples described in conjunction with the embodiments disclosed herein can be realized in electronic hardware, computer software or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the components and steps of the examples have been described in general terms in the above description. Whether the functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. The skilled person can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0202] The above describes in detail a communication control method, a computer program product and a basic input / output system provided by the present application. The principles and implementation manners of the present application are described by using specific examples in the present application, and the above description of the examples is only applicable to help understand the method of the present application and the core idea thereof. It should be noted that, for those skilled in the art, without departing from the principles of the present application, some improvements and modifications can be made to the present application, and these improvements and modifications also fall within the protection scope of the claims of the present application.

Claims

1. A communication control method characterized by comprising: The application is applied to a basic input / output system, comprising: In the case of resetting the motherboard of the server, the first maximum payload of each target link is obtained; the target devices in each target link communicate with each other through a target protocol; the target devices include a virtual switch; Each target link is grouped to obtain at least one link group; the virtual switches of the target links in the same link group are connected to different ports of the same physical switch; For each link group, the first maximum payload with the minimum value is determined from the first maximum payloads of the target links included in the link group; According to the first maximum payload with the minimum value of each link group, the second maximum payload of each target device in the link group is updated, so that the target links in the link group communicate based on the first maximum payload with the minimum value; According to the first maximum payload with the minimum value of each link group, the second maximum payload of each target device in the link group is updated, comprising: For any target device, the second maximum payload of the target device is obtained from the target register of the target device; In the case where the second maximum payload in the target register and the first maximum payload with the minimum value are determined to be different, the second maximum payload in the target register is updated to the first maximum payload with the minimum value; In the case where the second maximum payload in the target register and the first maximum payload with the minimum value are determined to be the same, the updating of the target register of the target device is skipped.

2. The communication control method according to claim 1, characterized by, After the updating of the second maximum payload of each target device in the link group according to the first maximum payload with the minimum value of each link group, the method further comprises: In the case of detecting the access of a new first target device, a first target link where the first target device is located is determined; The new first maximum payload of each target device in the first target link after the access of the first target device is obtained; In the case where the new first maximum payload is smaller than the first maximum payload before the access of the first target device, the second maximum payload of each target device in the link group to which the first target link belongs is updated according to the new first maximum payload.

3. The communication control method according to claim 1, characterized by, After the updating of the second maximum payload in the target register to the first maximum payload with the minimum value, the method further comprises: A soft reset instruction is triggered through a command line tool of the server to reset the motherboard of the server.

4. The communication control method according to claim 1, characterized by, After the determination that the second maximum payload in the target register and the first maximum payload with the minimum value are the same, the method further comprises: In the case where the second maximum payload of each target device is determined to be the same as the first maximum payload with the minimum value of the link group to which the target device belongs, a preset boot program is loaded to make the server enter an operating system.

5. The communication control method according to any one of claims 1 to 4, characterized by, Each target device sends the second maximum payload in its target register to other target devices on the target link to which it belongs, and updates the second maximum payload in its target register to the second maximum payload of the other target device if it is determined that the acquired second maximum payload of the other target device is less than its own second maximum payload.

6. The communication control method according to claim 1, characterized by, Each target device is configured to perform segmentation processing and encapsulation processing on eastbound data to be sent based on the second maximum payload in its target register, to obtain at least one transaction layer data packet; and send the at least one transaction layer data packet to other target devices. The payload of each transaction layer data packet is not greater than the second maximum payload in the target register.

7. The communication control method according to any one of claims 1 to 4, characterized by, The target link includes a central processing unit, a virtual switch, and a plurality of target devices connected to the virtual switch; and the plurality of target devices include at least part of a graphics processing unit, a network card, and a hard disk.

8. A computer program product, characterised in that, The application is applied to a basic input / output system, and includes: An acquisition module is configured to acquire first maximum payloads of each target link in the case of resetting a mainboard of a server; a plurality of target devices in each target link communicate with each other through a target protocol; and the target devices include a virtual switch. A first processing module is configured to group the target links to obtain at least one link group; and virtual switches of target links in a same link group are connected to different ports of a same physical switch. A second processing module is configured to determine, for each link group, a first maximum payload with a minimum value from first maximum payloads of target links included in the link group. A third processing module is configured to update second maximum payloads of target devices in the link group based on the first maximum payload with the minimum value corresponding to each link group, so that the target links in the link group communicate with each other based on the first maximum payload with the minimum value. The third processing module is specifically configured to: For any target device, acquire the second maximum payload of the target device from a target register of the target device. If it is determined that the second maximum payload in the target register and the first maximum payload with the minimum value are different, update the second maximum payload in the target register to the first maximum payload with the minimum value. If it is determined that the second maximum payload in the target register and the first maximum payload with the minimum value are the same, skip updating the target register of the target device.

9. The computer program product of claim 8, wherein, The computer program product further includes: A fourth processing module is configured to: In the case of detecting that a new first target device is accessed, determine a first target link on which the first target device is located; Acquire new first maximum payloads of target devices in the first target link that are determined again after the first target device is accessed; and In the case of detecting that a new first target device is accessed, determine a first target link on which the first target device is located; Acquire new first maximum payloads of target devices in the first target link that are determined again after the first target device is accessed; and In a case that the new first maximum payload is less than the first maximum payload before the first target device accesses, updating second maximum payloads of target devices in a link group to which the first target link belongs according to the new first maximum payload.

10. The computer program product of claim 8, wherein, The computer program product further comprises: The loading module is configured to load a preset guide program to make the server enter an operating system in a case that the second maximum payloads of the target devices are all the same as the first maximum payload of the link group with the smallest value.

11. A basic input / output system, characterized by, The basic input output system comprises the computer program product according to any one of claims 9-10.

12. An electronic device, comprising: The computer program product comprises: The memory is configured to store a computer program; The processor is configured to implement the steps of the communication control method according to any one of claims 1-7 when executing the computer program.

13. A non-transitory computer readable storage medium, comprising: The non-volatile computer readable storage medium stores a computer program, and the computer program is executed by the processor to implement the steps of the communication control method according to any one of claims 1-7.

Images