Route control method, equipment and system for multi-GPU (Graphics Processing Unit) communication in virtual environment

Abstract

The invention discloses a routing control method, device and system for multi-GPU communication in a virtual environment, and the method comprises the steps: responding to an inter-GPU communication instruction through a first software control logic of a first virtual machine imported into a GPU card, when a point-to-point communication condition is met, sending data to a PCIE switch, and enabling the PCIE switch to carry out the routing control through a modified first data, directly sending to an opposite end GPU card of the first virtual machine; and in response to a cross-virtual-machine communication instruction through second software control logic of the first virtual machine imported into the network card and modified second data routing control, data is subjected to device address conversion and then sent to the PCIE switch, and the PCIE switch directly sends the data to a GPU card of the second virtual machine through modified third data routing control. Therefore, the problem that point-to-point communication between GPUs in a virtual environment and GPU remote direct memory access across virtual machines are limited is solved.

Description

Technical Field

[0001] The present application relates to the field of virtual machine technology, and in particular to a routing control method, device and system for multi-machine GPU communication in a virtual environment. Background Art

[0002] The new wave of AI, cloud computing and cloud services that has emerged in the past decade has changed human society in industry, transportation, education, entertainment, life and even the military. At present, related research is in the ascendant, products are booming, and the implementation scenarios are diverse. Related intelligent applications continue to maintain a high-speed development trend. This means that there is a stronger demand for server computing power, and at the same time, it also poses new challenges to server performance and stability. As the protagonist of AI, cloud computing and cloud services, the role of GPU servers is self-evident.

[0003] In the fields of general computing and graphics rendering, the two major application scenarios of GPU (Graphics Processing Unit) servers have a high demand for virtualization in today's hot cloud computing and cloud services. Especially in the field of general computing, GPU virtualization can be used to achieve computing power pooling and sharing to achieve the ultimate utilization of computing power. This involves a problem, the point-to-point communication between GPUs in a virtualized environment, and the implementation method of direct memory access between GPUs in two virtual machines.

[0004] For the model topology where both the GPU card and the network card that supports remote memory direct access are mounted under the PCIe Switch, the point-to-point communication between GPUs in the virtual environment and the remote direct memory access of GPUs across virtual machines are limited. Summary of the invention

[0005] The purpose of this application is to provide a routing control method, device and system for multi-machine GPU communication in a virtual environment, which is used to solve the problem that the existing model topology in which both GPU cards and network cards supporting remote memory direct access are mounted under PCIe Switch, and the point-to-point communication between GPUs in a virtual environment and the remote direct memory access of GPUs across virtual machines are limited.

[0006] In a first aspect, an embodiment of the present application provides a routing control method for multi-machine GPU communication in a virtual environment, the method comprising:

[0007] By importing the first software control logic of the first virtual machine of the GPU card, in response to the inter-GPU communication instruction, when it is determined that the point-to-point communication condition is met, the data is sent to the PCIE switch in a point-to-point communication manner, and the PCIE switch directly sends the received data to the opposite GPU card of the first virtual machine without passing through the CPU of the PCIE switch through the modified first data routing control;

[0008] By importing the second software control logic of the first virtual machine of the network card, in response to the cross-virtual machine communication instruction, the data is sent to the PCIE switch after device address conversion through the modified second data routing control, and the PCIE switch sends the received data directly to the GPU card of the second virtual machine without passing through the CPU of the PCIE switch through the modified third data routing control.

[0009] The embodiments of the present application can realize overall point-to-point communication between GPUs in the same virtual machine, as well as remote direct memory access between GPUs of multiple virtual machines, laying the foundation for training, reasoning and other high-performance computing, thereby solving the problem that the existing model topology in which both the GPU card and the network card supporting remote direct memory access are mounted under the PCIE switch, and the point-to-point communication between GPUs in the virtual environment and the remote direct memory access of GPUs across virtual machines are limited.

[0010] In some possible embodiments, the modified first data routing control is implemented in the following manner:

[0011] Modify the state of each state bit of the first data routing control register to a fully closed state to obtain a modified first data routing control;

[0012] The first data routing control register is a register in the PCIE switch corresponding to the GPU card of the first virtual machine.

[0013] By setting the first data routing controller as described above, it can be achieved that, for the GPU card, the data routing of its uplink in the virtual machine is not affected by the state of the data routing control register of the physical host.

[0014] In some possible embodiments, the modified third data routing control is implemented in the following manner:

[0015] Modify the direct communication indication state bit in the third data routing control register to an enabled state, thereby obtaining a modified third data routing control;

[0016] The third data routing control register is a register in the PCIE switch corresponding to the network card of the first virtual machine.

[0017] The direct communication indication state bit in the third data routing control register is modified to an enabled state, so that the data sent by the first virtual machine imported into the network card reaches the PCIE switch and can be directly sent to the network card of the second virtual machine without going through the memory.

[0018] In some possible embodiments, when it is determined that the point-to-point communication condition is met, sending the data to the PCIE switch in a point-to-point communication manner includes:

[0019] Obtaining device group information in a group register, wherein the device group information is information defined by a simulator of the first virtual machine, the device group information includes a device group ID, and an address of the group register has a specified offset in the address space;

[0020] When it is determined that the device group ID of the GPU card on the other end is consistent with the obtained device group ID, it is determined that the point-to-point communication condition is met;

[0021] Through the fourth data routing control modified at the virtual machine level, the data is sent to the PCIE switch in a point-to-point communication manner.

[0022] The embodiment of the present application modifies the underlying firmware logic of the virtual machine into which the GPU network card is imported, and through the obtained device group information, enables it to have the ability to determine whether point-to-point communication is supported, and through data routing control at the virtual machine level, realizes that data can be sent in a point-to-point manner.

[0023] In some possible embodiments, the device group information further includes point-to-point capability information, device identification and verification information, and when it is determined that the device group ID of the opposite GPU card is consistent with the acquired device group ID, it is determined that the point-to-point communication condition is met, including:

[0024] According to the point-to-point capability information, it is determined that the GPU card of the first virtual machine supports point-to-point communication, and according to the device identification and verification information, the opposite GPU card is identified and verified. When it is determined that the verification passes and the device group ID of the opposite GPU card is consistent with the obtained device group ID, it is determined that the point-to-point communication conditions are met.

[0025] The point-to-point capability information in the above device group information can be used to determine whether point-to-point communication is supported, and the peer GPU card can be identified and verified through the above device identification and verification information, thereby supporting point-to-point communication more comprehensively and accurately.

[0026] In some possible embodiments, the modified second data routing control is used to send the data to the PCIE switch after the device address is converted, including:

[0027] Scan all PCIE downstream bridge locations of the first virtual machine into which the GPU card is imported;

[0028] If a peer GPU card is scanned, the routing control function of the bridge position corresponding to the peer GPU card scanned in the second data routing control register is turned off to obtain a modified second data routing control;

[0029] The modified second data routing control is adopted to send the data to the PCIE switch after the device address is converted.

[0030] The embodiment of the present application can scan the PCIE downstream bridge position through the underlying logic of the firmware, and turn off the routing control of the GPU card, thereby realizing routing control at the software level, thereby directly sending data to the PCIE switch to support point-to-point communication between GPUs.

[0031] In some possible embodiments, disabling the routing control function of the bridge position corresponding to the GPU card scanned in the second data routing control register includes:

[0032] According to the routing control option added to the status bit of each bridge position in the second data routing control register, the routing control option of the bridge position corresponding to the scanned opposite-end GPU card is set to be closed.

[0033] In the embodiment of the present application, in the virtual machine UEFI firmware, a routing control option is added to the status bit of each bridge position in the second data routing control register, which corresponds to the downstream routing control register, thereby realizing interface normalization of routing control, and realizing standardized adjustment of routing control registers by adjusting the routing control option value without having to operate the register.

[0034] In some possible embodiments, sending the data to the PCIE switch after the device address is converted includes:

[0035] By modifying the register value in the address conversion register of the network card, the address conversion function is turned on;

[0036] Based on the address conversion register, the OS layer address when the second virtual machine sends data is converted into the physical layer address of the network card, and the data is sent to the PCIE switch through the physical layer address.

[0037] The embodiment of the present application modifies the address conversion register value so that the network card supports data address conversion, thereby ensuring normal data transmission.

[0038] In a second aspect, an embodiment of the present application provides a routing control system for multi-machine GPU communication in a virtual environment, including a PCIE switch, a first virtual machine mounted under the PCIE switch and importing different GPU cards, a first virtual machine mounted under the PCIE switch and importing a network card, a second virtual machine mounted under the PCIE switch and importing different GPU cards, and a second virtual machine mounted under the PCIE switch and importing a network card, wherein:

[0039] A first virtual machine imported into the GPU card is used to respond to the inter-GPU communication instruction through the first software control logic, and when it is determined that the point-to-point communication condition is met, send the data to the PCIE switch in a point-to-point communication manner, and the PCIE switch controls the received data directly to the opposite GPU card of the first virtual machine without passing through the CPU of the PCIE switch through the modified first data routing control;

[0040] The first virtual machine into which the network card is imported is used to respond to the cross-virtual machine communication instruction through the second software control logic, and send the data to the PCIE switch after the device address is converted through the modified second data routing control. The PCIE switch sends the received data directly to the GPU card of the second virtual machine without passing through the CPU of the PCIE switch through the modified third data routing control.

[0041] In the third aspect, another embodiment of the present application further provides a computer storage medium, which stores a computer program, and the computer program is used to enable a computer to execute the first software control logic of the first virtual machine imported into the GPU card in any method provided in the embodiments of the present application, or to execute the second software control logic of the first virtual machine imported into the network card in any method provided in the embodiments of the present application.

[0042] Other features and advantages of the present application will be described in the following description, and partly become apparent from the description, or be understood by practicing the present application. The purpose and other advantages of the present application can be realized and obtained by the structures specifically pointed out in the written description, claims, and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments of the present application will be briefly introduced below. Obviously, the drawings introduced below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0044] Figure 1 A schematic diagram of a routing control system for multi-machine GPU communication in a virtual environment provided according to an embodiment of the present application;

[0045] Figure 2 A schematic flow chart of a routing control method for multi-machine GPU communication in a virtual environment provided according to an embodiment of the present application;

[0046] Figure 3 A schematic diagram of a process for modifying a related data routing control register according to an embodiment of the present application;

[0047] Figure 4 A detailed flow chart of a routing control method for multi-machine GPU communication in a virtual environment provided according to an embodiment of the present application;

[0048] Figure 5 This is a structural diagram of a routing control device for multi-machine GPU communication in a virtual environment provided according to an embodiment of the present application. DETAILED DESCRIPTION

[0049] To further illustrate the technical solution provided by the embodiment of the present application, this is described in detail below in conjunction with the accompanying drawings and specific implementation methods. Although the embodiment of the present application provides the method operation steps as shown in the following embodiments or drawings, more or fewer operation steps may be included in the method based on routine or no creative labor. In the steps where there is no necessary causal relationship logically, the execution order of these steps is not limited to the execution order provided by the embodiment of the present application. The method can be executed in the order of the method shown in the embodiment or drawings or in parallel during the actual processing process or when the control device is executed.

[0050] The technical terms involved in the embodiments of the present application are explained below.

[0051] 1) GPU (Graphics Processing Unit) is a microprocessor specially designed to handle graphics and image computing tasks. It has a large number of parallel computing units and is very suitable for performing large-scale parallel computing tasks, not only limited to graphics rendering, but also including scientific computing, machine learning, deep learning and other high-performance computing fields.

[0052] 2) Virtualization is a computer technology that abstracts physical resources to create and run multiple independent virtual environments on a single physical platform. These virtual environments are usually called VMs (Virtual Machines). Virtualization technology can be applied to a variety of resources, including but not limited to computing, storage, and network resources.

[0053] 3) NVLink is a high-speed interconnect technology developed by NVIDIA to connect multiple NVIDIA GPUs for efficient point-to-point communication and data transmission. The goal of NVLink design is to provide higher bandwidth and lower latency than traditional PCIe interfaces, thereby significantly improving the data exchange speed between GPUs, which is particularly important for high-performance computing, deep learning, and graphics-intensive applications.

[0054] 4) PCIE (PCI Express) is a high-speed serial computer expansion bus standard used to connect various components inside the computer, such as graphics cards, network adapters, storage devices, etc.

[0055] 5) P2P communication (Point-to-Point Communication) is a data communication method in which two or more communication entities (such as GPU cards, computer machines, network devices or software processes) directly establish communication links to achieve direct data transmission. This communication method usually does not require forwarding of system memory, which can reduce communication delays and improve communication efficiency.

[0056] 6) RDMA (Remote Direct Memory Access) is a network communication technology that allows one node to directly access the memory of another node without going through the receiving node's central processing unit (CPU). The main advantage of RDMA technology is that it can significantly reduce data transmission latency and CPU load, and improve data transmission throughput and efficiency.

[0057] 7) Data routing control, also known as PCIE access control service, is a security mechanism defined in the PCIE specification that aims to restrict access to the configuration space of PCIE devices. The main goal of ACS (Access Control Server) is to prevent unauthorized access and malicious operations, thereby protecting the security and integrity of PCIE devices.

[0058] 8) PCIE Switch (PCI Express Switch) is a network device used to manage and extend PCIE signals in the PCIE architecture. PCIE Switches allow PCIE signals to be distributed between a host port and multiple downstream ports, thereby enabling connection and communication between PCIE devices. This technology enables a single PCIE port to be connected to multiple PCIE devices, improving the flexibility and scalability of the system.

[0059] 9) OVMF (Open Virtual Machine Firmware) is an open source UEFI (Unified Extensible Firmware Interface) firmware implementation designed specifically for virtualized environments. OVMF is jointly maintained by the UEFI Alliance and the Linux Foundation, and aims to provide a common UEFI firmware foundation for virtual machines, allowing virtual machines to start and run on various virtualization platforms.

[0060] For server models with NVLink, GPU point-to-point communication in a virtual environment is relatively easy to implement, but GPU remote direct memory access across virtual machines also has problems. The main reason for the current limited communication problem is that the routing control on the physical host side cannot be fully synchronized to the virtual machine, and some GPU devices do not support address translation. In view of the problem of limited point-to-point communication between GPUs in a virtual environment and limited GPU remote direct memory access across virtual machines in the related art, an embodiment of the present application provides a routing control method and device for multi-machine GPU point-to-point communication in a virtual environment. It is mainly aimed at the model topology in which both the GPU card and the network card that supports remote direct memory access are mounted under the PCIE Switch. The data routing control register involved is located on the uplink of the card side, that is, the downlink of the PCIe Switch. For the topology in which the GPU card and the network card are directly connected to the root port of the server motherboard CPU, there is no need to perform such related settings.

[0061] Other features and advantages of the present application will be described in the following description, and partly become apparent from the description, or be understood by practicing the present application. The purpose and other advantages of the present application can be realized and obtained by the structures specifically pointed out in the written description, claims, and drawings.

[0062] like Figure 1 The multi-machine GPU point-to-point communication system in the virtual environment provided by this application is shown, which mainly includes:

[0063] A PCIE switch, a first virtual machine mounted under the PCIE switch and importing different GPU cards, a first virtual machine mounted under the PCIE switch and importing a network card, a second virtual machine mounted under the PCIE switch and importing different GPU cards, and a second virtual machine mounted under the PCIE switch and importing a network card.

[0064] A virtual machine may import multiple different GPUs and mount them under a PCIE switch. When the virtual machine implements the corresponding business functions, it involves point-to-point communication between GPUs of the same virtual machine, communication between the GPU of the same virtual machine and the network card, and communication between the network card and the GPU. For cross-virtual machine communication scenarios, communication between GPUs of multiple virtual machines is achieved through the network card. The specific communication method is from the GPU of the first virtual machine to the network card of the first virtual machine, from the network card of the first virtual machine to the network card of the second virtual machine, and from the network card of the second virtual machine to the GPU of the second virtual machine.

[0065] A first virtual machine imported into the GPU card is used to respond to the inter-GPU communication instruction through the first software control logic, and when it is determined that the point-to-point communication condition is met, send the data to the PCIE switch in a point-to-point communication manner, and the PCIE switch controls the received data directly to the opposite GPU card of the first virtual machine without passing through the CPU of the PCIE switch through the modified first data routing control;

[0066] The first virtual machine into which the network card is imported is used to respond to the cross-virtual machine communication instruction through the second software control logic, and send the data to the PCIE switch after the device address is converted through the modified second data routing control. The PCIE switch sends the received data directly to the GPU card of the second virtual machine without passing through the CPU of the PCIE switch through the modified third data routing control.

[0067] The routing control method and device for multi-GPU point-to-point communication in a virtual environment in an embodiment of the present application are described in detail below with reference to the accompanying drawings.

[0068] like Figure 2 As shown, an embodiment of the present application provides a routing control method for multi-machine GPU communication in a virtual environment, the method comprising:

[0069] Step 201, by importing the first software control logic of the first virtual machine of the GPU card, responding to the inter-GPU communication instruction, when it is determined that the point-to-point communication condition is met, the data is sent to the PCIE switch in a point-to-point communication manner, and the PCIE switch controls the first modified data routing to directly send the received data to the opposite GPU card of the first virtual machine without passing through the CPU of the PCIE switch;

[0070] When point-to-point direct access is to be achieved between different GPU cards of the same virtual machine, it is necessary to modify the first data routing control of the PCIE exchange to release the mandatory control of the data flow direction by the host side, i.e., the PCIE switch. In the embodiment of the present application, for the PCIE switch, the modified first data routing control is obtained by modifying the value of the state bit of the data routing control register corresponding to the first virtual machine imported into the GPU card. Therefore, for the GPU card, within the virtual machine, the data routing of its uplink is not completely affected by the state of the data routing control register of the physical host.

[0071] In the embodiment of the present application, the communication between the GPU card and the network card of the same virtual machine can also be implemented in the same way to achieve point-to-point communication. That is, by importing the first software control logic of the first virtual machine of the GPU card, responding to the communication instruction between GPUs, when it is determined that the point-to-point communication conditions are met, the data is sent to the PCIE switch in a point-to-point communication manner, and the PCIE switch is controlled by the modified first data routing to directly send the received data to the opposite end network card of the first virtual machine without passing through the CPU of the PCIE switch.

[0072] In the embodiment of the present application, the first virtual machine into which the GPU card is imported involves two layers of operating systems OS, wherein the first layer of OS is the operating system OS of the GPU card, and the other layer of OS is the operating system OS at the virtual machine level. Therefore, in order to realize communication between the GPU cards of the same virtual machine, it is necessary to modify the open source simulator and the open source UEFI firmware logic of the first virtual machine.

[0073] In the embodiment of the present application, based on modifying the data routing control register of the corresponding PCIE switch so that the PCIE switch supports directly sending data to the peer GPU card or peer network card of the first virtual machine without passing through the CPU of the PCIE switch, it is necessary to further modify the open source simulator and open source UEFI firmware logic of the first virtual machine, so that the first virtual machine imported into the GPU card adopts the first software control logic, responds to the inter-GPU communication instruction, and sends the data to the PCIE switch in a point-to-point communication manner when it is determined that the point-to-point communication conditions are met.

[0074] The first virtual machine and the second virtual machine are two different virtual machines under PCIE. The solution applied to the first virtual machine can also be applied to the second virtual machine. Similarly, the solution applied to the second virtual machine can also be applied to the first virtual machine.

[0075] Step 202, by importing the second software control logic of the first virtual machine of the network card, in response to the cross-virtual machine communication instruction, the data is sent to the PCIE switch after the device address is converted through the modified second data routing control, and the PCIE switch sends the received data directly to the GPU card of the second virtual machine without passing through the CPU of the PCIE switch through the modified third data routing control.

[0076] When different virtual machines implement direct memory access, each virtual machine has a CPU card and a network card. If the GPU data of the first virtual machine is sent to the GPU of the second virtual machine, and the network card and the GPU card are under the same PCIE switch at the physical level, the two virtual machines implement direct access to remote memory through the network card in the following manner: the GPU card of the first virtual machine sends data to the network card of the first virtual machine, the network card of the first virtual machine to the network card of the second virtual machine, and the network card of the second virtual machine to the GPU card of the second virtual machine. Since the network card of the second virtual machine is to the second virtual machine. Therefore, the specific manner in which the PCIE switch sends the received data directly to the GPU card of the second virtual machine without passing through the CPU of the PCIE switch through the modified third data routing control is as follows: the PCIE switch sends the received data directly to the network card of the second virtual machine without passing through the CPU of the PCIE switch through the modified third data routing control, the network card of the second virtual machine sends the data to the GPU card of the second virtual machine in a point-to-point manner, and the network card of the second virtual machine sends the data to the GPU card of the second virtual machine in a point-to-point manner, which is the same as the above-mentioned first virtual machine implementing the point-to-point manner between GPUs, that is, by importing the software control logic of the second virtual machine of the network card, responding to the inter-GPU communication instruction, when it is determined that the point-to-point communication conditions are met, the data is sent to the PCIE switch in a point-to-point communication manner, and the PCIE switch sends the received data directly to the GPU card of the second virtual machine without passing through the CPU of the PCIE switch through the modified data routing control.

[0077] When direct access to remote memory is to be achieved between GPUs of different virtual machines, the third data routing control of the PCIE exchange needs to be modified to release the mandatory control of the data flow by the host side, i.e., the PCIE switch. In the embodiment of the present application, for the PCIE switch, the modified third data routing control is obtained by modifying the value of the state bit of the data routing control register corresponding to the second virtual machine imported into the network card. Thus, for the network card, within the virtual machine, the data routing of its uplink is not completely affected by the state of the data routing control register of the physical host.

[0078] In the embodiment of the present application, the first virtual machine into which the network card is imported involves two layers of operating systems OS. The first layer of OS is the operating system OS of the network card, and the other layer of OS is the operating system OS at the virtual machine level. Therefore, in order to achieve communication between direct access to remote memory of different virtual machines, it is necessary to modify the open source simulator and open source UEFI firmware logic of the first virtual machine.

[0079] In the embodiment of the present application, based on modifying the data routing control register of the corresponding PCIE switch so that the PCIE switch supports directly sending data to the peer GPU card or peer network card of the first virtual machine without passing through the CPU of the PCIE switch, it is necessary to further modify the open source simulator and open source UEFI firmware logic of the first virtual machine, so that the first virtual machine imported into the network card adopts the second software control logic, responds to the cross-virtual machine communication instruction, and sends the data to the PCIE switch after device address conversion through the modified second data routing control.

[0080] In the embodiment of the present application, the first virtual machine imported into the network card involves two layers of addresses. The first layer of addresses is the address at the physical layer, that is, the address of the network card that can be seen by the operating system OS of the network card, and the other layer of addresses is the address at the virtual machine layer, that is, the address that can be seen by the operating system OS. By way of example, the physical layer address is 01, and the virtual machine layer address is 20, but the two addresses belong to the same device. In order to realize the interaction between the network card of the first virtual machine and the GPU card of the second virtual machine, it is actually necessary to use the underlying hardware, which involves the address of the lowest physical layer. Therefore, if the network card wants to send data out, it is necessary to convert the address at the virtual machine layer into the address at the physical layer, that is, convert address 20 into 01 and then send it.

[0081] The embodiments of the present application can realize overall point-to-point communication between GPUs in the same virtual machine, as well as remote direct memory access between GPUs of multiple virtual machines, laying the foundation for training, reasoning and other high-performance computing, thereby solving the problem that the existing model topology in which both the GPU card and the network card supporting remote direct memory access are mounted under the PCIE switch, and the point-to-point communication between GPUs in the virtual environment and the remote direct memory access of GPUs across virtual machines are limited.

[0082] In some possible embodiments, the modified first data routing control is implemented in the following manner:

[0083] Modify the state of each state bit of the first data routing control register to a fully closed state to obtain a modified first data routing control;

[0084] The first data routing control register is a register in the PCIE switch corresponding to the GPU card of the first virtual machine.

[0085] For a PCIE switch, each device mounted under the PCIE switch has a corresponding data routing control register in the PCIE switch, and the data routing control register includes multiple status bits. For example, the data routing control register includes 16 status bits. By combining different values ​​of the 16 status bits, corresponding different data routing controls are obtained. In order to achieve this, in the embodiment of the present application, for a GPU card, in a virtual machine, the data routing of its uplink is not completely affected by the state of the data routing control register of the physical host. The specific setting method is to modify the state of each state bit of the first data routing control register to a fully closed state. For a GPU card, and it does not support device address conversion, so the state of its data routing control register in the physical host needs to be modified to a fully closed state to release the host-side mandatory control of the data flow. For example, the values ​​of the 16 status bits are all set to zero. Of course, other settings are also possible. After the PCIE switch receives the data sent by the first virtual machine imported into the GPU card, when it determines that the status bits of the corresponding data routing control register are all zero, the data is sent directly without going through the CPU, otherwise the data is sent to the CPU and sent after relevant processing by the CPU.

[0086] The PCIE switch uses the corresponding data routing protocol to exchange data. The above data routing protocol includes ACS. In the case of virtual machines, the present application sets all the status bits of the data routing control register corresponding to the first virtual machine imported into the GPU card to zero, and the ACS function can be turned off, allowing GPU to GPU access without going through the CPU memory; in another possible case, all the status bits of the data routing control register are set to 1, and direct access from GPU to GPU is not allowed.

[0087] In some possible embodiments, the modified third data routing control is implemented in the following manner:

[0088] Modify the direct communication indication state bit in the third data routing control register to an enabled state, thereby obtaining a modified third data routing control;

[0089] The third data routing control register is a register in the PCIE switch corresponding to the network card of the first virtual machine.

[0090] For a PCIE switch, each device mounted under the PCIE switch has a corresponding data routing control register in the PCIE switch, and the data routing control register includes multiple status bits. For example, the data routing control register includes 16 status bits. By combining different values ​​of the 16 status bits, corresponding different data routing controls are obtained. As mentioned above, the PCIE switch uses the corresponding data routing protocol for data exchange. In the case of the virtual machine involved in this application, the status bits in the data routing control register corresponding to different devices mounted under the PCIE switch correspond to different corresponding data routing protocol rules. The data routing control register corresponding to the first virtual machine imported into the network card includes a status bit indicating whether direct communication is allowed. In the embodiment of this application, the direct communication indication status bit in the third data routing control register is modified to an enabled state, so that the data sent by the first virtual machine imported into the network card reaches the PCIE switch, and can be directly sent to the network card of the second virtual machine without going through the CPU memory. The network card of the second virtual machine sends the data to the PCIE switch in the same way. The PCIE switch directly sends the data according to the above status bit of the corresponding data routing control register.

[0091] The modification of the data routing control register in the embodiment of the present application involves modifying the state of the first data routing control register corresponding to the first virtual machine imported into the GPU card, and modifying the state of the third data routing control register corresponding to the second virtual machine imported into the network card in the PCIE switch as the physical host end. For the network card, the network card has an address conversion register inside, so the address conversion register value of the network card needs to be modified in the embodiment of the present application. The address conversion register value of the network card can include one bit or multiple bits, and its value has two states, one state is to turn on the address conversion function, and the other state is to turn off the address conversion function. In the embodiment of the present application, it is necessary to modify it to turn on the address conversion function. In some possible embodiments, by modifying the register value in the address conversion register of the network card, the address conversion function is turned on; based on the address conversion register, the OS level address when the second virtual machine sends data is converted into the physical level address of the network card, and the data is sent to the PCIE switch through the physical level address.

[0092] The modification process of the above data routing control register and address conversion register is described below with reference to the accompanying drawings. Figure 3 As shown, it mainly includes:

[0093] Step 301, the PCIE switch starts to check the server topology;

[0094] The process of executing the server topology check needs to determine the device topology in the server through the physical address of the PCIE device hanging under the PCIE switch and the identification code generated by the device driver, and then determine the type of the device hanging under the PCIE switch. The specific server topology check process can adopt the topology check method in the relevant technology, which will not be described in detail here. Since each hanging PCIE device has a corresponding data routing control register in the PCIE switch, after determining the address of the hanging PCIE device, the address of the corresponding data routing control register can be determined.

[0095] Step 302, determine the type of the PCIE device attached, if it is a network card, execute step 303, if it is a GPU card, execute step 304;

[0096] Step 303, by modifying the register value in the address conversion register of the network card, the address conversion function is turned on;

[0097] Since the address translation function is a device-specific attribute, for devices that support address translation, such as network cards, you can use their drivers or special tools to modify the corresponding address translation registers. By modifying the register value in the address translation register of the network card, the address translation function is turned on.

[0098] Step 304, modifying the data routing control register corresponding to each PCIE device, releasing the routing control of the PCIE device, so as to support the direct transmission of the data sent by the PCIE device without passing through the CPU;

[0099] By capturing the value of the address conversion register of each PCIE device, a binary value escape operation is performed. According to the data routing control register principle of the PCIE device, different values ​​have different routing control modes according to corresponding rules and comply with the PCIE protocol. For the first data routing control register corresponding to the GPU card, the embodiment of the present application modifies the state of each status bit to a fully closed state. For the second data routing control register corresponding to the network card, the direct communication indication status bit, i.e., the "DirectTrans" function bit, is modified to an enabled state, so that the physical host side supports direct transmission of network card data.

[0100] Step 305, check the status of each data routing control register and address conversion register, and end when it is determined that all modifications are completed.

[0101] In the embodiments of the present application, the settings of the data routing control and address conversion related registers on the physical machine side can be integrated through code and packaged into an automated tool that can be executed with one click; regarding the logical modification of the virtual machine open source simulator and the open source UEFI firmware, after modification and compilation, they can be packaged into software and directly reused.

[0102] On the basis of modifying the above data routing control register and address translation register, at the software level, in order to realize the communication between GPU cards of the same virtual machine and the direct access to remote memory between different virtual machines, it is necessary to modify the open source simulator and the open source UEFI firmware logic of the first virtual machine at the same time to obtain the above first software control logic and second software control logic. The specific method of modifying the open source simulator and the open source UEFI firmware logic is given below.

[0103] First, the data interaction between PCIE devices such as GPU cards and network cards in the virtual machine is controlled by the relevant settings in the open source simulator and open source UEFI firmware of the virtual machine, so it is sufficient to modify the underlying logic of the open source simulator and open source UEFI firmware of the virtual machine. In order to realize the point-to-point communication of the GPU card in the virtual machine and the function of direct access to remote memory, the components of the virtualization platform need to be adjusted accordingly, which involves the communication logic of PCIE devices in the virtual machine simulator and UEFI firmware.

[0104] At the level of the open source simulator of the open source simulation platform QEMU, for GPU cards, the open source machine simulator that simulates the virtual machine should support directly passing the GPU card of the physical machine to the virtual machine to ensure that the virtual machine can directly access the hardware resources of the GPU card. NVIDIA, the manufacturer of the GPU card, has defined a specification, that is, when the virtual machine is not involved and the GPU runs on the bare metal, the GPU-related driver and calling tools have judgment logic that will determine which GPUs in the system can support point-to-point communication based on the GPU card status and topology. However, when the GPU card runs in the virtual machine, the PCIE topology is flattened and blurred. Through the PCIE address relationship between the GPUs in the virtual machine, it is impossible to determine whether the GPU card and topology are supported at the physical hardware level. Therefore, corresponding modifications need to be made at the underlying software level of the virtual machine, that is, the QEMU open source simulator level.

[0105] In some possible embodiments, when the first virtual machine imported into the GPU card determines that the point-to-point communication condition is met, sending data to the PCIE switch in a point-to-point communication manner includes:

[0106] Obtaining device group information in a group register, wherein the device group information is information defined by a simulator of the first virtual machine, the device group information includes a device group ID, and an address of the group register has a specified offset in the address space;

[0107] When it is determined that the device group ID of the GPU card on the other end is consistent with the obtained device group ID, it is determined that the point-to-point communication condition is met;

[0108] Through the fourth data routing control modified at the virtual machine level, the data is sent to the PCIE switch in a point-to-point communication manner.

[0109] In the embodiment of the present application, a device group definition and device group information are added to the simulator of the virtual machine, and a register group is added accordingly to allocate and identify the group ID of the verification device. Devices with the same group ID can perform direct point-to-point communication. In the address space of the GPU device, the address of the group register has a specified offset in the address space. The function of this register group is to obtain the address range in the PCIE configuration space for direct communication with the GPU card of the first virtual machine, and return the device group information in the group register as data of the simulated PCIE capability structure, thereby providing the NVIDIA GPU driver stack in the virtual machine with information related to whether point-to-point communication is supported.

[0110] In some possible embodiments, the device group information further includes point-to-point capability information, device identification and verification information, and when it is determined that the device group ID of the opposite GPU card is consistent with the acquired device group ID, it is determined that the point-to-point communication condition is met, including:

[0111] According to the point-to-point capability information, it is determined that the GPU card of the first virtual machine supports point-to-point communication, and according to the device identification and verification information, the opposite GPU card is identified and verified. When it is determined that the verification passes and the device group ID of the opposite GPU card is consistent with the obtained device group ID, it is determined that the point-to-point communication conditions are met.

[0112] The group register may be, but is not limited to, 16 bits, which includes, in addition to the device group ID, point-to-point capability information, device identification and verification information, and capability bits such as reserved bits. Through the definition of the point-to-point capability information, it can be determined whether point-to-point communication is supported, and through the device identification and verification information, the peer GPU card of the first virtual machine can be verified.

[0113] By using the above-mentioned software-level modification method provided in the embodiment of the present application, it can be ensured that the overall state is changed as little as possible, and the PCIE topology obtained by the NVIDIA driver and tools in the virtual machine will not be changed. In addition, the offset and address are static and do not change with changes in the GPU device itself and the power state of the virtual machine.

[0114] In some possible embodiments, the modified second data routing control is used to send the data to the PCIE switch after the device address is converted, including:

[0115] Scan all PCIE downstream bridge locations of the first virtual machine into which the GPU card is imported;

[0116] If a peer GPU card is scanned, the routing control function of the bridge position corresponding to the peer GPU card scanned in the second data routing control register is turned off to obtain a modified second data routing control;

[0117] The modified second data routing control is adopted to send the data to the PCIE switch after the device address is converted.

[0118] At the QEMU open source simulator level, for virtual machines that import network cards, it is necessary to adjust the data transmission path, reduce unnecessary copying and delays in the network stack, and enable data to be directly transferred from user space to RDMA devices. Implement efficient buffer allocation and management strategies to reduce memory copies and improve RDMA transmission performance, thereby optimizing the network stack to support devices to directly access remote memory data. The embodiment of the present application can use the simulator level of virtual machines that import network cards in related technologies. However, in order to support the network card to achieve direct access to remote memory, the underlying logic of the UEFI firmware needs to be modified.

[0119] Modify the underlying logic of the UEFI firmware, optimize the access method of the PCIE device configuration space, disassemble the data routing control logic, decouple the data flow method at the hardware level from the hardware protocol, and modify the model of interaction between the underlying hardware and the system. Analogous to the data routing control at the physical machine level, there are similar functions at the virtual machine level, so corresponding settings need to be made in the UEFI firmware of the virtual machine. The specific implementation method is to scan all downstream bridge positions in all PCIE links in the virtual machine. If the downstream scans a GPU class device according to the ClassCode (a field in the PCIE device descriptor used to identify the device type and function), the data routing control function of this bridge position is turned off. The scanning is implemented by enumerating and comparing all devices of the downstream bridge, comparing the scan result with the ClassCode of the target device, obtaining the return value, and judging whether the link has the target GPU card.

[0120] In some possible embodiments, disabling the routing control function of the bridge position corresponding to the GPU card scanned in the second data routing control register includes:

[0121] According to the routing control option added to the status bit of each bridge position in the second data routing control register, the routing control option of the bridge position corresponding to the scanned opposite-end GPU card is set to be closed.

[0122] In the embodiment of the present application, in the virtual machine UEFI firmware, a routing control option is added to the status bit of each bridge position in the second data routing control register, which corresponds to the downstream routing control register, thereby realizing interface normalization of routing control, and realizing standardized adjustment of routing control registers by adjusting the routing control option value without having to operate the register.

[0123] like Figure 4 The overall process diagram of routing control for multi-machine GPU communication in the virtual environment of this application mainly includes:

[0124] Step 401, modifying the status of each status bit of the first data routing control register corresponding to the first virtual machine imported into the GPU card in the PCIE switch to a fully closed state, thereby obtaining a modified first data routing control;

[0125] Step 402, modify the direct communication indication state bit in the third data routing control register corresponding to the second virtual machine imported into the network card in the PCIE switch to an enabled state, thereby obtaining a modified third data routing control;

[0126] Step 403, modifying the register value in the address translation register of the network card so that the address translation function is in an enabled state;

[0127] Step 404, adding a device group definition in the simulator of the first virtual machine into which the GPU card is imported, storing the corresponding device group information in the group register, and modifying the underlying logic of the UEFI firmware to obtain the first software control logic;

[0128] Step 405, for the first virtual machine imported into the network card, scan all PCIE downstream bridge locations of the first virtual machine imported into the GPU card;

[0129] Step 406, if a peer GPU card is scanned, disable the routing control function of the bridge position corresponding to the peer GPU card scanned in the second data routing control register, and obtain a modified second data routing control;

[0130] Step 407, by means of the first software control logic, in response to the inter-GPU communication instruction, obtaining device group information, and determining that the peer GPU card device group ID is consistent with the obtained device group ID, determining that the peer-to-peer communication condition is met;

[0131] Step 408, sending the data to the PCIE switch in a point-to-point communication manner through the fourth data routing control modified at the virtual machine level, and the PCIE switch directly sends the data through the modified first data routing control;

[0132] Step 409, through the second software control logic, in response to the cross-virtual machine communication instruction, through the modified second data routing control, the data is sent to the PCIE switch after device address conversion, and the PCIE switch directly sends the data through the modified third data routing control.

[0133] By utilizing the routing control method for point-to-point communication of multiple GPUs in a virtual environment provided by the embodiment of the present application, the data routing control register status can be captured and set with one click in the physical machine; the address conversion register status of devices such as network cards can be acquired and set with one click in the physical machine; the virtual machine open source simulator and open source UEFI firmware generated by modifying the source code compilation can be used to realize the installation of the virtual machine, the transparent import of the device and the environment deployment; the overall point-to-point communication between GPUs in the same virtual machine and the remote direct memory access between GPUs of multiple virtual machines can be realized, laying the foundation for training, reasoning and other high-performance computing.

[0134] Based on the same inventive concept, an embodiment of the present application also provides a routing control device for multi-machine GPU communication in a virtual environment, at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to execute the first software control logic of the first virtual machine imported into the GPU card, or to execute the second software control logic of the first virtual machine imported into the network card.

[0135] Refer to the following Figure 5 The routing control device 150 for multi-machine GPU communication in a virtual environment according to this embodiment of the present application is described. Figure 4 The routing control device 150 for multi-machine GPU communication in the displayed virtual environment is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0136] like Figure 5 As shown, the routing control device 150 for multi-machine GPU communication in a virtual environment is presented in the form of a general electronic device. The components of the routing control device 150 for multi-machine GPU communication in a virtual environment may include but are not limited to: the at least one processor 151, the at least one memory 152, and a bus 153 connecting different system components (including the memory 152 and the processor 151).

[0137] Bus 153 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, a processor, or a local bus using any of a variety of bus architectures.

[0138] The memory 152 may include a readable medium in the form of a volatile memory, such as a random access memory (RAM) 1521 and / or a cache memory 1522 , and may further include a read-only memory (ROM) 1523 .

[0139] The memory 152 may also include a program / utility 1525 having a set (at least one) of program modules 1524, such program modules 1524 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.

[0140] The routing control device 150 for multi-machine GPU communication in a virtual environment may also communicate with one or more external devices 154 (e.g., keyboards, pointing devices, etc.), and may also communicate with one or more devices that enable a user to interact with the routing control device 150 for multi-machine GPU communication in a virtual environment, and / or communicate with any device (e.g., routers, modems, etc.) that enables the routing control device 150 for multi-machine GPU communication in the virtual environment to communicate with one or more other electronic devices. Such communication may be performed through an input / output (I / O) interface 155. In addition, the routing control device 150 for multi-machine GPU communication in a virtual environment may also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN) and / or public network, such as the Internet) through a network adapter 156. As shown in the figure, the network adapter 156 communicates with other modules of the routing control device 150 for multi-machine GPU communication in a virtual environment through a bus 153. It should be understood that although not shown in the figure, other hardware and / or software modules can be used in conjunction with the routing control device 150 for multi-machine GPU communication in a virtual environment, including but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0141] In some possible implementations, various aspects of a method for routing control of multi-machine GPU communications in a virtual environment provided by the present application may also be implemented in the form of a program product, which includes program code. When the program product is run on a computer device, the program code is used to enable the computer device to execute the steps of a method for routing control of multi-machine GPU communications in a virtual environment according to various exemplary implementations of the present application described above in this specification.

[0142] The program product may employ any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0143] The program product for monitoring of the embodiment of the present application can adopt a portable compact disk read-only memory (CD-ROM) and include program code, and can be run on an electronic device. However, the program product of the present application is not limited to this. In this document, a readable storage medium can be any tangible medium containing or storing a program, which can be used by an instruction execution system, an apparatus or a device or used in combination with it.

[0144] The readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, wherein the readable program code is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The readable signal medium may also be any readable medium other than a readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0145] The program code embodied on the readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination of the foregoing.

[0146] The program code for performing the operations of the present application can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, etc., and also conventional procedural programming languages ​​such as "C" language or similar programming languages. The program code can be executed entirely on the user electronic device, partially on the user device, as an independent software package, partially on the user electronic device and partially on the remote electronic device, or entirely on the remote electronic device or server. In the case of a remote electronic device, the remote electronic device can be connected to the user electronic device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external electronic device (for example, using an Internet service provider to connect through the Internet).

[0147] It should be noted that, although several units or subunits of the device are mentioned in the above detailed description, this division is merely exemplary and not mandatory. In fact, according to the embodiments of the present application, the features and functions of two or more units described above can be embodied in one unit. Conversely, the features and functions of one unit described above can be further divided into multiple units to be embodied.

[0148] In addition, although the operations of the method of the present application are described in a specific order in the drawings, this does not require or imply that the operations must be performed in this specific order, or that all the operations shown must be performed to achieve the desired results. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step, and / or one step may be decomposed into multiple steps.

[0149] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0150] The present application is described with reference to the flowcharts and block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and block diagram, as well as the combination of the processes and boxes in the flowchart and block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and block diagram. Figure 1 Process or multiple processes and boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0151] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 Process or multiple processes and boxes Figure 1 A function specified in one or more boxes.

[0152] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 Process or multiple processes and boxes Figure 1 The steps for the functions specified in one or more boxes.

[0153] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the present application.

[0154] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A routing control method for multi-machine GPU communication in a virtual environment, characterized in that: The method comprises: By importing the first software control logic of the first virtual machine of the GPU card, in response to the inter-GPU communication instruction, when it is determined that the point-to-point communication condition is met, the data is sent to the PCIE switch in a point-to-point communication manner, and the PCIE switch directly sends the received data to the opposite GPU card of the first virtual machine without passing through the CPU of the PCIE switch through the modified first data routing control; By importing the second software control logic of the first virtual machine of the network card, in response to the cross-virtual machine communication instruction, the data is sent to the PCIE switch after device address conversion through the modified second data routing control, and the PCIE switch sends the received data directly to the GPU card of the second virtual machine without passing through the CPU of the PCIE switch through the modified third data routing control.

2. The method according to claim 1, characterized in that The modified first data routing control is implemented in the following manner: Modify the state of each state bit of the first data routing control register to a fully closed state to obtain a modified first data routing control; The first data routing control register is a register in the PCIE switch corresponding to the GPU card of the first virtual machine.

3. The method according to claim 1, characterized in that The modified third data routing control is implemented in the following manner: Modify the direct communication indication state bit in the third data routing control register to an enabled state, thereby obtaining a modified third data routing control; The third data routing control register is a register in the PCIE switch corresponding to the network card of the first virtual machine.

4. The method according to claim 1, characterized in that: When it is determined that the point-to-point communication conditions are met, the data is sent to the PCIE switch in a point-to-point communication mode, including: Obtaining device group information in a group register, wherein the device group information is information defined by a simulator of the first virtual machine, the device group information includes a device group ID, and an address of the group register has a specified offset in the address space; When it is determined that the device group ID of the GPU card on the other end is consistent with the obtained device group ID, it is determined that the point-to-point communication condition is met; Through the fourth data routing control modified at the virtual machine level, the data is sent to the PCIE switch in a point-to-point communication manner.

5. The method according to claim 4, characterized in that The device group information also includes point-to-point capability information, device identification and verification information. When it is determined that the device group ID of the opposite GPU card is consistent with the acquired device group ID, it is determined that the point-to-point communication conditions are met, including: According to the point-to-point capability information, it is determined that the GPU card of the first virtual machine supports point-to-point communication, and according to the device identification and verification information, the opposite GPU card is identified and verified. When it is determined that the verification passes and the device group ID of the opposite GPU card is consistent with the obtained device group ID, it is determined that the point-to-point communication conditions are met.

6. The method according to claim 1, characterized in that By means of the modified second data routing control, the data is sent to the PCIE switch after the device address is converted, including: Scan all PCIE downstream bridge locations of the first virtual machine into which the GPU card is imported; If a peer GPU card is scanned, the routing control function of the bridge position corresponding to the peer GPU card scanned in the second data routing control register is turned off to obtain a modified second data routing control; The modified second data routing control is adopted to send the data to the PCIE switch after the device address is converted.

7. The method according to claim 6, characterized in that Turning off the routing control function of the bridge position corresponding to the GPU card scanned in the second data routing control register includes: According to the routing control option added to the status bit of each bridge position in the second data routing control register, the routing control option of the bridge position corresponding to the scanned opposite-end GPU card is set to be closed.

8. The method according to claim 1, characterized in that The data is sent to the PCIE switch after the device address is converted, including: By modifying the register value in the address conversion register of the network card, the address conversion function is turned on; Based on the address conversion register, the OS layer address when the second virtual machine sends data is converted into the physical layer address of the network card, and the data is sent to the PCIE switch through the physical layer address.

9. A routing control system for multi-machine GPU communication in a virtual environment, characterized in that: It includes a PCIE switch, a first virtual machine mounted under the PCIE switch and importing different GPU cards, a first virtual machine mounted under the PCIE switch and importing a network card, a second virtual machine mounted under the PCIE switch and importing different GPU cards, and a second virtual machine mounted under the PCIE switch and importing a network card, wherein: A first virtual machine imported into the GPU card is used to respond to the inter-GPU communication instruction through the first software control logic, and when it is determined that the point-to-point communication condition is met, send the data to the PCIE switch in a point-to-point communication manner, and the PCIE switch controls the received data directly to the opposite GPU card of the first virtual machine without passing through the CPU of the PCIE switch through the modified first data routing control; The first virtual machine into which the network card is imported is used to respond to the cross-virtual machine communication instruction through the second software control logic, and send the data to the PCIE switch after the device address is converted through the modified second data routing control. The PCIE switch sends the received data directly to the GPU card of the second virtual machine without passing through the CPU of the PCIE switch through the modified third data routing control.

10. A routing control device for multi-machine GPU communication in a virtual environment, characterized in that: The invention comprises at least one processor; and a memory connected to the at least one processor in communication; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to execute the first software control logic of the first virtual machine imported into the GPU card in any one of claims 1 to 8, or to execute the second software control logic of the first virtual machine imported into the network card in any one of claims 1 to 8.

Images