Access control method and device and server

Abstract

The invention relates to an access control method and device and a server. The server comprises a host CPU, a plurality of GPUs, a host memory of the host CPU, a GPU memory of each GPU and a plurality of PCIe switches, an uplink port of each PCIe switch is connected to the host CPU, the plurality of GPUs are divided into a plurality of groups, the plurality of GPUs in the same group are connected to different downlink ports of the same PCIe switch, a first GPU determines a target address corresponding to a target memory needing to be accessed, and a second GPU determines a target address corresponding to the target memory needing to be accessed. Sending an access request generated according to the target address to the connected PCIe switch; and the PCIe switch forwards the access request of which the target address is the virtual address to the host CPU, and forwards the access request of which the target address is the physical address to a target GPU where a target memory connected to a downlink port of the PCIe switch is located. The limitation of the GPU access address range is broken through, and the memory access speed and efficiency in the server are ensured.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular to an access control method, device and server. Background Art

[0002] GPU server is a fast, stable and flexible computing service based on GPU for video encoding and decoding, deep learning, scientific computing and other scenarios. This server is equipped with multiple GPUs and has large memory, high-speed storage and network connection. It is usually used in situations that require high-performance computing, such as processing large-scale data, deep learning model training, image recognition, speech recognition, natural language processing, etc. The common topology of GPU servers is as follows: Figure 1 As shown in the figure. The GPU accesses the host memory (Host Memoryroy) and the PCIe memory bars (PCIe Memory Base Address Registe) of other GPUs through the outbound communication (PCIe Outbound) of the PCIe device. In order to complete this function, the PCIe outbound window (PCIe outbound Window) of each GPU must be performed as follows Figure 2 The configuration shown assumes that the address width of PCIe that the GPU can access is 39 bits, which limits the address range that the GPU can access to 0-512G. This means that the sum of the host memory and the memory of each GPU cannot exceed 512G. How to overcome this limitation is a technical problem that needs to be solved urgently. Summary of the invention

[0003] In view of this, the present disclosure proposes an access control method, device and server.

[0004] According to one aspect of the present disclosure, a server is provided, comprising a host CPU, a plurality of GPUs, a host memory of the host CPU, a GPU memory of each of the GPUs, and a plurality of PCIe switches, wherein an uplink port of each of the PCIe switches is connected to the host CPU, the plurality of GPUs are divided into a plurality of groups, and a plurality of GPUs in the same group are connected to different downlink ports of the same PCIe switch.

[0005] A first GPU among the multiple GPUs is used to determine a target address corresponding to a target memory to be accessed according to an address recorded in a PCIe outbound window of the first GPU, and send an access request generated according to the target address to a PCIe switch connected to the first GPU, wherein the target memory is any one of remaining GPU memories among the multiple GPU memories that are different from the GPU memory of the first GPU and the host memory;

[0006] The PCIe switch is configured to, upon receiving an access request, forward the access request whose target address is a virtual address to the host CPU, and forward the access request whose target address is a physical address to a target GPU where the target memory is located and which is connected to a downstream port of the PCIe switch;

[0007] Among them, the address recorded in the PCIe outbound window of each GPU includes at least one of the following: the virtual address of the host memory, the virtual address of the GPU memory corresponding to the out-of-group GPU with access requirements among the remaining GPUs in a different group from the GPU, and the physical address of the GPU memory corresponding to the in-group GPU with access requirements among the remaining GPUs in the same group as the GPU.

[0008] According to one aspect of the present disclosure, a server is provided, comprising a host CPU, a plurality of GPUs, a host memory of the host CPU, a GPU memory of each of the GPUs, and a plurality of PCIe switches, wherein an uplink port of each of the PCIe switches is connected to the host CPU, the plurality of GPUs are divided into a plurality of groups, and a plurality of GPUs in the same group are connected to different downlink ports of the same PCIe switch.

[0009] A first GPU among the multiple GPUs is used to determine a target address corresponding to a target memory to be accessed according to an address recorded in a PCIe outbound window of the first GPU, and send an access request generated according to the target address to a PCIe switch connected to the first GPU, wherein the target memory is any one of remaining GPU memories among the multiple GPU memories that are different from the GPU memory of the first GPU and the host memory;

[0010] The PCIe switch is used to forward the access request to the host CPU when receiving the access request;

[0011] The address recorded in the PCIe outbound window of each GPU includes at least one of the following: a virtual address of the host memory, and a virtual address of a GPU memory corresponding to a GPU with access requirements among the remaining GPUs different from the GPU.

[0012] In a possible implementation, the host CPU is used to, upon receiving an access request, determine a target physical address corresponding to a target address in the access request, and send the access request to a corresponding target PCIe switch according to the target physical address;

[0013] The target PCIe switch is used to send the access request to the target GPU where the target memory is located according to the target physical address, so that the access request is executed by the target GPU.

[0014] In a possible implementation manner, determining a target physical address corresponding to a target address in the access request includes:

[0015] Determine the target physical address corresponding to the target address in the access request according to the total address mapping relationship or the physical address of each GPU; or

[0016] Determine, according to the address sub-mapping relationship corresponding to the first GPU, a target physical address corresponding to the target address in the access request;

[0017] The total address mapping relationship is used to indicate the mapping relationship between the virtual address and the physical address of the host memory and the mapping relationship between the virtual address and the physical address of each GPU memory with access requirements;

[0018] The address sub-mapping relationship corresponding to the first GPU is used to indicate the mapping relationship between the virtual address and the physical address of the host memory and the mapping relationship between the virtual address and the physical address of the GPU memory corresponding to each of the out-of-group GPUs.

[0019] In a possible implementation, the host CPU is used to:

[0020] configuring virtual addresses for the host memory and the GPU memory with access requirements, wherein the GPU memory with access requirements is part or all of the multiple GPU memories in the server;

[0021] Mapping the virtual address of the host memory to the physical address of the host memory by using a memory management unit, and mapping the virtual address of each GPU memory with access requirements to the physical address of the GPU memory;

[0022] Creating a total address mapping relationship and an address sub-mapping relationship of each GPU according to a mapping relationship between a virtual address and a physical address of the host memory and a mapping relationship between a virtual address and a physical address of the GPU memory;

[0023] The PCIe outbound window of each GPU is configured based on the address sub-mapping relationship of each GPU.

[0024] In a possible implementation manner, each of the PCIe switches is controlled to disable a redirection function of a downstream port.

[0025] In a possible implementation, the range of virtual addresses configured for the host memory and the GPU memory with access requirements is within the restricted address range of the PCIe outbound window of each GPU.

[0026] According to one aspect of the present disclosure, an access control method is provided, which is applied to a host CPU in a server, wherein the server further comprises a plurality of GPUs, a host memory of the host CPU, a GPU memory of each of the GPUs, and a plurality of PCIe switches, wherein an uplink port of each of the PCIe switches is connected to the host CPU, the plurality of GPUs are divided into a plurality of groups, and a plurality of GPUs in the same group are connected to different downlink ports of the same PCIe switch, wherein the method comprises:

[0027] Upon receiving the access request forwarded by the PCIe switch, determining a target physical address corresponding to a target address in the access request, and sending the access request to a corresponding target PCIe switch according to the target physical address, so that the target PCIe switch sends the access request to a target GPU where the target memory is located according to the target physical address, so that the access request is executed by the target GPU;

[0028] The access request is generated by a first GPU among the multiple GPUs according to a target address and then sent to a PCIe switch, and forwarded to the host CPU when the PCIe switch determines that the target address is a virtual address, the target address is determined by the first GPU according to an address recorded in a PCIe outbound window of the first GPU and a target memory to be accessed, and the target memory is any one of the remaining GPU memories among the multiple GPU memories that are different from the GPU memory of the first GPU and the host memory;

[0029] The addresses recorded in the PCIe outbound window of each GPU include at least one of the following: the virtual address of the host memory, the virtual address of the GPU memory corresponding to the out-of-group GPU with access requirements among the remaining GPUs in a different group from the GPU, and the physical address of the GPU memory corresponding to the in-group GPU with access requirements among the remaining GPUs in the same group as the GPU.

[0030] According to one aspect of the present disclosure, an access control method is provided, which is applied to a host CPU in a server, wherein the server further comprises a plurality of GPUs, a host memory of the host CPU, a GPU memory of each of the GPUs, and a plurality of PCIe switches, wherein an uplink port of each of the PCIe switches is connected to the host CPU, the plurality of GPUs are divided into a plurality of groups, and a plurality of GPUs in the same group are connected to different downlink ports of the same PCIe switch, wherein the method comprises:

[0031] Upon receiving the access request forwarded by the PCIe switch, determining a target physical address corresponding to a target address in the access request, and sending the access request to a corresponding target PCIe switch according to the target physical address, so that the target PCIe switch sends the access request to a target GPU where the target memory is located according to the target physical address, so that the access request is executed by the target GPU;

[0032] The access request is generated by a first GPU among the multiple GPUs according to a target address and then sent to a PCIe switch and forwarded to the host CPU by the PCIe switch, the target address is determined by the first GPU according to an address recorded in a PCIe outbound window of the first GPU and a target memory to be accessed, and the target memory is any one of the remaining GPU memories among the multiple GPU memories that are different from the GPU memory of the first GPU and the host memory;

[0033] The address recorded in the PCIe outbound window of each GPU includes at least one of the following: a virtual address of the host memory, and a virtual address of a GPU memory corresponding to a GPU having access requirements among the remaining GPUs different from the GPU.

[0034] According to one aspect of the present disclosure, an access control method is provided, which is applied to a PCIe switch in a server, wherein the server includes a host CPU, multiple GPUs, a host memory of the host CPU, GPU memory of each of the GPUs, and multiple PCIe switches, wherein an uplink port of each of the PCIe switches is connected to the host CPU, the multiple GPUs are divided into multiple groups, and multiple GPUs in the same group are connected to different downlink ports of the same PCIe switch, wherein the method includes:

[0035] In the case of receiving an access request sent by a first GPU among the multiple GPUs, forwarding the access request to the host CPU; the access request is generated by the first GPU according to a target address, the target address is determined by the first GPU according to an address recorded in a PCIe outbound window of the first GPU and a target memory to be accessed, and the target memory is any one of the remaining GPU memories among the multiple GPU memories that are different from the GPU memory of the first GPU and the host memory; or,

[0036] When receiving an access request from the host CPU, sending the access request to a target GPU where the target memory is located according to a target physical address corresponding to the access request, so that the access request is executed by the target GPU;

[0037] The address recorded in the PCIe outbound window of each GPU includes at least one of the following: a virtual address of the host memory, and a virtual address of a GPU memory corresponding to a GPU with access requirements among the remaining GPUs different from the GPU.

[0038] According to one aspect of the present disclosure, an access control method is provided, which is applied to a PCIe switch in a server, wherein the server includes a host CPU, multiple GPUs, a host memory of the host CPU, GPU memory of each of the GPUs, and multiple PCIe switches, wherein an uplink port of each of the PCIe switches is connected to the host CPU, the multiple GPUs are divided into multiple groups, and multiple GPUs in the same group are connected to different downlink ports of the same PCIe switch, wherein the method includes:

[0039] Upon receiving an access request sent from a first GPU among the multiple GPUs, forwarding the access request whose target address is a virtual address to the host CPU, and forwarding the access request whose target address is a physical address to a target GPU where the target memory is located and connected to a downstream port of the PCIe switch; the access request is generated by the first GPU according to the target address, the target address is determined by the first GPU according to an address recorded in a PCIe outbound window of the first GPU and a target memory to be accessed, and the target memory is any one of the remaining GPU memories among the multiple GPU memories that are different from the GPU memory of the first GPU and the host memory; or,

[0040] When receiving an access request from the host CPU, sending the access request to a target GPU where the target memory is located according to a target physical address corresponding to the access request, so that the access request is executed by the target GPU;

[0041] Among them, the address recorded in the PCIe outbound window of each GPU includes at least one of the following: the virtual address of the host memory, the virtual address of the GPU memory corresponding to the out-of-group GPU with access requirements among the remaining GPUs in a different group from the GPU, and the physical address of the GPU memory corresponding to the in-group GPU with access requirements among the remaining GPUs in the same group as the GPU.

[0042] According to one aspect of the present disclosure, there is provided an access control method, which is applied to a GPU in a server, wherein the server includes a host CPU, multiple GPUs, a host memory of the host CPU, GPU memory of each of the GPUs, and multiple PCIe switches, wherein an uplink port of each of the PCIe switches is connected to the host CPU, the multiple GPUs are divided into multiple groups, and multiple GPUs in the same group are connected to different downlink ports of the same PCIe switch, wherein the method includes:

[0043] A first GPU among the multiple GPUs determines a target address of a target memory to be accessed according to an address recorded in a PCIe outbound window corresponding to the first GPU, and sends an access request generated according to the target address to a connected PCIe switch, so that the access request is forwarded to the host CPU by the PCIe switch, so that the host CPU determines a target physical address corresponding to the target address in the access request, and sends the access request to a corresponding target PCIe switch according to the target physical address, so that the target PCIe switch sends the access request to a target GPU where the target memory is located according to the target physical address;

[0044] When receiving the access request, the target GPU among the multiple GPUs executes the access request;

[0045] The target memory is any one of the remaining GPU memories different from the GPU memory of the first GPU among the multiple GPU memories and the host memory;

[0046] The addresses recorded in the PCIe outbound window of each of the GPUs include at least one of the following: the virtual address of the host memory, the virtual address of the GPU memory corresponding to the GPU outside the group that has access requirements among the remaining GPUs in a different group from the GPU, and the physical address of the GPU memory corresponding to the GPU within the group that has access requirements among the remaining GPUs in the same group as the GPU; or, the addresses recorded in the PCIe outbound window of each of the GPUs include at least one of the following: the virtual address of the host memory, the virtual address of the GPU memory corresponding to the GPU among the remaining GPUs that have access requirements that are different from the GPU.

[0047] According to another aspect of the present disclosure, an access control device is provided, comprising: a processor; and a memory for storing processor-executable instructions; wherein the processor is configured to implement the above method when executing the instructions stored in the memory.

[0048] According to another aspect of the present disclosure, a non-volatile computer-readable storage medium is provided, on which computer program instructions are stored, wherein the computer program instructions implement the above method when executed by a processor.

[0049] According to another aspect of the present disclosure, a computer program product is provided, including a computer-readable code, or a non-volatile computer-readable storage medium carrying the computer-readable code. When the computer-readable code runs in a processor of an electronic device, the processor in the electronic device executes the above method.

[0050] The access control method, device, and server provided by the embodiments of the present disclosure break through the limitation of the GPU access address range and ensure the speed and efficiency of memory access in the server.

[0051] Further features and aspects of the present disclosure will become apparent from the following detailed description of exemplary embodiments with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate exemplary embodiments, features, and aspects of the disclosure and, together with the description, serve to explain the principles of the disclosure.

[0053] Figure 1 A schematic diagram of a server topology is shown.

[0054] Figure 2 A schematic diagram showing the configuration of a PCIe outbound window of a GPU in the related art is shown.

[0055] Figure 3 A schematic diagram showing the configuration of a PCIe outbound window of a GPU in an embodiment of the present disclosure is shown.

[0056] Figure 4 , Figure 5 A schematic diagram of the generation and execution process of an access request in a server in an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0057] Various exemplary embodiments, features and aspects of the present disclosure will be described in detail below with reference to the accompanying drawings. The same reference numerals in the accompanying drawings represent elements with the same or similar functions. Although various aspects of the embodiments are shown in the accompanying drawings, the drawings are not necessarily drawn to scale unless otherwise specified.

[0058] The word “exemplary” is used exclusively herein to mean “serving as an example, example, or illustration.” Any embodiment described herein as “exemplary” is not necessarily to be construed as preferred or advantageous over other embodiments.

[0059] In addition, in order to better illustrate the present disclosure, numerous specific details are given in the following specific embodiments. It should be understood by those skilled in the art that the present disclosure can also be implemented without certain specific details. In some examples, methods, means, components and circuits well known to those skilled in the art are not described in detail in order to highlight the subject matter of the present disclosure.

[0060] In order to solve the above technical problems, the present disclosure provides an access control method, device and server, which break through the limitation of GPU access address range and ensure the speed and efficiency of memory access in the server.

[0061] like Figure 1As shown, the server provided by the embodiment of the present disclosure includes a host CPU (Host CPU) 10, multiple GPUs 20, a host memory (Host Memory) 11 of the host CPU, a GPU memory 21 of each of the GPUs 20, and multiple PCIe switches (PCIe switch) 30, the uplink port of each of the PCIe switches 30 is connected to the host CPU 10, the multiple GPUs 20 are divided into multiple groups, and multiple GPUs 20 in the same group are connected to different downlink ports of the same PCIe switch 30.

[0062] In some embodiments, Figure 1 As shown, the host CPU 10 and each GPU 20 can access the memory of other devices through the RDMA network (RDMANet) 40. Among them, RDMA (Remote Direct Memory Access) allows a device to directly access the memory of another device through the network without the intervention of the operating system or CPU, which can improve the efficiency and speed of data transmission, reduce the CPU load and network delay, and is particularly suitable for scenarios such as high-performance computing, big data processing, and real-time communication.

[0063] In some embodiments, the GPU memory 21 may be a PCIe memory bar (PCIe Memory Base Address Register), etc., which is not limited in the present disclosure.

[0064] In this embodiment, in the access control method executed by the server, the host CPU 10 may pre-execute the following configuration steps: configure virtual addresses for the host memory 11 and the GPU memory 21 with access requirements (the GPU memory 21 with access requirements may be part or all of all GPU memories 21); use the Input / Output Memory Management Unit (IOMMU) to map the virtual address of the host memory 11 to the physical address of the host memory 11, and map the virtual address of each GPU memory 21 with access requirements to the physical address of the GPU memory 21; create a total address mapping relationship and an address sub-mapping relationship of each GPU 20 according to the mapping relationship between the virtual address and the physical address of the host memory 11 and the mapping relationship between the virtual address and the physical address of the GPU memory 21; and configure the PCIe outbound window of each GPU 20 based on the address sub-mapping relationship of each GPU.

[0065] The total address mapping relationship is used to indicate the mapping relationship between the virtual address and the physical address of the host memory 11, and the mapping relationship between the virtual address and the physical address of each GPU memory with access requirements (that is, the GPU memory corresponding to each GPU with access requirements). The address sub-mapping relationship corresponding to each GPU is used to indicate the mapping relationship between the virtual address and the physical address of the host memory 11, and the mapping relationship between the virtual address and the physical address of the GPU memory of the GPU outside the group different from the GPU among the remaining GPUs with access requirements in the multiple GPUs.

[0066] In some embodiments, the range of virtual addresses configured for the host memory and the GPU memory with access requirements is within the restricted address range of the PCIe outbound window of each GPU. For example, if the address range of the PCIe outbound window of each GPU is 0-512G, when configuring the virtual addresses for the host memory 11 and each GPU memory 21, the range of the virtual addresses should be limited in combination with the address range that each GPU memory 21 can access, to ensure that the address recorded in the PCIe outbound window of each GPU 20 is within the range of 0-512G.

[0067] In this embodiment, the memory management unit IOMMU can determine the GPU memory that needs to be accessed by other GPUs according to actual usage needs, and then map the virtual address to the physical address of the GPU memory that needs to be accessed. In this way, within the address limit range of the memory management unit IOMMU, the address sub-mapping relationship can be set for each GPU and the PCIe outbound window can be configured to ensure normal memory access of the entire server.

[0068] Among them, when the host CPU executes different configuration strategies in the "configuration step", there are differences in the configuration of the PCIe outbound window corresponding to the GPU and the information that the PCIe switch needs to record, which is schematically explained below.

[0069] Configuration strategy 1:

[0070] by Figure 1 Take the server topology shown in the figure as an example. Figure 3As shown, the PCIe outbound window corresponding to GPU0 is configured according to "Configuration Strategy One". After configuration, the address recorded in the PCIe outbound window corresponding to GPU0 may include at least one of the following: the virtual address of the host memory 11, the physical address of the GPU memory of the GPU in the same group as GPU0 with access requirements, and the virtual address corresponding to the GPU memory of the GPU outside the group with access requirements in other GPUs in different groups from GPU0. Then, under "Configuration Strategy One", the address sub-mapping relationship corresponding to GPU0 can indicate the mapping relationship between the virtual address and the physical address of the host memory 11, and the mapping relationship between the virtual address and the physical address of the GPU memory of the GPU outside the group different from the GPU0. Then, under "Configuration Strategy One", each PCIe switch records the total address mapping relationship and / or the physical address of the GPU memory corresponding to the GPU connected to the downstream port. Similarly, other GPUs and PCIe switches are also set accordingly.

[0071] Configuration strategy 2:

[0072] by Figure 1 Taking the server topology shown as an example, the PCIe outbound window corresponding to GPU0 is configured according to "Configuration Strategy 2". After configuration, the address recorded in the PCIe outbound window corresponding to GPU0 may include at least one of the following: the virtual address of the host memory 11, and the virtual address of the GPU memory corresponding to the GPU with access requirements among the remaining GPUs different from GPU0. Then, under "Configuration Strategy 2", the address sub-mapping relationship corresponding to GPU0 can indicate the mapping relationship between the virtual address and the physical address of the host memory 11, and the mapping relationship between the virtual address and the physical address of the GPU memory corresponding to the GPU with access requirements among the remaining GPUs different from GPU0. Then, under "Configuration Strategy 2", each PCIe switch records the total address mapping relationship and / or the physical address of the GPU memory corresponding to each GPU. Similarly, other GPUs and PCIe switches are also set accordingly.

[0073] After the host CPU 10 completes the above-mentioned "configuration steps", there are differences in the memory access process implemented by the server executing the access control method according to different "configuration policies", which are explained separately below.

[0074] The memory access process under "Configuration Strategy 1" is as follows:

[0075] The host CPU 10 and any GPU 20 in the server can act as a requester to generate an access request for a target storage space in the target memory (a storage space in the target memory). The target memory can be any GPU memory 21 or host memory 11. The access request can indicate a target address corresponding to the target storage space.

[0076] If the requester is the host CPU 10 and the target memory is a GPU memory 21, the host CPU 10 can directly determine the target physical address of the target storage space in the target memory to be accessed based on the recorded physical addresses of the GPU memories 21, and generate an access request using the target physical address as the target address, and then send the access request to the target PCIe switch to which the GPU of the target memory is connected.

[0077] If the requester is the first GPU among multiple GPUs, and the target memory is the GPU memory 21 of a GPU in the same group as the first GPU. The first GPU can directly determine the target physical address of the target storage space in the target memory to be accessed based on the physical addresses of the GPU memories 21 of the same group (that is, the GPUs in the group) recorded in the PCIe outbound window, and generate an access request using the target physical address as the target address. The access request is then sent to the PCIe switch connected to the first GPU (since the PCIe switch connected to the first GPU will subsequently directly send the access request to the target GPU where the target memory is located, so that the access request is executed by the target GPU, the PCIe switch connected to the first GPU in this case is the target PCIe switch).

[0078] If the requester is the first GPU among multiple GPUs, and the target memory is the GPU memory 21 of a GPU in a different group from the first GPU, the first GPU can directly determine the virtual address of the target storage space in the target memory to be accessed based on the virtual address of the GPU memory 21 in the different group recorded in the PCIe outbound window, and generate an access request using the virtual address as the target address, and then send the access request to the PCIe switch to which the first GPU is connected.

[0079] If the requester is the first GPU among multiple GPUs and the target memory is the host memory 11, the first GPU can directly generate an access request using the virtual address of the host memory 11 recorded in the PCIe outbound window as the target address, and then send the access request to the PCIe switch to which the first GPU is connected.

[0080] The host CPU 10 may directly access the host memory 11 , and each GPU 20 may directly access its own GPU memory 21 . This may be set with reference to existing implementations, and the present disclosure does not impose any limitation on this.

[0081] For any PCIe switch 30: if an access request is received from a GPU 20 connected to a downstream port, first determine whether the target address in the access request is a virtual address based on the total address mapping relationship and / or physical address stored internally (the physical address stored in the PCIe switch 30 may include the physical address of the host memory 11 and the physical address of each GPU, or the physical address stored in the PCIe switch 30 may include the physical address of the host memory 11 and the physical address of the GPU with access request connected to the downstream port) (other methods may also be used to determine whether the target address is a virtual address, for example, when the GPU generates an access request, if the target address is a virtual address, a flag is directly added to the access request to indicate that the target address in the access request is a virtual address, etc., and the present disclosure does not limit this). If the target address is a virtual address, the access request is directly sent to the host CPU 20. If the target address is a physical address, the target address in the access request is directly determined as the target physical address, and the access request is sent to the target GPU corresponding to the target physical address. The target GPU is the GPU 20 that can access the target memory indicated by the target address. The target GPU is a GPU in the same group as the GPU that issues the access request and is connected to the PCIe switch 30 with the GPU that issues the access request. If an access request is received from the host CPU 10 connected to the uplink port, the target physical address corresponding to the target address in the access request is first determined based on the total address mapping relationship and / or physical address stored internally, and the access request is sent to the target GPU corresponding to the target physical address. After receiving the access request, the target GPU reads or writes data based on the target physical address corresponding to the access request to achieve memory access.

[0082] For the host CPU 10: If an access request forwarded by the PCIe switch 30 is received, the target physical address corresponding to the target address in the access request is first determined based on the total address mapping relationship stored internally. If it is then determined that the target memory corresponding to the target physical address is a GPU memory 21, the target GPU corresponding to the target memory corresponding to the target physical address is further determined, and then the PCIe switch 30 connected to the target GPU is determined as the target PCIe switch and the access request is sent to the target PCIe switch. The target GPU is the GPU 20 that can access the target memory indicated by the target address. If it is determined that the target memory corresponding to the target physical address is the host memory 11, the host CPU 10 directly executes the access request, reads or writes data for the target physical address, and implements memory access.

[0083] For example,

[0084] like Figure 4As shown, if the requester is the host CPU 10 and the target memory is the GPU memory 1, the control process of the access request is as follows Figure 4 The purple line 4 shown in . The host CPU 10 can directly generate an access request based on the physical address of GPU memory 1, where the physical address of the target storage space in GPU memory 1 is the target address and also the target physical address. Then the access request is sent to the PCIe switch 0 connected to GPU1. After determination (see above for details), PCIe switch 0 will directly send the access request to the corresponding GPU1 (that is, the target GPU). After receiving the access request, GPU1 reads or writes data based on the target physical address (that is, the target address) in the access request to achieve memory access.

[0085] like Figure 4 As shown, if the first GPU of the requesting party is GPU0 and the target memory is GPU memory 1, the control process of the access request is as follows Figure 4 The blue line 1 shown in . Then GPU0 can directly determine the target physical address of the target storage space in GPU memory 1 to be accessed based on the physical address of GPU memory 1 of each GPU in the group recorded in the PCIe outbound window, and generate an access request using the target physical address as the target address. The access request is then sent to the PCIe switch 0 to which GPU1 is connected. After determination (see above for details), PCIe switch 0 will directly send the access request to the corresponding GPU1 (that is, the target GPU). After receiving the access request, GPU1 reads or writes data based on the target physical address (that is, the target address) in the access request to achieve memory access.

[0086] like Figure 4 As shown, if the first GPU of the requesting party is GPU0 and the target memory is GPU memory n, the control process of the access request is as follows Figure 4The green line 3 shown in FIG. Then GPU0 can directly determine the virtual address of the target storage space in GPU memory n to be accessed according to the virtual address of GPU memory n recorded in the PCIe outbound window, and generate an access request using the virtual address as the target address. Then the access request is sent to PCIe switch 0 connected to GPU memory 0. After PCIe switch 0 is determined (see above for details), it will directly send the access request to host CPU 10. After receiving the access request, host CPU 10 first determines the target physical address corresponding to the target address in the access request based on the total address mapping relationship stored internally. Then further determine the target GPU "GPUn" corresponding to the target memory "GPU memory n" corresponding to the target physical address, and then send the access request to the target PCIe switch "PCIe switch m" connected to GPUn. After the target PCIe switch "PCIe switch m" is determined (see above for details), it will determine the target physical address corresponding to the target address, and then send the access request to the target GPU "GPUn" corresponding to the target physical address. After receiving the access request, the target GPU "GPUn" reads or writes data based on the target physical address corresponding to the target address in the access request to implement memory access.

[0087] like Figure 4 As shown, if the first GPU of the requesting party is GPU0 and the target memory is host memory 11, the control process of the access request is as follows Figure 4 The orange line 2 shown in . GPU0 can directly determine the virtual address of the target storage space to be accessed in the host memory 11 according to the virtual address of the host memory 11 recorded in the PCIe outbound window, and generate an access request using the virtual address as the target address. The access request is then sent to the PCIe switch 0 to which the GPU memory 0 is connected. After determination (see above for details), the PCIe switch 0 will directly send the access request to the host CPU 10. After receiving the access request, the host CPU 10 first determines the target address in the access request based on the total address mapping relationship of the internal storage to determine that the target memory to be accessed is the host memory 11, and then directly reads or writes data according to the target physical address corresponding to the target address to achieve memory access.

[0088] The memory access process under configuration strategy 2 is as follows:

[0089] The host CPU 10 and any GPU 20 in the server can act as requesters to generate access requests for a target storage space in the target memory (a storage space in the target memory). The target memory can be any GPU memory 21 or host memory 11. The access request can indicate a target address corresponding to the target storage space.

[0090] If the requester is the host CPU 10 and the target memory is a GPU memory 21, the host CPU 10 can directly determine the target physical address of the target storage space in the target memory to be accessed based on the recorded physical addresses of the GPU memories 21, and generate an access request using the target physical address as the target address, and then send the access request to the target PCIe switch to which the GPU of the target memory is connected.

[0091] If the requester is the first GPU among multiple GPUs and the target memory is a GPU memory 21 or a host memory 11, the first GPU can directly determine the virtual address of the target storage space in the target memory to be accessed according to the virtual address recorded in the PCIe outbound window, and generate an access request using the virtual address as the target address, and then send the access request to the PCIe switch to which the first GPU is connected.

[0092] The host CPU 10 may directly access the host memory 11 , and each GPU 20 may directly access its own GPU memory 21 . This may be set with reference to existing implementations, and the present disclosure does not impose any limitation on this.

[0093] For any PCIe switch 30: if an access request is received from a GPU 20 connected to the downstream port, the access request is directly sent to the host CPU 20. If an access request is received from a host CPU 10 connected to the upstream port, the target physical address corresponding to the target address in the access request is first determined based on the total address mapping relationship and / or physical address stored internally (the physical address stored in the PCIe switch 30 may include the physical address of the host memory 11 and the physical address of each GPU, or the physical address stored in the PCIe switch 30 may include the physical address of the host memory 11 and the physical address of the GPU that the GPU connected to the downstream port needs to access), and the access request is sent to the target GPU corresponding to the target physical address, which is the GPU 20 that can access the target memory indicated by the target address. After receiving the access request, the target GPU reads or writes data based on the target physical address corresponding to the access request to achieve memory access.

[0094] For the host CPU 10: If an access request forwarded by the PCIe switch 30 is received, the target physical address corresponding to the target address in the access request is first determined based on the total address mapping relationship stored internally. If it is then determined that the target memory corresponding to the target physical address is a GPU memory 21, the target GPU corresponding to the target memory corresponding to the target physical address is further determined, and then the access request is sent to the PCIe switch 30 (i.e., the target PCIe switch) to which the target GPU is connected. The target GPU is the GPU 20 that can access the target memory indicated by the target address. If it is determined that the target memory corresponding to the target physical address is the host memory 11, the host CPU 10 directly executes the access request, reads or writes data to the target physical address, and implements memory access.

[0095] For example,

[0096] like Figure 5 As shown, if the requester is the host CPU 10 and the target memory is the GPU memory 1, the control process of the access request is as follows Figure 5 The purple line 4 shown in . The host CPU 10 can directly generate an access request based on the physical address of GPU memory 1, where the physical address of the target storage space in GPU memory 1 is the target address and also the target physical address. Then the access request is sent to the PCIe switch 0 connected to GPU1. After determination (see above for details), PCIe switch 0 will directly send the access request to the corresponding GPU1 (that is, the target GPU). After receiving the access request, GPU1 reads or writes data based on the target physical address (that is, the target address) in the access request to achieve memory access.

[0097] like Figure 5 As shown, if the first GPU of the requesting party is GPU0 and the target memory is GPU memory 1, the control process of the access request is as follows Figure 5The blue line 1 shown in FIG. Then GPU0 can directly determine the virtual address of the target storage space in GPU memory 1 to be accessed according to the virtual address recorded in the PCIe outbound window, and generate an access request using the virtual address as the target address. Then the access request is sent to PCIe switch 0. PCIe switch 0 directly sends the access request to host CPU 10. After receiving the access request, host CPU 10 first determines the target physical address corresponding to the target address in the access request based on the total address mapping relationship stored internally. Then the target GPU "GPU1" corresponding to the target memory "GPU memory 1" corresponding to the target physical address is further determined, and then the access request is sent to PCIe switch 0 (i.e., target PCIe switch) connected to GPU1. The target PCIe switch "PCIe switch 0" determines the target physical address corresponding to the target address in the access request, and then sends the access request to the target GPU "GPU1" corresponding to the target physical address. After receiving the access request, the target GPU "GPU1" reads or writes data based on the target physical address corresponding to the target address in the access request to achieve memory access.

[0098] like Figure 5 As shown, if the first GPU of the requesting party is GPU0 and the target memory is GPU memory n, the control process of the access request is as follows Figure 5 The green line 3 shown in . Then GPU0 can directly determine the virtual address of the target storage space in GPU memory n to be accessed according to the virtual address of GPU memory n recorded in the PCIe outbound window, and generate an access request using the virtual address as the target address. Then the access request is sent to PCIe switch 0 to which GPU memory 0 is connected. PCIe switch 0 directly sends the access request to host CPU 10. After receiving the access request, host CPU 10 first determines the target physical address corresponding to the target address in the access request based on the total address mapping relationship stored internally. Then further determine the target GPU "GPUn" corresponding to the target memory "GPU memory n" corresponding to the target physical address, and then send the access request to PCIe switch m (that is, the target PCIe switch) connected to GPUn. After determination (see above for details), PCIe switch m will determine the target physical address corresponding to the target address, and then send the access request to GPUn (that is, the target GPU) corresponding to the target physical address. After receiving the access request, GPUn reads or writes data based on the target physical address corresponding to the target address in the access request to implement memory access.

[0099] like Figure 5As shown, if the first GPU of the requesting party is GPU0 and the target memory is host memory 11, the control process of the access request is as follows Figure 5 The orange line 2 shown in FIG. Then GPU0 can directly determine the virtual address of the target storage space in the host memory 11 to be accessed according to the virtual address of the host memory 11 recorded in the PCIe outbound window, and generate an access request using the virtual address as the target address. Then the access request is sent to the PCIe switch 0 to which the GPU memory 0 is connected. The PCIe switch 0 directly sends the access request to the host CPU 10. After receiving the access request, the host CPU 10 first determines the target address in the access request based on the total address mapping relationship stored internally to determine that the target memory to be accessed is the host memory 11, and then directly reads or writes data according to the target physical address corresponding to the target address to achieve memory access.

[0100] In this embodiment, during the process of the server executing the access control method to implement memory access, the server controls the redirection function (Redirection) of closing the downstream port of each PCIe switch 30 .

[0101] The embodiment of the present disclosure also provides an access control method, which is applied to a host CPU in a server, wherein the server further comprises a plurality of GPUs, a host memory of the host CPU, a GPU memory of each of the GPUs, and a plurality of PCIe switches, wherein an uplink port of each of the PCIe switches is connected to the host CPU, the plurality of GPUs are divided into a plurality of groups, and a plurality of GPUs in the same group are connected to different downlink ports of the same PCIe switch, wherein the method comprises:

[0102] Upon receiving the access request forwarded by the PCIe switch, determining a target physical address corresponding to a target address in the access request, and sending the access request to a corresponding target PCIe switch according to the target physical address, so that the target PCIe switch sends the access request to a target GPU where the target memory is located according to the target physical address, so that the access request is executed by the target GPU;

[0103] The access request is generated by a first GPU among the multiple GPUs according to a target address and then sent to a PCIe switch, and forwarded to the host CPU when the PCIe switch determines that the target address is a virtual address, the target address is determined by the first GPU according to an address recorded in a PCIe outbound window of the first GPU and a target memory to be accessed, and the target memory is any one of the remaining GPU memories among the multiple GPU memories that are different from the GPU memory of the first GPU and the host memory;

[0104] The addresses recorded in the PCIe outbound window of each GPU include at least one of the following: the virtual address of the host memory, the virtual address of the GPU memory corresponding to the out-of-group GPU with access requirements among the remaining GPUs in a different group from the GPU, and the physical address of the GPU memory corresponding to the in-group GPU with access requirements among the remaining GPUs in the same group as the GPU.

[0105] The disclosed embodiment also provides another access control method, which is applied to a host CPU in a server. The server further includes multiple GPUs, a host memory of the host CPU, GPU memory of each GPU, and multiple PCIe switches. The uplink port of each PCIe switch is connected to the host CPU. The multiple GPUs are divided into multiple groups. Multiple GPUs in the same group are connected to different downlink ports of the same PCIe switch. The method includes:

[0106] Upon receiving the access request forwarded by the PCIe switch, determining a target physical address corresponding to a target address in the access request, and sending the access request to a corresponding target PCIe switch according to the target physical address, so that the target PCIe switch sends the access request to a target GPU where the target memory is located according to the target physical address, so that the access request is executed by the target GPU;

[0107] The access request is generated by a first GPU among the multiple GPUs according to a target address and then sent to a PCIe switch and forwarded to the host CPU by the PCIe switch, the target address is determined by the first GPU according to an address recorded in a PCIe outbound window of the first GPU and a target memory to be accessed, and the target memory is any one of the remaining GPU memories among the multiple GPU memories that are different from the GPU memory of the first GPU and the host memory;

[0108] The address recorded in the PCIe outbound window of each GPU includes at least one of the following: a virtual address of the host memory, and a virtual address of a GPU memory corresponding to a GPU having access requirements among the remaining GPUs different from the GPU.

[0109] The embodiment of the present disclosure also provides an access control method, which is applied to a PCIe switch in a server, wherein the server includes a host CPU, multiple GPUs, a host memory of the host CPU, GPU memory of each of the GPUs, and multiple PCIe switches, wherein an uplink port of each of the PCIe switches is connected to the host CPU, the multiple GPUs are divided into multiple groups, and multiple GPUs in the same group are connected to different downlink ports of the same PCIe switch, wherein the method includes:

[0110] In the case of receiving an access request sent by a first GPU among the multiple GPUs, forwarding the access request to the host CPU; the access request is generated by the first GPU according to a target address, the target address is determined by the first GPU according to an address recorded in a PCIe outbound window of the first GPU and a target memory to be accessed, and the target memory is any one of the remaining GPU memories among the multiple GPU memories that are different from the GPU memory of the first GPU and the host memory; or,

[0111] When receiving an access request from the host CPU, sending the access request to a target GPU where the target memory is located according to a target physical address corresponding to the access request, so that the access request is executed by the target GPU;

[0112] The address recorded in the PCIe outbound window of each GPU includes at least one of the following: a virtual address of the host memory, and a virtual address of a GPU memory corresponding to a GPU with access requirements among the remaining GPUs different from the GPU.

[0113] The embodiment of the present disclosure also provides an access control method, which is applied to a PCIe switch in a server, wherein the server includes a host CPU, multiple GPUs, a host memory of the host CPU, GPU memory of each of the GPUs, and multiple PCIe switches, wherein an uplink port of each of the PCIe switches is connected to the host CPU, the multiple GPUs are divided into multiple groups, and multiple GPUs in the same group are connected to different downlink ports of the same PCIe switch, wherein the method includes:

[0114] Upon receiving an access request sent from a first GPU among the multiple GPUs, forwarding the access request whose target address is a virtual address to the host CPU, and forwarding the access request whose target address is a physical address to a target GPU where the target memory is located and connected to a downstream port of the PCIe switch; the access request is generated by the first GPU according to the target address, the target address is determined by the first GPU according to an address recorded in a PCIe outbound window of the first GPU and a target memory to be accessed, and the target memory is any one of the remaining GPU memories among the multiple GPU memories that are different from the GPU memory of the first GPU and the host memory; or,

[0115] When receiving an access request from the host CPU, sending the access request to a target GPU where the target memory is located according to a target physical address corresponding to the access request, so that the access request is executed by the target GPU;

[0116] Among them, the address recorded in the PCIe outbound window of each GPU includes at least one of the following: the virtual address of the host memory, the virtual address of the GPU memory corresponding to the out-of-group GPU with access requirements among the remaining GPUs in a different group from the GPU, and the physical address of the GPU memory corresponding to the in-group GPU with access requirements among the remaining GPUs in the same group as the GPU.

[0117] The embodiment of the present disclosure also provides an access control method, which is applied to a GPU in a server, wherein the server includes a host CPU, multiple GPUs, a host memory of the host CPU, GPU memory of each of the GPUs, and multiple PCIe switches, wherein an uplink port of each of the PCIe switches is connected to the host CPU, the multiple GPUs are divided into multiple groups, and multiple GPUs in the same group are connected to different downlink ports of the same PCIe switch, wherein the method includes:

[0118] A first GPU among the multiple GPUs determines a target address of a target memory to be accessed according to an address recorded in a PCIe outbound window corresponding to the first GPU, and sends an access request generated according to the target address to a connected PCIe switch, so that the access request is forwarded to the host CPU by the PCIe switch, so that the host CPU determines a target physical address corresponding to the target address in the access request, and sends the access request to a corresponding target PCIe switch according to the target physical address, so that the target PCIe switch sends the access request to a target GPU where the target memory is located according to the target physical address;

[0119] When receiving the access request, the target GPU among the multiple GPUs executes the access request;

[0120] The target memory is any one of the remaining GPU memories different from the GPU memory of the first GPU among the multiple GPU memories and the host memory;

[0121] The addresses recorded in the PCIe outbound window of each of the GPUs include at least one of the following: the virtual address of the host memory, the virtual address of the GPU memory corresponding to the GPU outside the group that has access requirements among the remaining GPUs in a different group from the GPU, and the physical address of the GPU memory corresponding to the GPU within the group that has access requirements among the remaining GPUs in the same group as the GPU; or, the addresses recorded in the PCIe outbound window of each of the GPUs include at least one of the following: the virtual address of the host memory, the virtual address of the GPU memory corresponding to the GPU among the remaining GPUs that have access requirements that are different from the GPU.

[0122] It should be noted that although the access control method and the server are described above by taking the above embodiments as examples, those skilled in the art will appreciate that the present disclosure should not be limited thereto. In fact, the user can flexibly set the method steps executed by each step and each hardware according to personal preferences and / or actual application scenarios, as long as it complies with the technical solution of the present disclosure.

[0123] The embodiment of the present disclosure also provides a computer-readable storage medium on which computer program instructions are stored, and the computer program instructions implement the above method when executed by a processor. The computer-readable storage medium can be a volatile or non-volatile computer-readable storage medium.

[0124] An embodiment of the present disclosure further proposes an electronic device, comprising: a processor; and a memory for storing instructions executable by the processor; wherein the processor is configured to implement the above method when executing the instructions stored in the memory.

[0125] The embodiments of the present disclosure also provide a computer program product, including a computer-readable code, or a non-volatile computer-readable storage medium carrying the computer-readable code. When the computer-readable code runs in a processor of an electronic device, the processor in the electronic device executes the above method.

[0126] The present disclosure may be a system, a method and / or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for causing a processor to implement various aspects of the present disclosure.

[0127] A computer-readable storage medium may be a tangible device that can hold and store instructions used by an instruction execution device. A computer-readable storage medium may be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. More specific examples of computer-readable storage media (a non-exhaustive list) include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disk read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device, such as a punch card or a raised structure in a groove on which instructions are stored, and any suitable combination of the foregoing. As used herein, a computer-readable storage medium is not to be interpreted as a transient signal per se, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagating through a waveguide or other transmission medium (e.g., a light pulse through a fiber optic cable), or an electrical signal transmitted through a wire.

[0128] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, optical fiber transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in the computer-readable storage medium in each computing / processing device.

[0129] The computer program instructions for performing the operation of the present disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-related instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages, such as Smalltalk, C++, etc., and conventional procedural programming languages, such as "C" language or similar programming languages. Computer-readable program instructions may be executed completely on a user's computer, partially on a user's computer, as an independent software package, partially on a user's computer, partially on a remote computer, or completely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., using an Internet service provider to connect via the Internet). In some embodiments, an electronic circuit, such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), may be customized by utilizing the state information of the computer-readable program instructions, and the electronic circuit may execute the computer-readable program instructions, thereby realizing various aspects of the present disclosure.

[0130] Various aspects of the present disclosure are described herein with reference to the flowcharts and / or block diagrams of the methods, devices (systems) and computer program products according to the embodiments of the present disclosure. It should be understood that each box in the flowchart and / or block diagram and the combination of each box in the flowchart and / or block diagram can be implemented by computer-readable program instructions.

[0131] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the processor of the computer or other programmable data processing device, a device that implements the functions / actions specified in one or more boxes in the flowchart and / or block diagram is generated. These computer-readable program instructions can also be stored in a computer-readable storage medium, and these instructions cause the computer, programmable data processing device, and / or other equipment to work in a specific manner, so that the computer-readable medium storing the instructions includes a manufactured product, which includes instructions for implementing various aspects of the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0132] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operating steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to implement the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0133] The flow chart and block diagram in the accompanying drawings show the possible architecture, function and operation of the system, method and computer program product according to multiple embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a part of a module, program segment or instruction, and a part of the module, program segment or instruction includes one or more executable instructions for realizing the specified logical function. In some alternative implementations, the function marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two continuous square boxes can actually be executed substantially in parallel, and they can sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs the specified function or action, or can be implemented with a combination of special hardware and computer instructions.

[0134] The embodiments of the present disclosure have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and changes will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or technical improvements in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A server, characterized in that: The system comprises a host CPU, multiple GPUs, a host memory of the host CPU, GPU memory of each of the GPUs, and multiple PCIe switches, wherein an uplink port of each of the PCIe switches is connected to the host CPU, the multiple GPUs are divided into multiple groups, and multiple GPUs in the same group are connected to different downlink ports of the same PCIe switch. A first GPU among the multiple GPUs is used to determine a target address corresponding to a target memory to be accessed according to an address recorded in a PCIe outbound window of the first GPU, and send an access request generated according to the target address to a PCIe switch connected to the first GPU, wherein the target memory is any one of remaining GPU memories among the multiple GPU memories that are different from the GPU memory of the first GPU and the host memory; The PCIe switch is configured to, upon receiving an access request, forward the access request whose target address is a virtual address to the host CPU, and forward the access request whose target address is a physical address to a target GPU where the target memory is located and which is connected to a downstream port of the PCIe switch; Among them, the address recorded in the PCIe outbound window of each GPU includes at least one of the following: the virtual address of the host memory, the virtual address of the GPU memory corresponding to the out-of-group GPU with access requirements among the remaining GPUs in a different group from the GPU, and the physical address of the GPU memory corresponding to the in-group GPU with access requirements among the remaining GPUs in the same group as the GPU.

2. A server, characterized in that: The system comprises a host CPU, multiple GPUs, a host memory of the host CPU, GPU memory of each of the GPUs, and multiple PCIe switches, wherein an uplink port of each of the PCIe switches is connected to the host CPU, the multiple GPUs are divided into multiple groups, and multiple GPUs in the same group are connected to different downlink ports of the same PCIe switch. A first GPU among the multiple GPUs is used to determine a target address corresponding to a target memory to be accessed according to an address recorded in a PCIe outbound window of the first GPU, and send an access request generated according to the target address to a PCIe switch connected to the first GPU, wherein the target memory is any one of remaining GPU memories among the multiple GPU memories that are different from the GPU memory of the first GPU and the host memory; The PCIe switch is used to forward the access request to the host CPU when receiving the access request; The address recorded in the PCIe outbound window of each GPU includes at least one of the following: a virtual address of the host memory, and a virtual address of a GPU memory corresponding to a GPU with access requirements among the remaining GPUs different from the GPU.

3. The server according to claim 1 or 2, characterized in that: The host CPU is used to determine, when receiving an access request, a target physical address corresponding to a target address in the access request, and send the access request to a corresponding target PCIe switch according to the target physical address; The target PCIe switch is used to send the access request to the target GPU where the target memory is located according to the target physical address, so that the access request is executed by the target GPU.

4. The server according to claim 3, characterized in that: Determining a target physical address corresponding to a target address in the access request includes: Determine the target physical address corresponding to the target address in the access request according to the total address mapping relationship or the physical address of each GPU; or Determine, according to the address sub-mapping relationship corresponding to the first GPU, a target physical address corresponding to the target address in the access request; The total address mapping relationship is used to indicate the mapping relationship between the virtual address and the physical address of the host memory and the mapping relationship between the virtual address and the physical address of each GPU memory with access requirements; The address sub-mapping relationship corresponding to the first GPU is used to indicate the mapping relationship between the virtual address and the physical address of the host memory and the mapping relationship between the virtual address and the physical address of the GPU memory corresponding to each of the out-of-group GPUs.

5. The server according to claim 4, characterized in that: The host CPU is used to: configuring virtual addresses for the host memory and the GPU memory with access requirements, wherein the GPU memory with access requirements is part or all of the multiple GPU memories in the server; Mapping the virtual address of the host memory to the physical address of the host memory by using a memory management unit, and mapping the virtual address of each GPU memory with access requirements to the physical address of the GPU memory; Creating a total address mapping relationship and an address sub-mapping relationship of each GPU according to a mapping relationship between a virtual address and a physical address of the host memory and a mapping relationship between a virtual address and a physical address of the GPU memory; The PCIe outbound window of each GPU is configured based on the address sub-mapping relationship of each GPU.

6. The server according to claim 1 or 2, characterized in that: Control each of the PCIe switches to disable the redirection function of the downstream port.

7. The server according to claim 5, characterized in that: The range of virtual addresses configured for the host memory and the GPU memory with access requirements is within the restricted address range of the PCIe outbound window of each GPU.

8. An access control method, characterized in that: The method is applied to a host CPU in a server, wherein the server further comprises a plurality of GPUs, a host memory of the host CPU, a GPU memory of each of the GPUs, and a plurality of PCIe switches, wherein an uplink port of each of the PCIe switches is connected to the host CPU, the plurality of GPUs are divided into a plurality of groups, and a plurality of GPUs in the same group are connected to different downlink ports of the same PCIe switch, and the method comprises: Upon receiving the access request forwarded by the PCIe switch, determining a target physical address corresponding to a target address in the access request, and sending the access request to a corresponding target PCIe switch according to the target physical address, so that the target PCIe switch sends the access request to a target GPU where the target memory is located according to the target physical address, so that the access request is executed by the target GPU; The access request is generated by a first GPU among the multiple GPUs according to a target address and then sent to a PCIe switch, and forwarded to the host CPU when the PCIe switch determines that the target address is a virtual address, the target address is determined by the first GPU according to an address recorded in a PCIe outbound window of the first GPU and a target memory to be accessed, and the target memory is any one of the remaining GPU memories among the multiple GPU memories that are different from the GPU memory of the first GPU and the host memory; The addresses recorded in the PCIe outbound window of each GPU include at least one of the following: the virtual address of the host memory, the virtual address of the GPU memory corresponding to the out-of-group GPU with access requirements among the remaining GPUs in a different group from the GPU, and the physical address of the GPU memory corresponding to the in-group GPU with access requirements among the remaining GPUs in the same group as the GPU.

9. An access control method, characterized in that: The method is applied to a host CPU in a server, wherein the server further comprises a plurality of GPUs, a host memory of the host CPU, a GPU memory of each of the GPUs, and a plurality of PCIe switches, wherein an uplink port of each of the PCIe switches is connected to the host CPU, the plurality of GPUs are divided into a plurality of groups, and a plurality of GPUs in the same group are connected to different downlink ports of the same PCIe switch, and the method comprises: Upon receiving the access request forwarded by the PCIe switch, determining a target physical address corresponding to a target address in the access request, and sending the access request to a corresponding target PCIe switch according to the target physical address, so that the target PCIe switch sends the access request to a target GPU where the target memory is located according to the target physical address, so that the access request is executed by the target GPU; The access request is generated by a first GPU among the multiple GPUs according to a target address and then sent to a PCIe switch and forwarded to the host CPU by the PCIe switch, the target address is determined by the first GPU according to an address recorded in a PCIe outbound window of the first GPU and a target memory to be accessed, and the target memory is any one of the remaining GPU memories among the multiple GPU memories that are different from the GPU memory of the first GPU and the host memory; The address recorded in the PCIe outbound window of each GPU includes at least one of the following: a virtual address of the host memory, and a virtual address of a GPU memory corresponding to a GPU having access requirements among the remaining GPUs different from the GPU.

10. An access control method, characterized in that: A PCIe switch applied to a server, the server comprising a host CPU, multiple GPUs, a host memory of the host CPU, GPU memory of each of the GPUs, and multiple PCIe switches, the uplink port of each of the PCIe switches being connected to the host CPU, the multiple GPUs being divided into multiple groups, multiple GPUs in the same group being connected to different downlink ports of the same PCIe switch, the method comprising: In the case of receiving an access request sent by a first GPU among the multiple GPUs, forwarding the access request to the host CPU; the access request is generated by the first GPU according to a target address, the target address is determined by the first GPU according to an address recorded in a PCIe outbound window of the first GPU and a target memory to be accessed, and the target memory is any one of the remaining GPU memories among the multiple GPU memories that are different from the GPU memory of the first GPU and the host memory; or, When receiving an access request from the host CPU, sending the access request to a target GPU where the target memory is located according to a target physical address corresponding to the access request, so that the access request is executed by the target GPU; The address recorded in the PCIe outbound window of each GPU includes at least one of the following: a virtual address of the host memory, and a virtual address of a GPU memory corresponding to a GPU with access requirements among the remaining GPUs different from the GPU.

11. An access control method, characterized in that: A PCIe switch applied to a server, the server comprising a host CPU, multiple GPUs, a host memory of the host CPU, GPU memory of each of the GPUs, and multiple PCIe switches, the uplink port of each of the PCIe switches being connected to the host CPU, the multiple GPUs being divided into multiple groups, multiple GPUs in the same group being connected to different downlink ports of the same PCIe switch, the method comprising: Upon receiving an access request sent from a first GPU among the multiple GPUs, forwarding the access request whose target address is a virtual address to the host CPU, and forwarding the access request whose target address is a physical address to a target GPU where the target memory is located and connected to a downstream port of the PCIe switch; the access request is generated by the first GPU according to the target address, the target address is determined by the first GPU according to an address recorded in a PCIe outbound window of the first GPU and a target memory to be accessed, and the target memory is any one of the remaining GPU memories among the multiple GPU memories that are different from the GPU memory of the first GPU and the host memory; or, When receiving an access request from the host CPU, sending the access request to a target GPU where the target memory is located according to a target physical address corresponding to the access request, so that the access request is executed by the target GPU; Among them, the address recorded in the PCIe outbound window of each GPU includes at least one of the following: the virtual address of the host memory, the virtual address of the GPU memory corresponding to the out-of-group GPU with access requirements among the remaining GPUs in a different group from the GPU, and the physical address of the GPU memory corresponding to the in-group GPU with access requirements among the remaining GPUs in the same group as the GPU.

12. An access control method, characterized in that: A GPU applied to a server, the server comprising a host CPU, multiple GPUs, a host memory of the host CPU, GPU memory of each of the GPUs, and multiple PCIe switches, the uplink port of each of the PCIe switches being connected to the host CPU, the multiple GPUs being divided into multiple groups, multiple GPUs in the same group being connected to different downlink ports of the same PCIe switch, the method comprising: A first GPU among the multiple GPUs determines a target address of a target memory to be accessed according to an address recorded in a PCIe outbound window corresponding to the first GPU, and sends an access request generated according to the target address to a connected PCIe switch, so that the access request is forwarded to the host CPU by the PCIe switch, so that the host CPU determines a target physical address corresponding to the target address in the access request, and sends the access request to a corresponding target PCIe switch according to the target physical address, so that the target PCIe switch sends the access request to a target GPU where the target memory is located according to the target physical address; When receiving the access request, the target GPU among the multiple GPUs executes the access request; The target memory is any one of the remaining GPU memories different from the GPU memory of the first GPU among the multiple GPU memories and the host memory; The addresses recorded in the PCIe outbound window of each of the GPUs include at least one of the following: the virtual address of the host memory, the virtual address of the GPU memory corresponding to the GPU outside the group that has access requirements among the remaining GPUs in a different group from the GPU, and the physical address of the GPU memory corresponding to the GPU within the group that has access requirements among the remaining GPUs in the same group as the GPU; or, the addresses recorded in the PCIe outbound window of each of the GPUs include at least one of the following: the virtual address of the host memory, the virtual address of the GPU memory corresponding to the GPU among the remaining GPUs that have access requirements that are different from the GPU.

13. An access control device, characterized in that: include: processor; a memory for storing processor-executable instructions; Wherein, the processor is configured to implement the method described in any one of claims 8 to 12 when executing the instructions stored in the memory.

14. A non-volatile computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by a processor, the method according to any one of claims 8 to 12 is implemented.

Images