Access control method, apparatus and server

Abstract

The present disclosure relates to an access control method, device and server. The server comprises a host CPU, a plurality of GPUs, a host memory of the host CPU, a GPU memory of each GPU and a plurality of PCIe switches, the uplink port of each PCIe switch is connected to the host CPU, the plurality of GPUs are divided into a plurality of groups, the plurality of GPUs in the same group are connected to different downlink ports of the same PCIe switch, a first GPU determines a target address corresponding to a target memory to be accessed, and generates an access request according to the target address and sends the access request to the connected PCIe switch; the PCIe switch forwards the access request with the target address as a virtual address to the host CPU, and forwards the access request with the target address as a physical address to the target GPU where the target memory is located and connected to the downlink port of the PCIe switch. The limitation of the GPU access address range is broken, and the speed and efficiency of memory access in the server are guaranteed.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of computer, and particularly relates to an access control method and device and a server. BACKGROUND

[0002] GPU server is a kind of GPU-based application in video coding, deep learning, scientific computing and other various scenes of fast, stable and flexible computing services. Such server is equipped with multiple GPUs, and also has large memory, high-speed storage and network connection functions, and is usually used in occasions requiring high-performance computing such as processing large-scale data, deep learning model training, image recognition, speech recognition, natural language processing, etc. The common topology structure of GPU server is as shown in Figure 1 GPU accesses the host memory (Host Memory) and the PCIe memory base address register (PCIe Memory Base Address Registe) of other GPUs through PCIe device outbound communication (PCIe Outbound). In order to complete this function, the PCIe outbound window (PCIe outbound Window) of each GPU needs to be configured as shown in Figure 2 Assuming that the address width that the GPU can access the PCIe is 39 bits, the address range that the GPU can access is limited to 0-512G, which makes the sum of the host memory and the memory of each GPU cannot exceed 512G. How to break through this limitation is a technical problem to be solved. SUMMARY

[0003] Therefore, the present disclosure provides an access control method, device and server.

[0004] According to an aspect of the present disclosure, a server is provided, which comprises a host CPU, a plurality of GPUs, a host memory of the host CPU, a GPU memory of each of the GPUs, and a plurality of PCIe switches, an uplink port of each of the PCIe switches being connected to the host CPU, the plurality of GPUs being divided into a plurality of groups, a plurality of GPUs in a same group being connected to different downlink ports of a same PCIe switch,

[0005] A first GPU in the plurality of GPUs is configured to determine a target address corresponding to a target memory to be accessed according to an address recorded in a PCIe outbound window of the first GPU, and send an access request generated according to the target address to a PCIe switch connected to the first GPU, the target memory being any one of the remaining GPU memories in the plurality of GPU memories different from the GPU memory of the first GPU and the host memory.

[0006] The PCIe switch is configured to, in response to receiving an access request, forward the access request with a target address being a virtual address to the host CPU, and forward the access request with a target address being a physical address to a target GPU where the target memory is located and which is connected to a downstream port of the PCIe switch.

[0007] The address recorded in the PCIe outbound window of each GPU includes at least one of a virtual address of the host memory, a virtual address of a GPU memory corresponding to a group-out GPU having an access demand in the remaining GPUs other than the GPU, and a physical address of a GPU memory corresponding to a group-in GPU having an access demand in the remaining GPUs in the same group as the GPU.

[0008] According to an aspect of the present disclosure, a server is provided, which includes a host CPU, a plurality of GPUs, a host memory of the host CPU, a GPU memory of each of the GPUs, and a plurality of PCIe switches, an upstream port of each of the PCIe switches being connected to the host CPU, the plurality of GPUs being divided into a plurality of groups, a plurality of GPUs in the same group being connected to different downstream ports of the same PCIe switch,

[0009] A first GPU in the plurality of GPUs is configured to determine a target address corresponding to a target memory having an access demand according to an address recorded in a PCIe outbound window of the first GPU, and send an access request generated according to the target address to a PCIe switch connected to the first GPU, the target memory being any one of the remaining GPU memories other than the GPU memory of the first GPU and the host memory.

[0010] The PCIe switch is configured to, in response to receiving an access request, forward the access request to the host CPU.

[0011] The address recorded in the PCIe outbound window of each GPU includes at least one of a virtual address of the host memory, a virtual address of a GPU memory corresponding to a GPU having an access demand in the remaining GPUs other than the GPU.

[0012] In a possible implementation, the host CPU is configured to, in response to receiving an access request, determine a target physical address corresponding to a target address in the access request, and send the access request to a corresponding target PCIe switch according to the target physical address.

[0013] The target PCIe switch is configured to send the access request to a target GPU where the target memory is located according to the target physical address, so that the access request is executed by the target GPU.

[0014] In a possible implementation, determining the target physical address corresponding to the target address in the access request comprises:

[0015] According to the total address mapping relationship or the physical address of each GPU, the target physical address corresponding to the target address in the access request is determined; or

[0016] According to the address sub-mapping relationship corresponding to the first GPU, the target physical address corresponding to the target address in the access request is determined.

[0017] The total address mapping relationship is used to indicate the mapping relationship between the virtual address and the physical address of the host memory, and the mapping relationship between the virtual address and the physical address of each GPU memory having access demand.

[0018] The address sub-mapping relationship corresponding to the first GPU is used to indicate the mapping relationship between the virtual address and the physical address of the host memory, and the mapping relationship between the virtual address and the physical address of the GPU memory corresponding to each out-group GPU.

[0019] In a possible implementation, the host CPU is configured to:

[0020] The host memory and the GPU memory having access demand are configured with virtual addresses, and the GPU memory having access demand is part or all of the plurality of GPU memories in the server.

[0021] The memory management unit is used to map the virtual address of the host memory to the physical address of the host memory, and map the virtual address of each GPU memory having access demand to the physical address of the GPU memory.

[0022] According to the mapping relationship between the virtual address and the physical address of the host memory and the mapping relationship between the virtual address and the physical address of the GPU memory, a total address mapping relationship and an address sub-mapping relationship of each GPU are created.

[0023] Based on the address sub-mapping relationship of each GPU, a PCIe outbound window of each GPU is configured.

[0024] In a possible implementation, each PCIe switch is controlled to close the redirection function of the downstream port.

[0025] In a possible implementation, the range of the virtual addresses configured for the host memory and the GPU memory having access demand is within the limit address range of the PCIe outbound window of each GPU.

[0026] According to an aspect of the present disclosure, an access control method is provided, which is applied to a host CPU in a server, the server further comprising a plurality of GPUs, a host memory of the host CPU, GPU memories of the GPUs, and a plurality of PCIe switches, an uplink port of each of the PCIe switches being connected to the host CPU, the plurality of GPUs being divided into a plurality of groups, and the plurality of GPUs in the same group being connected to different downlink ports of the same PCIe switch, the method comprising:

[0027] In a case where the access request forwarded by the PCIe switch is received, a target physical address corresponding to a target address in the access request is determined, and the access request is sent to a corresponding target PCIe switch according to the target physical address, so that the target PCIe switch sends the access request to a target GPU where the target memory is located according to the target physical address, so that the access request is executed by the target GPU;

[0028] The access request is generated by a first GPU in the plurality of GPUs according to a target address and sent to the PCIe switch, and in a case where the PCIe switch determines that the target address is a virtual address, the access request is forwarded to the host CPU, the target address is determined by the first GPU according to an address recorded in a PCIe outbound window of the first GPU and a target memory to be accessed, and the target memory is any one of the remaining GPU memories of the plurality of GPUs different from the GPU memory of the first GPU and the host memory.

[0029] The address recorded in the PCIe outbound window of each of the GPUs comprises at least one of the following: a virtual address of the host memory, a virtual address of a GPU memory corresponding to a group-out GPU in the remaining GPUs different from the group of the GPU and having an access demand, and a physical address of a GPU memory corresponding to a group-in GPU in the remaining GPUs in the same group as the GPU and having an access demand.

[0030] According to an aspect of the present disclosure, an access control method is provided, which is applied to a host CPU in a server, the server further comprising a plurality of GPUs, a host memory of the host CPU, GPU memories of the GPUs, and a plurality of PCIe switches, an uplink port of each of the PCIe switches being connected to the host CPU, the plurality of GPUs being divided into a plurality of groups, and the plurality of GPUs in the same group being connected to different downlink ports of the same PCIe switch, the method comprising:

[0031] In a case where the access request forwarded by the PCIe switch is received, a target physical address corresponding to a target address in the access request is determined, and the access request is sent to a corresponding target PCIe switch according to the target physical address, so that the target PCIe switch sends the access request to a target GPU where the target memory is located according to the target physical address, so that the access request is executed by the target GPU.

[0032] The access request is generated by a first GPU in the plurality of GPUs according to a target address and sent to the PCIe switch and forwarded to the host CPU by the PCIe switch, the target address is determined by the first GPU according to an address recorded in a PCIe outbound window of the first GPU and a target memory to be accessed, and the target memory is any one of the remaining GPU memories different from the GPU memory of the first GPU in the plurality of GPU memories and the host memory.

[0033] The address recorded in the PCIe outbound window of each GPU includes at least one of the following: a virtual address of the host memory, and a virtual address of a GPU memory corresponding to a GPU in the remaining GPUs different from the GPU and having an access demand.

[0034] According to an aspect of the present disclosure, an access control method is provided, which is applied to a PCIe switch in a server, the server including a host CPU, a plurality of GPUs, a host memory of the host CPU, GPU memories of the GPUs, and a plurality of PCIe switches, an uplink port of each PCIe switch being connected to the host CPU, the plurality of GPUs being divided into a plurality of groups, a plurality of GPUs in a same group being connected to different downlink ports of a same PCIe switch, and the method including:

[0035] In a case where an access request sent by a first GPU in the plurality of GPUs is received, the access request is forwarded to the host CPU, the access request being generated by the first GPU according to a target address, the target address being determined by the first GPU according to an address recorded in a PCIe outbound window of the first GPU and a target memory to be accessed, and the target memory being any one of the remaining GPU memories different from the GPU memory of the first GPU in the plurality of GPU memories and the host memory.

[0036] In a case where an access request from the host CPU is received, the access request is sent to a target GPU where a target memory is located according to a target physical address corresponding to the access request, so that the access request is executed by the target GPU;

[0037] The addresses recorded in the PCIe outbound windows of the GPUs include at least one of the following: a virtual address of the host memory, a virtual address of a GPU memory corresponding to a GPU in the remaining GPUs other than the GPU and having a demand for access.

[0038] According to an aspect of the present disclosure, an access control method is provided, which is applied to a PCIe switch in a server. The server includes a host CPU, a plurality of GPUs, a host memory of the host CPU, a GPU memory of each of the GPUs, and a plurality of PCIe switches. An uplink port of each of the PCIe switches is connected to the host CPU. The plurality of GPUs are divided into a plurality of groups. A plurality of GPUs in a same group are connected to different downlink ports of a same PCIe switch. The method includes:

[0039] In a case where an access request is received from a first GPU in the plurality of GPUs, the access request with the target address being a virtual address is forwarded to the host CPU, and the access request with the target address being a physical address is forwarded to a target GPU in which the target memory is located and which is connected to a downlink port of the PCIe switch. The access request is generated by the first GPU according to a target address. The target address is determined by the first GPU according to an address recorded in a PCIe outbound window of the first GPU and the target memory to be accessed. The target memory is any one of the remaining GPU memories in the plurality of GPU memories and the host memory other than the GPU memory of the first GPU. Alternatively,

[0040] In a case where an access request is received from the host CPU, the access request is sent to a target GPU in which a target memory is located according to a target physical address corresponding to the access request, so that the access request is executed by the target GPU.

[0041] The addresses recorded in the PCIe outbound windows of the GPUs include at least one of the following: a virtual address of the host memory, a virtual address of a GPU memory corresponding to a group-out GPU in the remaining GPUs other than the GPU and having a demand for access, and a physical address of a GPU memory corresponding to a group-in GPU in the remaining GPUs in the same group as the GPU and having a demand for access.

[0042] According to an aspect of the present disclosure, an access control method is provided, which is applied to a GPU in a server. The server includes a host CPU, a plurality of GPUs, a host memory of the host CPU, a GPU memory of each of the GPUs, and a plurality of PCIe switches. An uplink port of each of the PCIe switches is connected to the host CPU. The plurality of GPUs are divided into a plurality of groups. A plurality of GPUs in a same group are connected to different downlink ports of a same PCIe switch. The method includes:

[0043] The first GPU in the plurality of GPUs determines a target address of a target memory to be accessed according to an address recorded in a PCIe outbound window corresponding to the first GPU, and sends an access request generated according to the target address to a connected PCIe switch, so that the access request is forwarded by the PCIe switch to the host CPU, so that the host CPU determines a target physical address corresponding to the target address in the access request, and sends the access request to a corresponding target PCIe switch according to the target physical address, so that the target PCIe switch sends the access request to a target GPU where the target memory is located according to the target physical address;

[0044] The target GPU in the plurality of GPUs executes the access request when receiving the access request;

[0045] The target memory is any one of the remaining GPU memories different from the GPU memory of the first GPU and the host memory in the plurality of GPU memories.

[0046] The addresses recorded in the PCIe outbound window of each GPU include at least one of the following: a virtual address of the host memory, a virtual address of a GPU memory corresponding to a group-out GPU in the remaining GPUs having access demand and different from the group of the GPU, and a physical address of a GPU memory corresponding to a group-in GPU in the remaining GPUs having access demand and in the same group as the GPU; or the addresses recorded in the PCIe outbound window of each GPU include at least one of the following: a virtual address of the host memory, and a virtual address of a GPU memory corresponding to a GPU in the remaining GPUs having access demand and different from the GPU.

[0047] According to another aspect of the present disclosure, an access control apparatus is provided, comprising: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to implement the above method when executing the instructions stored in the memory.

[0048] According to another aspect of the present disclosure, a non-volatile computer readable storage medium having computer program instructions stored thereon is provided, wherein the computer program instructions are executed by a processor to implement the above method.

[0049] According to another aspect of the present disclosure, a computer program product is provided, comprising computer readable code or a non-volatile computer readable storage medium carrying computer readable code, when the computer readable code is run in a processor of an electronic device, the processor in the electronic device executes the above method.

[0050] The access control method, device and server provided by the embodiments of the present disclosure break through the limitation of the GPU access address range, and ensure the speed and efficiency of memory access in the server.

[0051] Other features and aspects of the present disclosure will become apparent from the following detailed description of exemplary embodiments, taken in conjunction with the accompanying drawings. BRIEF DESCRIPTION OF DRAWINGS

[0052] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate exemplary embodiments, features, and aspects of the present disclosure and serve to explain the principles of the present disclosure.

[0053] Figure 1 A schematic diagram of a topology of a server is shown.

[0054] Figure 2 A schematic diagram of configuration of a PCIe outbound window of a GPU in the related art is shown.

[0055] Figure 3 A schematic diagram of configuration of a PCIe outbound window of a GPU in the embodiments of the present disclosure is shown.

[0056] Figure 4 、 Figure 5 A schematic diagram of generation and execution process of an access request in the server in the embodiments of the present disclosure is shown. DETAILED DESCRIPTION

[0057] Various exemplary embodiments, features and aspects of the present disclosure will be described in detail below with reference to the accompanying drawings. The same reference numbers in the drawings represent the same or similar elements. Although various aspects of the embodiments are shown in the drawings, the drawings are not necessarily drawn to scale unless specifically indicated.

[0058] The word "exemplary" is used herein to mean "serving as an example, instance, or illustration." Any implementation described herein as "exemplary" is not necessarily to be construed as preferred or advantageous over other implementations.

[0059] In addition, in order to better illustrate the present disclosure, numerous specific details are given in the following detailed description. Those skilled in the art should understand that the present disclosure can also be implemented without some specific details. In some examples, methods, means, elements and circuits that are well known to those skilled in the art are not described in detail in order to highlight the main ideas of the present disclosure.

[0060] To solve the above technical problems, the present disclosure provides an access control method, device and server. The limitation of the GPU access address range is broken through, and the speed and efficiency of memory access in the server are ensured.

[0061] As Figure 1As shown, the server provided in this embodiment includes a host CPU 10, multiple GPUs 20, host memory 11 of the host CPU, GPU memory 21 of each GPU 20, and multiple PCIe switches 30. The uplink ports of each PCIe switch 30 are connected to the host CPU 10. The multiple GPUs 20 are divided into multiple groups, and multiple GPUs 20 in the same group are connected to different downlink ports of the same PCIe switch 30.

[0062] In some embodiments, such as Figure 1 As shown, the host CPU 10 and each GPU 20 can access the memory of other devices through the RDMA network (RDMANet) 40. RDMA (Remote Direct Memory Access) allows devices to directly access the memory of another device over a network without the intervention of the operating system or CPU. This improves data transfer efficiency and speed, reduces CPU load and network latency, and is particularly suitable for high-performance computing, big data processing, and real-time communication scenarios.

[0063] In some embodiments, GPU memory 21 may be a PCIe memory module (PCIe Memory Base Address Register), etc., and this disclosure does not limit it.

[0064] In this embodiment, in the access control method executed by the server, the host CPU 10 may pre-execute the following configuration steps: configure virtual addresses for the host memory 11 and the GPU memory 21 with access requirements (the GPU memory 21 with access requirements can be part or all of all GPU memory 21); use the Input / Output Memory Management Unit (IOMMU) to map the virtual addresses of the host memory 11 to the physical addresses of the host memory 11, and to map the virtual addresses of each GPU memory 21 with access requirements to the physical addresses of that GPU memory 21; create a total address mapping relationship and an address sub-mapping relationship for each GPU 20 based on the mapping relationship between the virtual addresses and physical addresses of the host memory 11 and the mapping relationship between the virtual addresses and physical addresses of the GPU memory 21; configure the PCIe outbound window of each GPU 20 based on the address sub-mapping relationship of each GPU.

[0065] The total address mapping relationship is used to indicate the mapping relationship between the virtual address and the physical address of the host memory 11, and the mapping relationship between the virtual address and the physical address of each GPU memory (i.e. the GPU memory corresponding to each GPU having access demand). The address sub-mapping relationship corresponding to each GPU is used to indicate the mapping relationship between the virtual address and the physical address of the host memory 11, and the mapping relationship between the virtual address and the physical address of the GPU memory of the out-group GPU in the group different from the GPU among the GPUs having access demand and the rest of the GPUs in the group different from the GPU.

[0066] In some embodiments, the range of the virtual address configured for the host memory and the GPU memory having access demand is within the limit address range of the PCIe outbound window of each GPU. For example, if the address range of the PCIe outbound window of each GPU is 0-512G, the range of the virtual address configured for the host memory 11 and each GPU memory 21 should be limited in combination with the address range that can be accessed by each GPU memory 21 itself, to ensure that the address recorded by the PCIe outbound window of each GPU 20 is within the range of 0-512G.

[0067] In this embodiment, the memory management unit IOMMU can determine the GPU memory that needs to be accessed by other GPUs according to the actual use requirement, and then perform the mapping between the virtual address and the physical address of the GPU memory that needs to be accessed. In this way, the address sub-mapping relationship can be set for each GPU within the address limit range of the memory management unit IOMMU, and the PCIe outbound window is configured, to ensure the normal memory access implementation of the entire server.

[0068] In this embodiment, the memory management unit IOMMU can determine the GPU memory that needs to be accessed by other GPUs according to the actual use requirement, and then perform the mapping between the virtual address and the physical address of the GPU memory that needs to be accessed. In this way, the address sub-mapping relationship can be set for each GPU within the address limit range of the memory management unit IOMMU, and the PCIe outbound window is configured, to ensure the normal memory access implementation of the entire server.

[0069] Configuration strategy one:

[0070] Take the server topology structure shown in FIG. 1 as an example, as shown in FIG. 2, the PCIe outbound window of each GPU is configured as follows. Figure 1 Figure 3 ​As shown, the PCIe outbound window corresponding to GPU0 is configured according to "Configuration Strategy 1". After configuration, the address recorded in the PCIe outbound window corresponding to GPU0 can include at least one of the following: the virtual address of host memory 11, the physical address of the GPU memory of the GPUs in the same group (GPU1, GPU2, and GPU3) that have access needs, and the virtual address of the GPU memory of the GPUs outside the group that have access needs from other GPUs in different groups than GPU0. Under "Configuration Strategy 1", the address sub-mapping relationship corresponding to GPU0 can indicate the mapping relationship between the virtual address and physical address of host memory 11, and the mapping relationship between the virtual address and physical address of the GPU memory of GPUs outside the group that are different from GPU0. Under "Configuration Strategy 1", each PCIe switch records the total address mapping relationship and / or the physical address of the GPU memory corresponding to the GPU connected to the downlink port. Similarly, other GPUs and PCIe switches are also configured accordingly.

[0071] Configuration Strategy Two:

[0072] by Figure 1 Taking the server topology shown as an example, the PCIe outbound window corresponding to GPU0 is configured according to "Configuration Strategy Two". After configuration, the address recorded in the PCIe outbound window corresponding to GPU0 can include at least one of the following: the virtual address of host memory 11, or the virtual address of the GPU memory corresponding to the GPU with access requirements among the remaining GPUs different from GPU0. Therefore, under "Configuration Strategy Two", the address sub-mapping relationship corresponding to GPU0 can indicate the mapping relationship between the virtual address and physical address of the host memory 11, and the mapping relationship between the virtual address and physical address of the GPU memory corresponding to the GPU with access requirements among the remaining GPUs different from GPU0. Therefore, under "Configuration Strategy Two", each PCIe switch records the total address mapping relationship and / or the physical address of the GPU memory corresponding to each GPU. Similarly, other GPUs and PCIe switches are also configured accordingly.

[0073] After the host CPU 10 completes the above "configuration steps", the memory access process implemented by the server using access control methods varies depending on the "configuration policy". These differences will be explained below.

[0074] The memory access process under "Configuration Strategy 1" is as follows:

[0075] The host CPU 10 and any GPU 20 in the server can act as requesters to generate access requests for a target storage space (a storage space within the target memory) in the target memory. The target memory can be any GPU memory 21 or host memory 11. The access request can indicate the target address corresponding to the target storage space.

[0076] If the requestor is the host CPU 10 and the target memory is a GPU memory 21 of a GPU, the host CPU 10 can directly determine the target physical address of the target storage space in the target memory according to the physical address of each GPU memory 21 recorded, and generate an access request with the target physical address as the target address. Then the access request is sent to the target PCIe switch connected to the GPU of the target memory.

[0077] If the requestor is a first GPU in the plurality of GPUs and the target memory is a GPU memory 21 of a GPU in the same group as the first GPU, the first GPU can directly determine the target physical address of the target storage space in the target memory according to the physical address of each GPU memory 21 (i.e., the GPU in the group) in the same group recorded in the PCIe outbound window, and generate an access request with the target physical address as the target address. Then the access request is sent to the PCIe switch connected to the first GPU (since the PCIe switch connected to the first GPU will directly send the access request to the target GPU where the target memory is located, so that the access request is executed by the target GPU, the PCIe switch connected to the first GPU in this case is the target PCIe switch).

[0078] If the requestor is a first GPU in the plurality of GPUs and the target memory is a GPU memory 21 of a GPU in a different group from the first GPU, the first GPU can directly determine the virtual address of the target storage space in the target memory according to the virtual address of the GPU memory 21 in the different group recorded in the PCIe outbound window, and generate an access request with the virtual address as the target address. Then the access request is sent to the PCIe switch connected to the first GPU.

[0079] If the requestor is a first GPU in the plurality of GPUs and the target memory is the host memory 11, the first GPU can directly generate an access request with the virtual address of the host memory 11 recorded in the PCIe outbound window as the target address. Then the access request is sent to the PCIe switch connected to the first GPU.

[0080] The host CPU 10 can directly access the host memory 11, and each GPU 20 can directly access its own GPU memory 21, which can be set according to the prior art, and the present disclosure does not limit this.

[0081] For any PCIe switch 30, if an access request sent by a GPU 20 connected via downstream port is received, it is determined whether the target address in the access request is a virtual address based on the total address mapping relationship and / or the physical address stored internally (the physical address stored by the PCIe switch 30 can include the physical address of the host memory 11 and the physical address of each GPU, or the physical address stored by the PCIe switch 30 can include the physical address of the host memory 11 and the physical address of the GPU with access demand connected via downstream port). If the target address is a virtual address, the access request is directly sent to the host CPU 20. If the target address is a physical address, the target physical address corresponding to the target address in the access request is determined, and the access request is sent to the target GPU corresponding to the target physical address. The target GPU is the GPU 20 capable of accessing the target memory indicated by the target address, and is a GPU in the group of the GPU sending the access request and connected to the PCIe switch 30. If an access request sent by the host CPU 10 connected via upstream port is received, the target physical address corresponding to the target address in the access request is determined based on the total address mapping relationship and / or the physical address stored internally, and the access request is sent to the target GPU corresponding to the target physical address. After receiving the access request, the target GPU performs data reading or writing based on the target physical address corresponding to the access request, thereby realizing memory access.

[0082] For the host CPU 10, if an access request forwarded by the PCIe switch 30 is received, the target physical address corresponding to the target address in the access request is determined based on the total address mapping relationship stored internally. Then, if it is determined that the target memory corresponding to the target physical address is a GPU memory 21, the target GPU corresponding to the target memory is further determined, and then the PCIe switch 30 connected to the target GPU is determined as the target PCIe switch, and the access request is sent to the target PCIe switch. The target GPU is the GPU 20 capable of accessing the target memory indicated by the target address. If it is determined that the target memory corresponding to the target physical address is the host memory 11, the host CPU 10 directly executes the access request, performs data reading or writing on the target physical address, and realizes memory access.

[0083] For example,

[0084] For example, Figure 4As shown, if the requester is host CPU 10 and the target memory is GPU memory 1, the access request control process is as follows: Figure 4 As shown by purple line 4, the host CPU 10 can directly generate an access request based on the physical address of GPU memory 1. The physical address of the target storage space in GPU memory 1 is both the target address and the target physical address. This access request is then sent to the PCIe switch 0 connected to GPU 1. After verification (see above), PCIe switch 0 will directly send the access request to the corresponding GPU 1 (i.e., the target GPU). Upon receiving the access request, GPU 1 reads or writes data based on the target physical address (i.e., the target address) in the access request, thus achieving memory access.

[0085] like Figure 4 As shown, if the requesting GPU is GPU0 and the target memory is GPU memory 1, the access request control process refers to... Figure 4 As shown by blue line 1, GPU0 can directly determine the target physical address of the target storage space in GPU memory 1 to be accessed based on the physical addresses of GPU memory 1 of the GPUs in each group recorded in the PCIe outbound window, and generate an access request using this target physical address as the target address. This access request is then sent to PCIe switch 0, which is connected to GPU1. After verification (see above), PCIe switch 0 will directly send the access request to the corresponding GPU1 (i.e., the target GPU). Upon receiving the access request, GPU1 reads or writes data based on the target physical address (i.e., the target address) in the access request, thus achieving memory access.

[0086] like Figure 4 As shown, if the requesting GPU is GPU0 and the target memory is GPU memory n, the access request control process refers to... Figure 4As shown by green line 3, GPU0 can directly determine the virtual address of the target storage space located in GPU memory n based on the virtual address of GPU memory n recorded in the PCIe outbound window, and generate an access request using this virtual address as the target address. This access request is then sent to PCIe switch 0 connected to GPU memory 0. After determination (see above), PCIe switch 0 will directly send the access request to host CPU 10. Upon receiving the access request, host CPU 10 first determines the target physical address corresponding to the target address in the access request based on the total address mapping relationship of its internal storage. Then, it further determines the target GPU "GPUn" corresponding to the target memory "GPU memory n" corresponding to the target physical address, and then sends the access request to the target PCIe switch "PCIe switch m" connected to GPUn. After determination (see above), the target PCIe switch "PCIe switch m" determines the target physical address corresponding to the target address, and then sends the access request to the target GPU "GPUn" corresponding to the target physical address. After receiving the access request, the target GPU "GPUn" reads or writes data based on the target physical address corresponding to the target address in the access request, thereby realizing memory access.

[0087] like Figure 4 As shown, if the requesting GPU is GPU0 and the target memory is host memory 11, the access request control process refers to... Figure 4 As shown by orange line 2, GPU0 can directly determine the virtual address of the target storage space in host memory 11 based on the virtual address of host memory 11 recorded in the PCIe outbound window, and generate an access request using this virtual address as the target address. This access request is then sent to PCIe switch 0 connected to GPU memory 0. After verification (see above), PCIe switch 0 directly sends the access request to host CPU 10. Upon receiving the access request, host CPU 10 first determines the target address in the access request based on the total address mapping relationship of its internal storage. If the target memory is host memory 11, it directly reads or writes data based on the target physical address corresponding to the target address, thus achieving memory access.

[0088] The memory access process under configuration strategy two is as follows:

[0089] The host CPU 10 and any GPU 20 in the server can act as requesters to generate access requests for a target storage space (a storage space within the target memory) in the target memory. The target memory can be any GPU memory 21 or host memory 11. The access request can indicate the target address corresponding to the target storage space.

[0090] If the requestor is the host CPU 10 and the target memory is a GPU memory 21, the host CPU 10 can directly determine the target physical address of the target storage space in the target memory according to the recorded physical addresses of the GPU memories 21, generate an access request with the target physical address as the target address, and then send the access request to the target PCIe switch connected to the GPU of the target memory.

[0091] If the requestor is a first GPU in the plurality of GPUs and the target memory is a GPU memory 21 or the host memory 11, the first GPU can directly determine the virtual address of the target storage space in the target memory according to the recorded virtual address in the PCIe outbound window, generate an access request with the virtual address as the target address, and then send the access request to the PCIe switch connected to the first GPU.

[0092] The host CPU 10 can directly access the host memory 11, and each GPU 20 can directly access its own GPU memory 21. The existing implementation mode can be referred to for setting, and the present disclosure does not limit this.

[0093] For any PCIe switch 30, if an access request sent by a GPU 20 connected to the downstream port is received, the access request is directly sent to the host CPU 20. If an access request sent by the host CPU 10 connected to the upstream port is received, the target physical address corresponding to the target address in the access request is first determined based on the stored total address mapping relationship and / or physical address (the physical address stored by the PCIe switch 30 can include the physical address of the host memory 11 and the physical address of each GPU, or the physical address stored by the PCIe switch 30 can include the physical address of the host memory 11 and the physical address of the GPU to be accessed by the GPU connected to the downstream port), and the access request is sent to the target GPU corresponding to the target physical address, which is the GPU 20 capable of accessing the target memory indicated by the target address. After receiving the access request, the target GPU performs data reading or writing based on the target physical address corresponding to the access request, thereby realizing memory access.

[0094] For the host CPU 10: if receiving the access request forwarded by the PCIe switch 30, first determine the target physical address corresponding to the target address in the access request based on the internally stored total address mapping relationship. Then if determining that the target memory corresponding to the target physical address is a certain GPU memory 21, further determine the target GPU corresponding to the target memory corresponding to the target physical address, and then send the access request to the PCIe switch 30 (i.e. the target PCIe switch) connected to the target GPU, which is the GPU 20 capable of accessing the target memory indicated by the target address. If determining that the target memory corresponding to the target physical address is the host memory 11, the host CPU 10 directly executes the access request, reads or writes data for the target physical address, and realizes memory access.

[0095] For example,

[0096] As Figure 5 shown, if the requestor is the host CPU 10 and the target memory is the GPU memory 1, the control process of the access request refers to the purple line 4 shown in Figure 5 . The host CPU 10 can directly generate an access request according to the physical address of the GPU memory 1, wherein the physical address of the target storage space in the GPU memory 1 is the target address and also the target physical address. Then send the access request to the PCIe switch 0 connected to the GPU 1. The PCIe switch 0 will directly send the access request to the corresponding GPU 1 (i.e. the target GPU) after determination (see above). After receiving the access request, the GPU 1 reads or writes data based on the target physical address (i.e. the target address) in the access request, and realizes memory access.

[0097] As Figure 5 shown, if the requestor is the first GPU, which is the GPU 0, and the target memory is the GPU memory 1, the control process of the access request refers to the purple line 4 shown in Figure 5GPU0 can determine the virtual address of the target storage space in GPU memory 1 to be accessed according to the virtual address recorded in the PCIe outbound window, and generate an access request with the virtual address as the target address. Then the access request is sent to the PCIe switch 0. The PCIe switch 0 directly sends the access request to the host CPU 10. After receiving the access request, the host CPU 10 determines the target physical address corresponding to the target address in the access request based on the internal stored total address mapping relationship. Then the host CPU 10 further determines the target GPU "GPU1" corresponding to the target memory "GPU memory 1" corresponding to the target physical address, and sends the access request to the PCIe switch 0 connected to the GPU1 (i.e. the target PCIe switch). The target PCIe switch "PCIe switch 0" determines the target physical address corresponding to the target address in the access request, and then sends the access request to the target GPU "GPU1" corresponding to the target physical address. After receiving the access request, the target GPU "GPU1" performs data reading or writing based on the target physical address corresponding to the target address in the access request, and realizes memory access.

[0098] As shown in Figure 5 If the requesting first GPU is GPU0 and the target memory is GPU memory n, the control process of the access request refers to the green line 3 shown in Figure 5 GPU0 can determine the virtual address of the target storage space in GPU memory n to be accessed according to the virtual address of GPU memory n recorded in the PCIe outbound window, and generate an access request with the virtual address as the target address. Then the access request is sent to the PCIe switch 0 connected to the GPU memory 0. The PCIe switch 0 directly sends the access request to the host CPU 10. After receiving the access request, the host CPU 10 determines the target physical address corresponding to the target address in the access request based on the internal stored total address mapping relationship. Then the host CPU 10 further determines the target GPU "GPUn" corresponding to the target memory "GPU memory n" corresponding to the target physical address, and sends the access request to the PCIe switch m (i.e. the target PCIe switch) connected to the GPUn. The PCIe switch m determines the target physical address corresponding to the target address after determination (detailed above), and then sends the access request to the GPUn (i.e. the target GPU) corresponding to the target physical address. After receiving the access request, the GPUn performs data reading or writing based on the target physical address corresponding to the target address in the access request, and realizes memory access.

[0099] As shown in Figure 5As shown, if the requesting GPU is GPU0 and the target memory is host memory 11, the access request control process refers to... Figure 5 As shown by orange line 2, GPU0 can directly determine the virtual address of the target storage space in host memory 11 based on the virtual address of host memory 11 recorded in the PCIe outbound window, and generate an access request using this virtual address as the target address. This access request is then sent to PCIe switch 0 connected to GPU memory 0. PCIe switch 0 directly sends the access request to host CPU 10. Upon receiving the access request, host CPU 10 first determines the target address in the access request based on the total address mapping relationship of its internal storage. Confirming that the target memory is host memory 11, it directly reads or writes data based on the target physical address corresponding to the target address, thus achieving memory access.

[0100] In this embodiment, during the process of the server executing the access control method to implement memory access, it controls the redirection function of the downlink port of each PCIe switch 30 to be turned off.

[0101] This disclosure also provides an access control method applied to a host CPU in a server. The server further includes multiple GPUs, host memory of the host CPU, GPU memory of each GPU, and multiple PCIe switches. Uplink ports of each PCIe switch are connected to the host CPU. The multiple GPUs are divided into multiple groups, and multiple GPUs within the same group are connected to different downlink ports of the same PCIe switch. The method includes:

[0102] Upon receiving an access request forwarded by the PCIe switch, the target physical address corresponding to the target address in the access request is determined, and the access request is sent to the corresponding target PCIe switch according to the target physical address, so that the target PCIe switch sends the access request to the target GPU where the target memory is located according to the target physical address, so that the access request is executed by the target GPU.

[0103] The access request is generated by the first GPU among the plurality of GPUs based on the target address and sent to the PCIe switch. If the PCIe switch determines that the target address is a virtual address, it is forwarded to the host CPU. The target address is determined by the first GPU based on the address recorded in the first GPU's PCIe outbound window and the target memory to be accessed. The target memory is any one of the other GPU memory that is different from the first GPU's GPU memory and the host memory.

[0104] The addresses recorded in the PCIe outbound windows of the GPUs include at least one of the following: a virtual address of the host memory, a virtual address of a GPU memory corresponding to a group-out GPU having an access demand in the remaining GPUs different from the GPU, and a physical address of a GPU memory corresponding to a group-in GPU having an access demand in the remaining GPUs in the same group as the GPU.

[0105] The disclosure also provides another access control method, which is applied to a host CPU in a server, the server further comprising a plurality of GPUs, a host memory of the host CPU, a GPU memory of each of the GPUs, and a plurality of PCIe switches, an uplink port of each of the PCIe switches being connected to the host CPU, the plurality of GPUs being divided into a plurality of groups, a plurality of GPUs in the same group being connected to different downlink ports of the same PCIe switch, the method comprising:

[0106] In a case where the access request forwarded by the PCIe switch is received, a target physical address corresponding to a target address in the access request is determined, and the access request is sent to a target PCIe switch corresponding to the target physical address, so that the target PCIe switch sends the access request to a target GPU where the target memory is located according to the target physical address, so that the access request is executed by the target GPU;

[0107] The access request is generated by a first GPU in the plurality of GPUs according to a target address and sent to the PCIe switch and forwarded to the host CPU by the PCIe switch, the target address is determined by the first GPU according to an address recorded in a PCIe outbound window of the first GPU and a target memory to be accessed, and the target memory is any one of the remaining GPU memories different from the GPU memory of the first GPU in the plurality of GPU memories and the host memory.

[0108] The addresses recorded in the PCIe outbound windows of the GPUs include at least one of the following: a virtual address of the host memory, a virtual address of a GPU memory corresponding to a group-out GPU having an access demand in the remaining GPUs different from the GPU.

[0109] The disclosure also provides an access control method, which is applied to a PCIe switch in a server, the server comprising a host CPU, a plurality of GPUs, a host memory of the host CPU, a GPU memory of each of the GPUs, and a plurality of PCIe switches, an uplink port of each of the PCIe switches being connected to the host CPU, the plurality of GPUs being divided into a plurality of groups, a plurality of GPUs in the same group being connected to different downlink ports of the same PCIe switch, the method comprising:

[0110] forward the access request to the host CPU, the access request being generated by the first GPU according to a target address, the target address being determined by the first GPU according to an address recorded in a PCIe outbound window of the first GPU and a target memory to be accessed, the target memory being any one of the rest of GPU memories in the plurality of GPU memories different from the GPU memory of the first GPU and the host memory; or

[0111] in a case where an access request from the host CPU is received, sending the access request to a target GPU where the target memory is located according to a target physical address corresponding to the access request, so that the access request is executed by the target GPU;

[0112] The address recorded in the PCIe outbound window of each GPU includes at least one of the following: a virtual address of the host memory, a virtual address of a GPU memory corresponding to a GPU in the rest of GPUs different from the GPU and having an access demand.

[0113] The disclosure also provides an access control method applied to a PCIe switch in a server, the server including a host CPU, a plurality of GPUs, a host memory of the host CPU, a GPU memory of each GPU, and a plurality of PCIe switches, an uplink port of each PCIe switch being connected to the host CPU, the plurality of GPUs being divided into a plurality of groups, a plurality of GPUs in the same group being connected to different downlink ports of the same PCIe switch, the method including:

[0114] forwarding the access request to the host CPU, the access request being generated by the first GPU according to a target address, the target address being determined by the first GPU according to an address recorded in a PCIe outbound window of the first GPU and a target memory to be accessed, the target memory being any one of the rest of GPU memories in the plurality of GPU memories different from the GPU memory of the first GPU and the host memory; or

[0115] in a case where an access request from the host CPU is received, sending the access request to a target GPU where the target memory is located according to a target physical address corresponding to the access request, so that the access request is executed by the target GPU;

[0116] The addresses recorded in the PCIe outbound window of each GPU include at least one of the following: a virtual address of the host memory, a virtual address of a GPU memory corresponding to a group-out GPU having an access demand among the remaining GPUs different from the group of the GPU, and a physical address of a GPU memory corresponding to a group-in GPU having an access demand among the remaining GPUs in the same group as the GPU.

[0117] The disclosure also provides an access control method applied to a GPU in a server. The server includes a host CPU, a plurality of GPUs, a host memory of the host CPU, a GPU memory of each of the GPUs, and a plurality of PCIe switches. An uplink port of each of the PCIe switches is connected to the host CPU. The plurality of GPUs are divided into a plurality of groups. The plurality of GPUs in the same group are connected to different downlink ports of the same PCIe switch. The method includes:

[0118] A first GPU among the plurality of GPUs determines a target address of a target memory that needs to be accessed according to an address recorded in a PCIe outbound window corresponding to the first GPU, and sends an access request generated according to the target address to a connected PCIe switch, so that the access request is forwarded by the PCIe switch to the host CPU. The host CPU determines a target physical address corresponding to the target address in the access request, and sends the access request to a corresponding target PCIe switch according to the target physical address, so that the target PCIe switch sends the access request to a target GPU where the target memory is located according to the target physical address.

[0119] The target GPU among the plurality of GPUs executes the access request when the target GPU receives the access request.

[0120] The target memory is any one of the remaining GPU memories different from the GPU memory of the first GPU among the plurality of GPU memories and the host memory.

[0121] The addresses recorded in the PCIe outbound window of each GPU include at least one of the following: a virtual address of the host memory, a virtual address of a GPU memory corresponding to a group-out GPU having an access demand among the remaining GPUs different from the group of the GPU, and a physical address of a GPU memory corresponding to a group-in GPU having an access demand among the remaining GPUs in the same group as the GPU. Alternatively, the addresses recorded in the PCIe outbound window of each GPU include at least one of the following: a virtual address of the host memory, a virtual address of a GPU memory corresponding to a GPU having an access demand among the remaining GPUs different from the GPU.

[0122] It should be noted that although the above-mentioned embodiments are introduced as examples for the access control method and the server, those skilled in the art can understand that the present disclosure should not be limited thereto. In fact, the user can flexibly set the steps and the method steps performed by each hardware according to personal preferences and / or actual application scenarios, as long as the technical solutions of the present disclosure are met.

[0123] The present disclosure also provides a computer readable storage medium having stored thereon computer executable instructions that, when executed by a processor, enable performance of the above method.

[0124] The present disclosure also provides an electronic device, comprising: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to implement the above method when executing the instructions stored in the memory.

[0125] The present disclosure also provides a computer program product, comprising computer readable code or a non-volatile computer readable storage medium carrying computer readable code, when the computer readable code is run in the processor of an electronic device, the processor in the electronic device performs the above method.

[0126] The present disclosure can be a system, a method, and / or a computer program product. The computer program product can include a computer readable storage medium (or media) having computer readable program instructions thereon for causing a processor to carry out aspects of the present disclosure.

[0127] The computer readable storage medium can be a tangible device that can retain and store instructions for use by an instruction execution device. The computer readable storage medium can be, for example, but is not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of the computer readable storage medium include the following: a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), a static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanically encoded device such as punch-cards or punched tape, and any suitable combination of the foregoing. A computer readable storage medium, as used herein, is not to be construed as being transitory signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide or other transmission media (e.g., light pulses passing through a fiber-optic cable), or electrical signals transmitted through a wire.

[0128] Computer readable program instructions described herein can be downloaded to respective computing / processing devices from a computer readable storage medium or to an external computer or external storage device via a network, for example, the Internet, a local area network, a wide area network and / or a wireless network. The network can comprise copper transmission cables, optical transmission fibers, wireless transmission, routers, firewalls, switches, gateway computers and / or edge servers. A network adapter card or network interface in each computing / processing device receives computer readable program instructions from the network and forwards the computer readable program instructions for storage in a computer readable storage medium within the respective computing / processing device.

[0129] Computer readable program instructions for carrying out operations of the present disclosure can be assembler instructions, instruction-set-architecture (ISA) instructions, machine instructions, machine dependent instructions, microcode, firmware instructions, state-setting data, or either source code or object code written in any combination of one or more programming languages, including an object oriented programming language such as Smalltalk, C++ or the like, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The computer readable program instructions can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider). In some embodiments, electronic circuitry including, for example, programmable logic circuitry, field-programmable gate arrays (FPGA), or programmable logic arrays (PLA) can execute the computer readable program instructions by utilizing state information of the computer readable program instructions to personalize the electronic circuitry, in order to perform aspects of the present disclosure.

[0130] The computer readable program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other device to produce a computer implemented process, such that the instructions which execute on the computer, other programmable data processing apparatus, or other device implement the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0131] These computer readable program instructions can be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks. These computer readable program instructions can also be stored in a computer readable storage medium that can include a non-transitory computer readable storage medium that can be a computer- readable storage medium having no data storage cycles that change state. The instructions can be executed by one or more processors of a computer, to cause a series of operational steps to be performed on the computer to produce a computer-implemented process. The instructions can also cause one or more processors of a computer or other programmable data processing apparatus to

[0132] The computer readable program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer implemented process, such that the instructions which execute on the computer, other programmable data processing apparatus, or other device implement the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0133] The computer readable program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer implemented process, such that the instructions which execute on the computer, other programmable data processing apparatus, or other device implement the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0134] Embodiments of the present disclosure have been described above, and the description is intended to be illustrative of the embodiments and not restrictive of the disclosure. Many modifications and variations of the described embodiments are possible in light of this disclosure without departing from the scope and spirit of the described embodiments. The choice of words in this document is intended to best explain the principles of the embodiments, the practical application, or technical improvement over prior art, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. A server, characterized by The system comprises a host CPU, a plurality of GPUs, a host memory of the host CPU, GPU memories of the GPUs, and a plurality of PCIe switches, uplink ports of the PCIe switches being connected to the host CPU, the plurality of GPUs being divided into a plurality of groups, and the plurality of GPUs in the same group being connected to different downlink ports of the same PCIe switch, a first GPU in the plurality of GPUs is configured to determine a target address corresponding to a target memory to be accessed according to an address recorded in a PCIe outbound window of the first GPU, and send an access request generated according to the target address to a PCIe switch connected to the first GPU, the target memory being any one of the remaining GPU memories different from the GPU memory of the first GPU in the plurality of GPU memories and the host memory; the PCIe switch is configured to, in a case where an access request is received, forward the access request with a virtual address as the target address to the host CPU, and forward the access request with a physical address as the target address to a target GPU where the target memory is located and connected to a downlink port of the PCIe switch; wherein the address recorded in the PCIe outbound window of each GPU comprises at least one of the following: a virtual address of the host memory, a virtual address of a GPU memory corresponding to a group-out GPU having an access demand in the remaining GPUs different from the GPU in the same group, and a physical address of a GPU memory corresponding to a group-in GPU having an access demand in the remaining GPUs in the same group as the GPU; the PCIe switch is configured to, in a case where an access request is received, forward the access request with a virtual address as the target address to the host CPU, and forward the access request with a physical address as the target address to a target GPU where the target memory is located and connected to a downlink port of the PCIe switch; 2. A server, characterized by The system comprises a host CPU, a plurality of GPUs, a host memory of the host CPU, GPU memories of the GPUs, and a plurality of PCIe switches, uplink ports of the PCIe switches being connected to the host CPU, the plurality of GPUs being divided into a plurality of groups, and the plurality of GPUs in the same group being connected to different downlink ports of the same PCIe switch, a first GPU in the plurality of GPUs is configured to determine a target address corresponding to a target memory to be accessed according to an address recorded in a PCIe outbound window of the first GPU, and send an access request generated according to the target address to a PCIe switch connected to the first GPU, the target memory being any one of the remaining GPU memories different from the GPU memory of the first GPU in the plurality of GPU memories and the host memory; the PCIe switch is configured to, in a case where an access request is received, forward the access request to the host CPU; The address recorded in the PCIe outbound window of each GPU includes at least one of the following: a virtual address of the host memory, and a virtual address of a GPU memory corresponding to a GPU having an access demand among the remaining GPUs different from the GPU. The total address mapping relationship and / or the physical address of the GPU memory corresponding to each GPU are recorded in each PCIe switch, and the total address mapping relationship is used to indicate a mapping relationship between the virtual address and the physical address of the host memory and a mapping relationship between the virtual address and the physical address of each GPU memory having an access demand.

3. The server of claim 1 or 2, wherein The host CPU is configured to, in response to receiving an access request, determine a target physical address corresponding to a target address in the access request, and send the access request to a target PCIe switch corresponding to the target physical address. The target PCIe switch is configured to send the access request to a target GPU where the target memory is located according to the target physical address, so that the access request is executed by the target GPU.

4. The server of claim 3, wherein, The target physical address corresponding to the target address in the access request is determined by: determining the target physical address corresponding to the target address in the access request according to a total address mapping relationship or a physical address of each GPU; or determining the target physical address corresponding to the target address in the access request according to an address sub-mapping relationship corresponding to the first GPU. The address sub-mapping relationship corresponding to the first GPU is used to indicate a mapping relationship between the virtual address and the physical address of the host memory and a mapping relationship between the virtual address and the physical address of the GPU memory corresponding to each out-of-group GPU.

5. The server of claim 4, wherein, The host CPU is configured to: configure virtual addresses for the host memory and the GPU memory having an access demand, the GPU memory having an access demand being part or all of the plurality of GPU memories in the server; map the virtual address of the host memory to the physical address of the host memory and map the virtual address of each GPU memory having an access demand to the physical address of the GPU memory by using a memory management unit; create a total address mapping relationship and an address sub-mapping relationship of each GPU according to a mapping relationship between the virtual address and the physical address of the host memory and a mapping relationship between the virtual address and the physical address of the GPU memory; configure the PCIe outbound window of each GPU based on the address sub-mapping relationship of each GPU.

6. The server of claim 1 or 2, wherein, Each PCIe switch is controlled to close the redirection function of the downlink port.

7. The server of claim 5, wherein, The range of the virtual addresses configured for the host memory and the GPU memory having an access demand is within the limit address range of the PCIe outbound window of each GPU.

8. An access control method characterized by, The application is applied to a host CPU in a server, the server further comprises a plurality of GPUs, a host memory of the host CPU, a GPU memory of each of the GPUs, and a plurality of PCIe switches, an uplink port of each of the PCIe switches is connected to the host CPU, the plurality of GPUs are divided into a plurality of groups, a plurality of GPUs in a same group are connected to different downlink ports of a same PCIe switch, and the method comprises the following steps: In a case where an access request forwarded by the PCIe switch is received, a target physical address corresponding to a target address in the access request is determined, and the access request is sent to a corresponding target PCIe switch according to the target physical address, so that the target PCIe switch sends the access request to a target GPU where a target memory is located according to the target physical address, and the access request is executed by the target GPU; The access request is generated by a first GPU in the plurality of GPUs according to a target address and sent to the PCIe switch, and in a case where the PCIe switch determines that the target address is a virtual address, the access request is forwarded to the host CPU, the target address is determined by the first GPU according to an address recorded in a PCIe outbound window of the first GPU and a target memory to be accessed, and the target memory is any one of the remaining GPU memories of the plurality of GPUs different from the GPU memory of the first GPU and the host memory; The address recorded in the PCIe outbound window of each of the GPUs comprises at least one of the following: a virtual address of the host memory, a virtual address of a GPU memory corresponding to a group-out GPU in the remaining GPUs different from the group of the GPU and having an access demand, and a physical address of a GPU memory corresponding to a group-in GPU in the remaining GPUs in the same group as the GPU and having an access demand; The total address mapping relationship and / or the physical address of the GPU memory corresponding to the GPU connected to the downlink port are recorded in each of the PCIe switches, and the total address mapping relationship is used to indicate a mapping relationship between the virtual address and the physical address of the host memory and a mapping relationship between the virtual address and the physical address of each GPU memory having an access demand.

9. An access control method characterized by, The application is applied to a host CPU in a server, the server further comprises a plurality of GPUs, a host memory of the host CPU, a GPU memory of each of the GPUs, and a plurality of PCIe switches, an uplink port of each of the PCIe switches is connected to the host CPU, the plurality of GPUs are divided into a plurality of groups, a plurality of GPUs in a same group are connected to different downlink ports of a same PCIe switch, and the method comprises the following steps: In a case where an access request forwarded by the PCIe switch is received, a target physical address corresponding to a target address in the access request is determined, and the access request is sent to a corresponding target PCIe switch according to the target physical address, so that the target PCIe switch sends the access request to a target GPU where a target memory is located according to the target physical address, and the access request is executed by the target GPU; The access request is generated by a first GPU in the plurality of GPUs according to a target address, and is sent to a PCIe switch and forwarded to the host CPU by the PCIe switch. The address recorded in the PCIe outbound window of each GPU includes at least one of the following: a virtual address of the host memory, a virtual address of a GPU memory corresponding to a GPU with access demand in the remaining GPUs different from the GPU. The PCIe switch records a total address mapping relationship and / or a physical address of the GPU memory corresponding to each GPU, and the total address mapping relationship is used to indicate a mapping relationship between a virtual address and a physical address of the host memory, and a mapping relationship between a virtual address and a physical address of each GPU memory with access demand.

10. An access control method characterized by, The PCIe switch is applied to a server, the server includes a host CPU, a plurality of GPUs, a host memory of the host CPU, a GPU memory of each GPU, and a plurality of PCIe switches, an uplink port of each PCIe switch is connected to the host CPU, the plurality of GPUs are divided into a plurality of groups, a plurality of GPUs in the same group are connected to different downlink ports of the same PCIe switch, and the method comprises: In a case where an access request sent by a first GPU in the plurality of GPUs is received, the access request is forwarded to the host CPU; the access request is generated by the first GPU according to a target address, and the target address is determined by the first GPU according to an address recorded in a PCIe outbound window of the first GPU and a target memory with access demand; and the target memory is any one of the remaining GPU memories different from the GPU memory of the first GPU and the host memory. In a case where an access request from the host CPU is received, the access request is sent to a target GPU where the target memory is located according to a target physical address corresponding to the access request, so that the access request is executed by the target GPU. The address recorded in the PCIe outbound window of each GPU includes at least one of the following: a virtual address of the host memory, a virtual address of a GPU memory corresponding to a GPU with access demand in the remaining GPUs different from the GPU. The PCIe switch records a total address mapping relationship and / or a physical address of the GPU memory corresponding to each GPU, and the total address mapping relationship is used to indicate a mapping relationship between a virtual address and a physical address of the host memory, and a mapping relationship between a virtual address and a physical address of each GPU memory with access demand.

11. An access control method characterized by, The application relates to a PCIe switch applied to a server, wherein the server comprises a host CPU, a plurality of GPUs, a host memory of the host CPU, GPU memories of the GPUs, and a plurality of PCIe switches; an uplink port of each PCIe switch is connected to the host CPU; the plurality of GPUs are divided into a plurality of groups; a plurality of GPUs in the same group are connected to different downlink ports of the same PCIe switch; and the method comprises the following steps: in a case that an access request sent by a first GPU in the plurality of GPUs is received, forwarding an access request with a target address being a virtual address to the host CPU, and forwarding an access request with the target address being a physical address to a target GPU where target memory connected to a downlink port of the PCIe switch is located; the access request is generated by the first GPU according to a target address; the target address is determined by the first GPU according to an address recorded in a PCIe outbound window of the first GPU and target memory to be accessed; and the target memory is any one of the rest GPU memories different from the GPU memory of the first GPU and the host memory in the plurality of GPU memories; or, in a case that an access request from the host CPU is received, sending the access request to a target GPU where target memory is located according to a target physical address corresponding to the access request, so that the access request is executed by the target GPU; wherein the address recorded in the PCIe outbound window of each GPU comprises at least one of the following: a virtual address of the host memory, a virtual address of GPU memory corresponding to a group-out GPU in the rest GPUs different from the group of the GPU and having an access demand, and a physical address of GPU memory corresponding to a group-in GPU in the rest GPUs in the same group as the GPU and having an access demand; a total address mapping relationship and / or a physical address of GPU memory corresponding to a GPU connected to a downlink port are recorded in each PCIe switch, and the total address mapping relationship is used for indicating a mapping relationship between a virtual address and a physical address of the host memory and a mapping relationship between virtual addresses and physical addresses of the GPU memories having an access demand.

12. An access control method characterized by, The application relates to a GPU applied to a server, wherein the server comprises a host CPU, a plurality of GPUs, a host memory of the host CPU, GPU memories of the GPUs, and a plurality of PCIe switches; an uplink port of each PCIe switch is connected to the host CPU; the plurality of GPUs are divided into a plurality of groups; a plurality of GPUs in the same group are connected to different downlink ports of the same PCIe switch; and the method comprises the following steps: The first GPU in the plurality of GPUs determines a target address of a target memory to be accessed according to an address recorded in a PCIe outbound window corresponding to the first GPU, and sends an access request generated according to the target address to a connected PCIe switch, so that the access request is forwarded by the PCIe switch to the host CPU, so that the host CPU determines a target physical address corresponding to the target address in the access request, and sends the access request to a corresponding target PCIe switch according to the target physical address, so that the target PCIe switch sends the access request to a target GPU where the target memory is located according to the target physical address; The target GPU in the plurality of GPUs executes the access request when receiving the access request; The target memory is any one of the remaining GPU memories different from the GPU memory of the first GPU and the host memory in the plurality of GPU memories; The addresses recorded in the PCIe outbound windows of the GPUs include at least one of the following: a virtual address of the host memory, a virtual address of a GPU memory corresponding to a group-out GPU having an access demand in the remaining GPUs different from the group of the GPU, and a physical address of a GPU memory corresponding to a group-in GPU having an access demand in the remaining GPUs in the same group as the GPU, and the PCIe switches record a total address mapping relationship and / or a physical address of a GPU memory corresponding to a GPU connected to a downlink port; or the addresses recorded in the PCIe outbound windows of the GPUs include at least one of the following: a virtual address of the host memory, a virtual address of a GPU memory corresponding to a GPU having an access demand in the remaining GPUs different from the GPU, and the PCIe switches record a total address mapping relationship and / or a physical address of a GPU memory corresponding to each of the GPUs. The total address mapping relationship is used to indicate a mapping relationship between a virtual address and a physical address of the host memory, and a mapping relationship between a virtual address and a physical address of each GPU memory having an access demand.

13. An access control device, characterized in that Comprise: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to implement the method of any one of claims 8 to 12 when executing the instructions stored in the memory.

14. A non-transitory computer readable storage medium having stored thereon computer program instructions, wherein, The computer program instructions are executed by the processor to implement the method of any one of claims 8 to 12. The computer program instructions are executed by the processor to implement the method of any one of claims 8 to 12.

Images