Method for direct remote memory access for graphics processors, computing device, computer-readable storage medium and computer program product

Abstract

The present application relates to a method for direct remote memory access of a graphics processor, a computing device, a computer readable storage medium and a computer program product. The method comprises: obtaining page granularity of source device memory and target device memory to be communicated respectively; in response to determining that at least one of the source device memory and the target device memory is not aligned to a predetermined size page granularity of PCIE, constructing a virtual address space mapped to a physical address space of the device unaligned memory at the predetermined size page granularity, so that the mapped virtual address space meets the requirements of a PCIE page table; and communicating between the memories of the source device and the target device in a direct remote memory access manner based on the constructed virtual address space. The present application can avoid configuring additional memory page aligned physical memory as an intermediate cache, and can simplify memory call operations, and significantly reduce transmission delay.

Description

TECHNICAL FIELD

[0001] Embodiments of the present application generally relate to the field of artificial intelligence, and more particularly to a method for GPU Direct RDMA, a computing device, a computer readable storage medium, and a computer program product. BACKGROUND

[0002] The implementation of GPU Direct RDMA (GDR) communication technology depends on the correct mapping of GPU memory to PCIE address space, which requires the granularity of GPU memory pages to be aligned with the PCIE page granularity (e.g., 4KB alignment). However, the memory used by some manufacturers' GPUs in the calculation process is not 4KB aligned. This mismatch in page granularity will result in the inability to directly map GPU memory to PCIE address space, and thus the inability to communicate between devices through GDR technology.

[0003] In the traditional method for GPU Direct RDMA, when facing non-4KB granularity devices, multiple dedicated 4KB architecture memories are needed to be allocated as intermediate caches, which can be directly mapped to the address space of the PCIE BAR (PCI Express Base Address Register, PCIE BAR), and then copied to the 4KB intermediate cache through the memcpy (Memory Copy) operation, and GDR transmission is performed through the 4KB intermediate cache. In addition, each time a segment of data is transmitted, a memory zero operation needs to be performed on the intermediate cache. Therefore, the traditional method for GPU Direct RDMA requires additional intermediate cache configuration, as well as memory copy and memory zero operations for the intermediate cache when communicating with non-4KB granularity devices. It should be understood that for low-latency, high-bandwidth large model scenarios, these additional memory operations will affect the overall performance of the model.

[0004] In summary, the traditional method for GPU Direct RDMA has the following disadvantages: for memory page non-aligned devices, additional memory page aligned physical memory needs to be configured as an intermediate cache, and the memory call operation is complex and has high latency. SUMMARY

[0005] The present application provides a method for GPU Direct RDMA, a computing device, a computer readable storage medium, and a computer program product, which do not require additional memory page aligned physical memory as an intermediate cache, and can simplify the memory call operation, significantly reducing the transmission delay.

[0006] According to a first aspect of the present application, there is provided a method for direct remote memory access of a graphics processor, the method comprising: obtaining page granularity of source device memory and target device memory to be communicated, respectively, the source device and the target device comprising the graphics processor, respectively; in response to determining that at least one of the source device memory and the target device memory is aligned to a predetermined size page granularity of PCIE, constructing a virtual address space for communication mapped to a physical address space of device unaligned memory at the predetermined size page granularity, such that the mapped virtual address space meets requirements of a PCIE page table; and communicating between the memories of the source device and the target device in a manner of direct remote memory access based on the constructed virtual address space for communication.

[0007] According to a second aspect of the present application, there is also provided a computing device. The computing device comprises: at least one processor; and a memory communicatively connected with the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the computing device to perform the method of the first aspect of the present application.

[0008] According to a third aspect of the present application, there is also provided a computer readable storage medium. The computer readable storage medium has stored thereon a computer program which, when executed by a machine, performs the method of the first aspect of the present application.

[0009] According to a fourth aspect of the present application, there is also provided a computer program product comprising a computer program which, when executed by a machine, performs the method of the first aspect of the present application.

[0010] In some embodiments, constructing a virtual address space mapped to a physical address space of device unaligned memory at a predetermined size page granularity comprises: mapping, for the physical address space of device unaligned memory, the predetermined size data in each channel of the physical address space as a whole in a channel order to the constructed virtual address space, so that the constructed virtual address space for communication is aligned to the predetermined size page granularity of PCIE.

[0011] In some embodiments, the method for direct remote memory access of a graphics processor further comprises: mapping the constructed virtual address space for communication to a base address register of PCIE, so that an external device connected via the PCIE is able to directly access corresponding physical memory through the constructed virtual address space for communication.

[0012] In some embodiments, the method for direct remote memory access for a graphics processor further comprises causing both the original virtual address space and the constructed virtual address space for communication pointing to the same physical address space to be released, and releasing the same physical address space.

[0013] In some embodiments, causing both the original virtual address space and the constructed virtual address space for communication pointing to the same physical address space to be released, and releasing the same physical address space comprises: in response to determining that the constructed virtual address space for communication has been constructed, incrementing a reference count of a physical address space mapped to the virtual address space by 1; in response to either the original virtual address space or the constructed virtual address space for communication pointing to the same physical address space being released, decrementing the reference count of the physical address space by 1; and in response to the reference count of the same physical address space being 0, releasing the same physical address space.

[0014] In some embodiments, communicating between the source device and the target device in a direct remote memory access manner based on the constructed virtual address space for communication comprises: for the original page granularity unaligned memory in the source device and the target device, performing memory remapping based on the constructed virtual address space for communication, so that both the source device and the target device have a virtual address space aligned with a predetermined size page granularity of PCIE; and directly copying, in a direct remote memory access manner, physical memory data of the source device to physical memory of the target device through the predetermined size page granularity aligned virtual address space of the source device and the target device.

[0015] In some embodiments, the memory page granularity of the source device and the memory page granularity of the target device are the same or different.

[0016] In some embodiments, if the source device contains a memory pool, and the unaligned PCIE predetermined size page granularity memory of the source device is in the memory pool, the method further comprises: performing memory remapping on the entire memory pool to obtain a memory page granularity aligned PCIE predetermined size page granularity virtual address space for communication corresponding to the memory pool; in response to receiving a data transmission request, determining a pointer offset of the to-be-transmitted data in the memory page granularity aligned virtual address space based on a pointer offset of the to-be-transmitted data in the original virtual address space of the memory pool related to the data transmission request, so as to determine the location of the to-be-transmitted data in the memory page granularity aligned virtual address space; and directly transmitting, in a direct remote memory access manner, data in the physical memory of the source device to the physical memory of the target device through the location of the to-be-transmitted data in the memory page granularity aligned virtual address space for communication.

[0017] The application respectively acquires page granularity of source device memory and target device memory to be communicated, the source device and the target device respectively comprising a graphic processor; in response to determining that at least one of the source device memory and the target device memory is not aligned with a predetermined size page granularity of PCIE, constructing a virtual address space mapped with a physical address space of the device unaligned memory at the predetermined size page granularity of the virtual address space based on the physical address space of the device unaligned memory, so that the mapped virtual address space meets the requirement of PCIE page table, and communicating between the source device and the target device in a direct remote memory access manner based on the constructed virtual address space. Therefore, the application can realize direct remote memory access of the graphic processor without using additional aligned physical memory as an intermediate cache when the memory page granularity of the graphic processor device is not aligned with the predetermined size page granularity of PCIE, and can simplify memory calling operation and significantly reduce transmission delay.

[0018] It should be understood that the matters described in this section are not intended to identify key or essential features of embodiments of the application, nor are they used to limit the scope of the application. Other features of the application will be readily apparent from the following description. BRIEF DESCRIPTION OF DRAWINGS

[0019] The above and other features, advantages and aspects of embodiments of the application will become more apparent upon reading the following detailed description in conjunction with the accompanying drawings, in which like references refer to like elements. In the drawings:

[0020] Figure 1 A schematic diagram of a computing device implementing a method for direct remote memory access for a graphic processor according to embodiments of the application is schematically shown.

[0021] Figure 2 A flowchart of a method for direct remote memory access for a graphic processor according to embodiments of the application is shown.

[0022] Figure 3 A schematic diagram of constructing a virtual address space mapped with a physical address at a predetermined size page granularity according to some embodiments of the application is shown.

[0023] Figure 4 A schematic diagram of mapping the constructed virtual address space for communication to PCIE according to some embodiments of the application is shown.

[0024] Figure 5 A schematic diagram of a communication process between a source device and a target device according to some embodiments of the application is shown.

[0025] Figure 6 A flowchart of a method for releasing a physical address according to embodiments of the application is shown.

[0026] Figure 7 A flow chart of a method for transferring data in a physical memory of a source device directly to a physical memory of a target device is shown, according to an embodiment of the present application.

[0027] Figure 8 A schematic diagram of memory pool mapping is shown, according to some embodiments of the present application.

[0028] In the various drawings, like or corresponding elements are denoted by like or corresponding reference numerals. DETAILED DESCRIPTION

[0029] Preferred embodiments of the present application will be described herein below with reference to the accompanying drawings. While preferred embodiments of the present application are shown in the drawings, it is understood that the present application can be embodied in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and fully convey the scope of the present application to those skilled in the art.

[0030] The term "comprising" and variations thereof as used herein are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements is not necessarily limited to those elements, but can include other elements not expressly listed or inherent to such process, method, article, or apparatus. Unless otherwise stated, the term "or" as used herein is to be interpreted as "and / or," i.e., the mathematical, that includes at least one of the items listed. The term "based on" is to be interpreted as "based, at least in part, on." The term "one example embodiment" and "an example embodiment" mean "at least one example embodiment." The term "another embodiment" means "at least one additional embodiment." The terms "a first," "a second," etc. do not require a strict counting but can be understood as specifying a certain instance of an object regardless of its count.

[0031] As described previously, the conventional method for direct remote memory access for a graphics processor has the disadvantage that, for a device that is not aligned with a memory page, an additional configuration of a memory page-aligned physical memory as an intermediate cache is required, and the memory call operation is complex and has high latency.

[0032] To at least partially address one or more of the aforementioned problems and other potential issues, an exemplary embodiment of the present invention provides a method for direct remote memory access for a graphics processor. In this method, page granularity of source device memory and target device memory to be communicated are obtained, each including a graphics processor; in response to determining at least one unaligned PCIe page granularity of the source device memory and target device memory of a predetermined size, a virtual address space mapped to the physical address space of the unaligned memory is constructed at the predetermined page granularity, such that the mapped virtual address space satisfies the requirements of the PCIe page table; and communication is performed between the memory of the source device and the target device via direct remote memory access based on the constructed virtual address space. Therefore, the present invention enables direct remote memory access for a graphics processor when the memory pages of the graphics processor device are not aligned to a predetermined PCIe page granularity, without requiring additional page-aligned physical memory as an intermediate cache, and simplifies memory access operations, significantly reducing transmission latency.

[0033] The following combination Figure 1 A computing device 100 illustrating an embodiment of the present invention, according to an embodiment of the present invention, performs a method for direct remote memory access for a graphics processor. Figure 1 As shown, the computing device 100 may have one or more processing units, including dedicated processing units such as graphics processing units (GPUs), field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), general-purpose computing on graphics processing units (GPGPUs), and general-purpose processing units such as CPUs. The computing device 100 also includes at least: a memory page granularity fetching unit 102, a virtual address space construction unit 104, and a direct remote memory access unit 106.

[0034] Regarding the memory page granularity acquisition unit 102, it is used to acquire the page granularity of the source device memory and the target device memory to be communicated, respectively. The source device and the target device each include a graphics processor.

[0035] Regarding the virtual address space construction unit 104, in response to determining at least one unaligned PCIe page granularity of a predetermined size in the source device memory and the target device memory, it constructs a virtual address space for communication mapped to the physical address space at a predetermined size page granularity based on the physical address space of the unaligned memory of the device, such that the mapped virtual address space satisfies the requirements of the PCIe page table.

[0036] Regarding the direct remote memory access unit 106, it is used to communicate between the memory of the source device and the target device in a direct remote memory access manner based on the constructed virtual address space.

[0037] The following will combine Figure 2 and Figure 4 A method 200 for direct remote memory access for a graphics processor, according to an embodiment of the present invention, is described. It should be understood that method 200 can, for example, be implemented in... Figure 1 The described computing device 100 performs the operation. Method 200 may also include additional actions not shown and / or the actions shown may be omitted; the scope of the invention is not limited in this respect.

[0038] In step 202, the computing device 100 obtains the memory page granularity of the source device memory and the target device memory to be communicated, respectively. The source device and the target device each include a graphics processor.

[0039] At step 204, if the computing device 100 determines at least one unaligned PCIe page granularity of a predetermined size in the source device memory and the target device memory, it constructs a virtual address space for communication mapped to the physical address space at the predetermined size page granularity based on the physical address space of the unaligned memory of the device, such that the mapped virtual address space satisfies the requirements of the PCIe page table.

[0040] Regarding the Peripheral Component Interconnect Express (PCIE) standard, the standard PCIE page granularity is 4 KB. However, in some embodiments, PCIE may also support other predetermined page granularities.

[0041] Graphics Processor Direct Remote Memory Access (GDR) technology maps GPU memory to the PCIe Base Address Register (BAR) address space, allowing other PCIe devices to directly read and write GPU memory from the source or target device. This avoids the performance overhead of CPU copying in traditional data transfer paths. For example, when the GPU driver maps the video memory window to the host address space, the base address and length of the entire BAR window must be aligned with the predetermined page granularity of the PCIe (e.g., 4KB alignment). Similarly, when the operating system allocates host physical address space to the BAR, it must also align with the predetermined page granularity of the PCIe. Therefore, the implementation of GDR technology relies on the correct mapping of GPU memory to the PCIe address space, requiring the GPU's memory page granularity to be aligned with the PCIe page granularity.

[0042] It should be understood that the direct remote memory access method for graphics processors provided by this invention can be applied to direct remote memory access between devices in scenarios where the memory page granularity of the graphics processor device is not aligned with the predetermined size page granularity of PCIe. Therefore, at least one of the source and target devices has memory pages that are not aligned with the predetermined size page granularity of PCIe. For example, the predetermined size page granularity of PCIe is 4KB, and at least one of the source and target device memory has a page granularity that is not 4KB.

[0043] Regarding the source and target devices, the memory page granularity of the source device and the target device may be the same or different. For example, the memory page granularity of the source device is 2KB, and the memory page granularity of the target device is 4KB; for example, the memory page granularity of the source device is 512B, and the memory page granularity of the target device is 1KB; for example, the memory page granularity of the source device is 2KB, and the memory page granularity of the target device is 2KB. Therefore, this solution can be applied to source and target devices with different or the same memory page granularity, and has good compatibility.

[0044] A method for constructing a virtual address space mapped to the physical address space at a predetermined page granularity includes, for example, a computing device 100 mapping data of a predetermined size in each channel of the physical address space as a whole to the constructed virtual address space in channel order for a physical address space of unaligned memory, so that the constructed virtual address space for communication is aligned with the predetermined page granularity of PCIe.

[0045] Please refer to Figure 3 This illustrates the structure of the GPU memory in the graphics processing unit 300. For example... Figure 3As shown, the physical memory 330 of the graphics processor device 300 has a memory page granularity of 512B (grau=512B, glu refers to the smallest unit of atomic operation performed by hardware). The memory of the graphics processor device 300 is interleaved with different memory channels (channels 0 to 15) at a granularity of 512B. The raw pointer of the physical memory 330 of the graphics processor device 300 points to the raw virtual address space 310. The data in the raw virtual address space 310 is arranged in the channel order of the physical address space of the physical memory, from channel 0 to channel 15.

[0046] For example, the physical memory page granularity is not aligned with the predetermined PCIe page granularity of 4KB. Similarly, the original virtual address space 310 of the graphics processor device 300 is not aligned with the 4KB page granularity. To align with the predetermined PCIe page granularity of 4KB, a virtual address space 320 for communication is constructed based on the physical address space of the graphics processor device 300, mapped to the physical address space at a 4KB page granularity. For example, 4KB of data in each channel is treated as a whole (e.g., the first 8 rows of 512B data in channel 0, or the first 8 rows of 512B data in channel 1), and mapped to the virtual address space 320 for communication in order from channel 0 to channel 15. Thus, the constructed virtual address space 320 for communication is aligned with the predetermined PCIe page granularity of 4KB, thereby enabling the graphics processor device 300 to perform GDR communication.

[0047] It should be understood that when the page granularity of the device memory differs from the predetermined page granularity of PCIe, different mapping formulas can be obtained for constructing the virtual address space used for communication. Figure 3 This example only uses a raw memory page granularity of 512B and a PCIe pre-defined page granularity of 4KB. It should be understood that other cases could also exist, such as a raw memory page granularity of 2KB and a PCIe pre-defined page granularity of 4KB. Furthermore, Figure 3 The graphics processing unit 300 can be used as either a source or a target device, without limitation. Furthermore, when both the source and target devices exhibit memory page granularity misalignment, separate virtual address spaces for communication must be constructed for each device, ensuring that these spaces are aligned to the predetermined PCIe page granularity.

[0048] For example, in some embodiments, if a device simultaneously has an original virtual address space and a constructed virtual address space for communication (or a "new virtual address space"), the same physical address space of the device corresponds to two virtual address spaces. For example, please refer to... Figure 3For example, the red data "512B" in the first row and second column of channel 1 in the physical address space corresponds to the red position in the original virtual address space 310, and also corresponds to the red position in the virtual address space 320 used for communication; similarly, the blue data "512B" in the second row and first column of channel 0 in physical memory 330 corresponds to the blue position in the original virtual address space 310, and also corresponds to the blue position in the virtual address space 320 used for communication.

[0049] It should be understood that, given the possibility that the same physical address space simultaneously corresponds to both the original virtual address space 310 and the constructed virtual address space 320 used for communication, it is necessary to increment the reference count of the physical address space mapped to the virtual address space by "1". Furthermore, when releasing this same physical address space, it is necessary to ensure that all virtual addresses pointing to this same physical address space are released. For example, the computing device 100 releases both the original virtual address space and the constructed virtual address space used for communication that point to the same physical address space, thus releasing the same physical address space. The following will combine... Figure 6 The method for releasing physical address space is explained in detail in section 600, and will not be repeated here.

[0050] For example, in some embodiments, the constructed virtual address space for communication is mapped to the PCIe base address register so that external devices connected via the PCIe can directly access the physical memory corresponding to the constructed virtual address space for communication. For example, please refer to... Figure 4 , Figure 4 The diagram illustrates source device 400. Source device 400 includes: source device physical memory 410, source device kernel-mode driver 420, source device virtual address space 430, source device PCIe 440, source device 400's physical address space 411, source device 400's raw virtual address space 431, and source device 400's virtual address space 432 for communication. For example, source device 400's raw virtual address space 431 is not aligned to the predetermined page granularity of PCIe. Based on source device 400's physical address space 411, a virtual address space 432 for communication is constructed, mapped to source device 400's physical address space 411, with a predetermined page granularity. The virtual address space 432 for communication is aligned to the predetermined page granularity of PCIe. The virtual address space 432 for communication is memory-mapped into the base address register (i.e., PCIe BAR) of source device PCIe 440.

[0051] Thus, the device can be equipped with GDR communication technology. For example, an external device (such as the target device) connected via the source device's PCIE440 can directly access the corresponding physical memory through the constructed virtual address space for communication.

[0052] At step 206, the computing device 100 communicates between the memory of the source device and the target device via direct remote memory access based on the constructed virtual address space for communication.

[0053] For example, in some embodiments, the computing device 100 performs memory remapping (i.e., remapping of the physical address space of memory to the virtual address space) based on a constructed virtual address space for communication for memory in the source device and the target device that is not aligned at the original page granularity, so that both the source device and the target device have a virtual address space aligned with the predetermined size page granularity of PCIe; and directly copies the physical memory data of the source device to the physical memory of the target device through the predetermined size page granularity aligned virtual address space of the source device and the target device in a direct remote memory access manner.

[0054] For example, please refer to Figure 6 , Figure 6 The diagram illustrates source device 400 and target device 500. The internal structure of source device 400 has been described above and will not be repeated here. Target device 500 includes, for example, target device physical memory 510, target device kernel-mode driver 520, and target device virtual address space 530. For instance, the original virtual address space 531 of target device 500 is not aligned to the predetermined page granularity of PCIe. Based on the physical address space 511 of target device 500, a virtual address space 532 for communication of target device 500 is constructed with a predetermined page granularity, mapping to the physical address space 511 of target device 500. The virtual address space 532 for communication of target device 500 is aligned to the predetermined page granularity of PCIe.

[0055] Please continue to refer to this. Figure 5 The original virtual memory address spaces of both the source device 400 and the target device 500 were not aligned with the predetermined page granularity of PCIe. After memory remapping, both have virtual address spaces aligned with the predetermined page granularity of PCIe (e.g., virtual address space 432 and virtual address space 532). Therefore, both have the access conditions for GDR communication technology. Through the virtual address space 532 of the target device 500 used for communication and the virtual address space 432 of the source device 400 used for communication, the physical memory data of the source device can be directly copied into the physical memory of the target device, that is, the direct remote memory access (GDR) between the two is realized.

[0056] In the above scheme, the memory page granularity of the source and target devices to be communicated is obtained respectively. If at least one unaligned PCIe page granularity of the source and target device memory is determined, a virtual address space mapped to the physical address space is constructed based on the physical address space of the unaligned memory, with a page granularity of the predetermined size. This ensures that the mapped virtual address space used for communication meets the requirements of the PCIe page table. The unaligned device can be either the source or target device, or neither device can be aligned. Thus, through memory remapping, both the source and target devices can have virtual memory spaces that meet the alignment requirements of the PCIe page table, thereby enabling both devices to meet the conditions for using Direct Remote Memory Access (GDR) technology. Then, based on the constructed virtual address space, communication is performed between the memory of the source and target devices using GDR. Therefore, this invention achieves direct remote memory access for the graphics processor without configuring additional physical memory aligned to memory pages as an intermediate cache, thus saving memory overhead. Furthermore, since no additional intermediate cache needs to be configured, there is no need for memory copying or memory clearing commands related to the intermediate cache, which simplifies memory access operations and significantly reduces transmission latency.

[0057] Furthermore, the memory page granularity of the source device and the memory page granularity of the target device can be the same or different, which makes the solution provided by this invention have good compatibility and can be applied to GPU devices with various page granularities.

[0058] Furthermore, the above solution implements the remapping logic between physical address space and virtual address space at the driver level, thus remaining transparent to upper-layer applications. Upper-layer libraries only need to call the remap API, requiring minimal code modification and offering a simple and quick operation. Moreover, the remapped virtual address is only used during communication and does not disrupt the data arrangement order of the physical address space within the original virtual address space.

[0059] The following will combine Figure 6 and Figure 5 A method 600 for releasing physical address space, according to an embodiment of the present invention, is described. It should be understood that method 600 can, for example, be implemented in... Figure 1 The described computing device 100 performs the operation. Method 600 may also include additional actions not shown and / or the actions shown may be omitted; the scope of the invention is not limited in this respect.

[0060] At step 602, if the computing device 100 determines that the constructed virtual address space for communication has been constructed, the reference count of the physical address space mapped to the virtual address space is incremented by "1".

[0061] For example, please refer toFigure 5 The original virtual address space 431 of the source device 400 is not aligned to the predetermined size page granularity of PCIe. A virtual address space for communication is constructed for the source device 400. If it is determined that the virtual address space 432 for communication of the source device 400 has been constructed, the reference count of the physical address space 411 of the source device 400 mapped to the virtual address space 432 for communication of the source device 400 is incremented by "1". Figure 5 In this context, the physical address space 411 of the source device 400 is simultaneously referenced by both the original virtual address space 431 of the source device 400 and the virtual address space 432 of the source device 400 used for communication.

[0062] For example, please refer to Figure 5 The original virtual address space 531 of the target device 500 is not aligned to the predetermined size page granularity of PCIe. For the virtual address space constructed for communication by the target device 500, if it is determined that the virtual address space 532 for communication of the target device 500 has been constructed, the reference count of the physical address space 511 of the target device 500 mapped to the virtual address space 532 for communication of the target device 500 is incremented by "1"; Figure 5 In the target device 500, the physical address space 511 is simultaneously referenced by the target device 500's original virtual address space 531 and the target device 500's virtual address space 532 used for communication.

[0063] At step 604, if the computing device 100 determines that the original virtual address space pointing to the same physical address space or the constructed virtual address space for communication has been released, the reference count of the physical address space is decremented by "1".

[0064] At step 606, if the computing device 100 determines that the reference count of the same physical address space is "0", the same physical address space is released.

[0065] For example, continuing with the above example, please refer to... Figure 5The source device 400's original virtual address space 431 and its communication-use virtual address space 432 both point to the same physical address space (e.g., the source device 400's physical address space 411). If either of these virtual address spaces is released, the reference count of the source device 400's physical address space 411 is decremented by "1". For example, if the reference count of the source device 400's physical address space 411 is "2", and after data transmission is complete, the source device 400's communication-use virtual address space 432 is released, updating the reference count of the source device 400's physical address space 411 to "1", and then the source device 400's original virtual address space 431 is also released, updating the reference count of the source device 400's physical address space 411 to "0", confirming that the reference count of the source device 400's physical address space 411 is updated to "0", then the source device 400's physical address space 411 can be released.

[0066] For example, continuing with the above examples, please refer to... Figure 5 The original virtual address space 531 and the communication-use virtual address space 532 of the target device 500 both point to the same physical address space (e.g., the physical address space 511 of the target device 500). If either of these two virtual address spaces is released, the reference count of the physical address space 511 of the target device 500 is decremented by "1". For example, if the reference count of the physical address space 511 of the target device 500 is "2", and after data transmission is completed, the communication-use virtual address space 532 of the target device 500 is released, the reference count of the physical address space 511 of the target device 500 is updated to "1", and then the original virtual address space 531 of the target device 500 is also released, the reference count of the physical address space 511 of the target device 500 is updated to "0", confirming that the reference count of the physical address space 511 of the target device 500 is updated to "0", the physical address space 511 of the target device 500 can be released.

[0067] In the above method, when the device has built a virtual address space for communication, there will be a situation in the device where two virtual address spaces point to the same physical address space. By incrementing the reference count of the physical address space by "1", the storage space corresponding to the physical address space can only be released after both the original virtual address space and the built virtual address space for communication are released, thus improving the stability of the device.

[0068] The following will combine Figure 7 and Figure 8 This invention describes a method 700 for directly transferring data from the physical memory of a source device to the physical memory of a target device, according to embodiments of the present invention. It should be understood that method 700 can, for example, be used in...Figure 1 The described computing device 100 performs the operation. Method 700 may also include additional actions not shown and / or the actions shown may be omitted; the scope of the invention is not limited in this respect.

[0069] In some embodiments, if the source device includes a memory pool and the unaligned PCIe memory of the source device at a predetermined page size granularity is in the memory pool, the present invention also provides a method for directly transferring data in the physical memory of the source device to the physical memory of the target device, which includes, for example, steps 702 to 706.

[0070] At step 702, the computing device 100 performs memory remapping on the entire memory pool to obtain a virtual address space with a predetermined size page granularity aligned to the memory page granularity of the memory pool.

[0071] For example, please refer to Figure 8 , Figure 8 The diagram illustrates the original virtual address space 810 of the memory pool. For example, if the memory page granularity of the original virtual address space 810 of the memory pool is not aligned with the predetermined size page granularity of PCIe, the entire memory pool can be remapped to obtain a virtual address space 820 for communication, corresponding to the memory page granularity of the memory pool aligned with the predetermined size page granularity of PCIe.

[0072] At step 704, if the computing device 100 receives a data transmission request, it determines the pointer offset of the data to be transmitted in the memory page-aligned virtual address space based on the pointer offset of the data to be transmitted in the original virtual address space corresponding to the memory pool associated with the data transmission request, so as to determine the position of the data to be transmitted in the memory page-aligned virtual address space for communication.

[0073] For example, please refer to Figure 8 For the location a1 (virtual address 814) of the data to be transmitted a in the original virtual address space 810 of the memory pool, the pointer offset 812 of the starting point of virtual address 814 in the original virtual address space 810 of the memory pool is determined, and then the pointer offset 822 of the data to be transmitted a in the virtual address space 820 used for communication in the memory pool is determined accordingly, thereby determining the location a2 (virtual address 824) of the data to be transmitted a in the virtual address space 820 used for communication in the memory pool.

[0074] At step 706, the computing device 100 directly transfers data from the physical memory of the source device to the physical memory of the target device via direct remote memory access, based on the location of the data to be transferred in the virtual address space aligned to the memory page granularity.

[0075] For example, please refer to Figure 8 By means of direct remote memory access, the data to be transmitted is directly transferred from the physical memory of the memory pool to the physical memory of the target device by using the location a1 (i.e., virtual address 814) of the original virtual address space 810 of the memory pool and the location a2 (i.e., virtual address 824) of the virtual address space 820 of the memory pool used for communication.

[0076] The above solution provides a solution for scenarios with memory pools that require multiple data transfers. Before the first data transfer, the memory pool, which is not aligned to a predetermined size for PCIe, is remapped. The entire memory pool is remapped to obtain a virtual address space aligned to a predetermined size for PCIe. The offset of each requested data in the original virtual address space of the memory pool is used to determine the offset of the requested data in the virtual address space of the memory pool aligned to PCIe for communication. This allows data in the physical memory of the source device to be directly transferred to the physical memory of the target device via direct remote memory access.

[0077] It should be understood that in large models, memory is often managed by a memory pool, and the memory corresponding to the transferred data is often only a segment within that memory pool. The above scheme allows for a complete memory mapping of the memory pool before model execution begins, obtaining a complete virtual address space aligned to the predetermined size for PCIe. This eliminates the need for repeated memory mapping during subsequent data transfers; the data's position within the virtual address space can be determined using offsets, enabling efficient data transfer through direct remote memory access. Finally, at the end of the process, the predetermined size virtual address space aligned to the predetermined size for PCIe in the memory pool is released. Therefore, this scheme avoids repeated memory mapping of the same memory block during model execution, reducing memory consumption and improving data transfer efficiency.

[0078] The various processes and procedures described above, such as methods 200, 600, and 700, can be executed at a computing device. This computing device may include, for example, at least one processor (at least one graphics processor and at least one central processing unit); and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor. In some embodiments, methods 200, 600, and 700 may be implemented as a computer software program or program product tangibly contained in a machine-readable medium. In some embodiments, part or all of the computer program may be loaded and / or installed on the computing device via read-only memory (ROM) and / or a communication unit. When the computer program is loaded into random-access memory (RAM) and executed by the GPU and CPU, one or more actions of methods 200, 600, and 700 described above can be performed.

[0079] This invention can be a method, apparatus, system, and / or computer program product. A computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for performing various aspects of the invention. The computer-readable storage medium may be a tangible device capable of holding and storing instructions used by an instruction execution device. The computer-readable storage medium may be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination thereof.

[0080] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to various computing / processing devices, or downloaded via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network, to an external computer or external storage device. Various aspects of the invention are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0081] These computer-readable program instructions can be provided to the central processing unit of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the central processing unit of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of a flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner. Thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of a flowchart and / or block diagram.

[0082] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction, which contains one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those shown in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0083] It should be understood that the various forms of processes shown above can be used to reorder, add, or delete steps. For example, the steps loaded in this application can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this application can be achieved, and this is not limited herein.

[0084] The specific embodiments described above do not constitute a limitation on the scope of protection of this application. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors.

Claims

1. A method for direct remote memory access for a graphics processor, characterized in that, include: Obtain the page granularity of the source device memory and the target device memory to be communicated, respectively. The source device and the target device each include a graphics processor. In response to determining a predetermined size page granularity of at least one unaligned PCIe memory in the source device memory and the target device memory, a virtual address space for communication is constructed based on the physical address space of the unaligned memory of the device at a predetermined size page granularity, such that the mapped virtual address space satisfies the requirements of the PCIe page table. as well as Based on the constructed virtual address space for communication, communication is conducted between the memory of the source device and the target device via direct remote memory access.

2. The method according to claim 1, characterized in that, Based on the physical address space of device-unaligned memory, constructing a virtual address space for communication that maps to the physical address space at a predetermined page granularity includes: For the physical address space of unaligned memory in the device, the data of a predetermined size in each channel of the physical address space is mapped as a whole to the constructed virtual address space in channel order, so that the constructed virtual address space for communication is aligned with the predetermined size page granularity of PCIe.

3. The method according to claim 2, characterized in that, Also includes: The constructed virtual address space for communication is mapped to the PCIe base address register so that external devices connected via the PCIe can directly access the corresponding physical memory through the constructed virtual address space for communication.

4. The method according to claim 1, characterized in that, Also includes: This releases both the original virtual address space and the constructed virtual address space used for communication, which both point to the same physical address space, thus releasing the same physical address space.

5. The method according to claim 4, characterized in that, This releases both the original virtual address space and the constructed virtual address space used for communication, both of which point to the same physical address space. Releasing the same physical address space includes: In response to determining that the constructed virtual address space for communication has been constructed, the reference count of the physical address space mapped to the virtual address space is incremented by "1"; In response to the release of the original virtual address space pointing to the same physical address space or the virtual address space constructed for communication, the reference count of the physical address space is decremented by "1"; and In response to the reference count of the same physical address space being "0", the same physical address space is released.

6. The method according to claim 1, characterized in that, Based on the constructed virtual address space for communication, communication between the source and target devices via direct remote memory access includes: For memory in both the source and target devices that is not aligned at the original page granularity, memory remapping is performed based on the constructed virtual address space used for communication, so that both the source and target devices have virtual address spaces aligned with the predetermined page granularity of PCIe; and By using direct remote memory access, the physical memory data of the source device is directly copied into the physical memory of the target device through virtual address spaces aligned with predetermined page granularity of the source and target devices.

7. The method according to claim 1, characterized in that, The memory page granularity of the source device and the memory page granularity of the target device may be the same or different.

8. The method according to claim 1, characterized in that, If the source device contains a memory pool, and the source device's unaligned PCIe memory with a predetermined page granularity is located in the memory pool, the method further includes: The entire memory pool is remapped to obtain a virtual address space for communication that corresponds to the memory page granularity of the memory pool and is aligned with the predetermined size page granularity of PCIe. In response to receiving a data transfer request, based on the pointer offset of the data to be transferred in the original virtual address space corresponding to the memory pool related to the data transfer request, the pointer offset of the data to be transferred in the memory page-granular-aligned virtual address space is determined, so as to determine the position of the data to be transferred in the memory page-granular-aligned virtual address space; and By means of direct remote memory access, data in the physical memory of the source device is directly transferred to the physical memory of the target device by means of the location of the data to be transferred in the virtual address space aligned with the memory page granularity.

9. A computing device, characterized in that, include: At least one processor; as well as A memory that is communicatively connected to the at least one processor; in The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-8.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a machine, performs the method according to any one of claims 1-8.

11. A computer program product, characterized in that, Includes a computer program, which, when executed by a machine, performs the method according to any one of claims 1-8.

Images