Virtual-to-real address translation lookup strategy for MCM-GPU

Abstract

The present invention discloses a virtual-to-real address translation lookup strategy for an MCM-GPU, comprising: setting a shared tracking unit in an input / output memory management unit (I / O MM) that initially records entry information in the L2TLBs of all GPU modules; when a page fault occurs in the L2TLB of a GPU module and a request is forwarded to the I / O MM, querying the shared tracking unit to determine whether there is a corresponding virtual-to-real address translation entry in other GPU modules; if so, directing the request to the corresponding GPU module to access the L2TLB and complete the virtual-to-real address translation; if not, performing a page table traversal in the I / O MM to obtain a virtual-to-real address translation result, writing the translation result information to the L2TLB of the GPU module that sent the request, and synchronously recording the written information in the shared tracking unit. The present invention can reduce the remote memory access overhead caused by performing page table traversal and improve overall performance.

Description

Technical Field

[0001] The present invention relates to the field of GPU technology, and in particular to a virtual-to-real address conversion lookup strategy for MCM-GPU. Background Art

[0002] With the continuous development of GPU (Graphics Processing Unit) technology, the number of GPU cores and computing power has continued to increase. For example, from the Fermi to Volta architectures, the number of streaming multi-processors (SMs) in GPUs has increased from 14 to 80. Due to manufacturing process and chip size limitations, integrating a large number of SMs on a single chip has become increasingly difficult. To further improve GPU performance and avoid chip size constraints, a GPU architecture designed using a multi-chip module (MCM) approach has been proposed. The MCM design is a discrete design in which the processor package contains multiple smaller chiplets, each containing only a fraction of the resources of a traditional monolithic system-on-a-chip (SoC). This makes it easier and cheaper to manufacture individual chiplets. These chiplets are connected together through fast, high-bandwidth chip package interconnects, providing more computing resources. The MCM-GPU architecture is a high-performance computing architecture that improves computing performance by integrating multiple modules in a single package. Compared with traditional high-throughput processor architectures, MCM-GPU has higher computing density, lower power consumption and better scalability. MCM-GPU improves computing performance by increasing the number of modules and can use more efficient interconnection technology without increasing the system footprint.

[0003] refer to Figure 1 , Figure 1The following is an example of an existing MCM-GPU architecture diagram. The existing MCM-GPU includes multiple GPU modules, and each GPU also includes a multi-level TLB (Translation Lookaside Buffer) for address translation. Specifically, in the GPU, each compute unit (CU) has a fully associative private L1 TLB, and all compute units share a larger L2 TLB. In an MCM-GPU, all GPUs are connected together through a high-speed bandwidth connection (such as InfiniBand) and connected to the CPU. On the CPU side, a shared input / output memory management unit (IOMMU) is used to process requests generated by all GPUs. Each GPU has its own local memory, but the page table is centralized in the CPU memory and controlled by the CPU. Therefore, if a page fault occurs in the GPU's L2 TLB, the request needs to be forwarded to the input / output memory management unit for a remote page table walk (PTW).

[0004] However, since the page table of the input / output memory management unit is located in the CPU-side memory, accessing this remote memory resource not only increases data transmission latency but also significantly increases the GPU's address translation overhead. Especially in high-concurrency access mode, the L2 TLBs of most GPU modules in the MCM-GPU will miss pages one after another in a short period of time, further increasing the burden on the IOMMU, thereby having a serious negative impact on the overall system performance. Summary of the Invention

[0005] In order to solve some or all of the technical problems existing in the above-mentioned prior art, the present invention provides a virtual-to-real address conversion lookup strategy for MCM-GPU.

[0006] The technical solutions of the present invention are as follows:

[0007] A virtual-to-real address translation lookup strategy for MCM-GPU is provided, comprising the following steps:

[0008] A shared tracking unit is set in the input / output memory management unit of the CPU connected to the MCM-GPU, and the shared tracking unit initially records the entry information in the private L2 TLB of all GPU modules in the MCM-GPU;

[0009] When a page fault occurs in the L2 TLB of a GPU module in the MCM-GPU and a request is forwarded to the input / output memory management unit, the information recorded in the shared tracking unit is queried to determine whether there is a corresponding virtual-to-real address translation table entry in other GPU modules;

[0010] If so, the request is directed to the corresponding GPU module to access the L2 TLB of the GPU module to complete the virtual-to-real address conversion;

[0011] If not, perform page table traversal in the input / output memory management unit to obtain the virtual-to-real address conversion result, write the virtual-to-real address conversion result information into the L2 TLB of the GPU module that sends the request, and synchronously record the write information into the shared tracking unit.

[0012] In some optional implementations, the strategy further includes:

[0013] If a page fault occurs in the L2 TLB when the request is directed to the corresponding GPU module to access the L2 TLB of the GPU module, a page table traversal is performed in the input / output memory management unit according to the request to obtain the virtual-to-real address conversion result, and the virtual-to-real address conversion result information is written into the L2 TLB of the GPU module that sent the request, and the written information is synchronously recorded in the shared tracking unit.

[0014] In some optional implementations, the strategy further includes:

[0015] When writing information to the shared tracking unit, if the available cache lines in the shared tracking unit are insufficient, the cache line that has not been used for the longest time is selected for information replacement.

[0016] In some optional embodiments, the shared tracking unit adopts a 4-way group associative structure, each 4-way group includes 4 cache lines, the cache lines are used to record table entry information in the L2 TLB, and each cache line includes a virtual page number, a GPU module list and a valid bit.

[0017] In some optional implementations, the strategy further includes:

[0018] For each 4-way group of the shared tracking unit, a corresponding 3-bit replacement array is set, and the 3-bit replacement array is used to indicate the usage time sequence of the 4 cache lines in the corresponding 4-way group.

[0019] In some optional embodiments, the three bits in the 3-bit replacement array are used to indicate the relative usage time sequence of the first cache line and the second cache line, the relative usage time sequence of the third cache line and the fourth cache line, and the relative usage time sequence of the first pair of cache lines consisting of the first cache line and the second cache line and the second pair of cache lines consisting of the third cache line and the fourth cache line, respectively.

[0020] In some optional embodiments, the first bit in the 3-bit replacement array is used to indicate the relative usage time sequence of the first cache line and the second cache line, the second bit is used to indicate the relative usage time sequence of the third cache line and the fourth cache line, and the third bit is used to indicate the relative usage time sequence of a first pair of cache lines consisting of the first cache line and the second cache line and a second pair of cache lines consisting of the third cache line and the fourth cache line.

[0021] In some optional embodiments, when the first bit in the 3-bit replacement array is 0, it indicates that the first cache line is used earlier than the second cache line, and when the first bit in the 3-bit replacement array is 1, it indicates that the first cache line is used later than the second cache line;

[0022] When the second bit in the 3-bit replacement array is 0, it indicates that the use time of the third cache line is earlier than the use time of the fourth cache line; when the second bit in the 3-bit replacement array is 1, it indicates that the use time of the third cache line is later than the use time of the fourth cache line;

[0023] When the third bit in the 3-bit replacement array is 0, it indicates that the first pair of cache lines is used earlier than the second pair of cache lines. When the first bit in the 3-bit replacement array is 1, it indicates that the first pair of cache lines is used later than the second pair of cache lines.

[0024] In some optional implementations, each time a cache line is accessed, the 3-bit replacement array is synchronously updated in the following manner:

[0025] When accessing the first cache line, set the first bit of the 3-bit replacement array to 1 and the third bit to 1;

[0026] When accessing the second cache line, set the first bit of the 3-bit replacement array to 0 and the third bit to 1;

[0027] When accessing the third cache line, the second bit of the 3-bit replacement array is set to 1 and the third bit is set to 0;

[0028] When the fourth cache line is accessed, the second bit of the 3-bit replacement array is set to 0 and the third bit is set to 0.

[0029] In some optional implementations, based on the 3-bit replacement array, the cache line that has not been used for a longer period is determined by:

[0030] Query the value of the third bit in the 3-bit replacement array. If the value of the third bit is 0, further query the value of the first bit. If the value of the third bit is 1, further query the value of the second bit.

[0031] If the value of the first bit is 0, the first cache line in the corresponding 4-way group is determined to be the cache line that has not been used for a longer time; if the value of the first bit is 1, the second cache line in the corresponding 4-way group is determined to be the cache line that has not been used for a longer time;

[0032] If the value of the second bit is 0, the third cache line in the corresponding 4-way group is determined to be a cache line that has not been used for a longer time. If the value of the second bit is 1, the fourth cache line in the corresponding 4-way group is determined to be a cache line that has not been used for a longer time.

[0033] The main advantages of the technical solution of the present invention are as follows:

[0034] The virtual-to-real address translation lookup strategy for an MCM-GPU of the present invention sets a shared tracking unit in an input / output memory management unit, and uses the shared tracking unit to record entry information in the private L2 TLBs of all GPU modules in the MCM-GPU. When a page fault occurs in the L2 TLB of a certain GPU module, the information recorded in the shared tracking unit can be queried to determine whether other GPU modules already have corresponding virtual-to-real address translation entries. If so, the L2 TLB of the corresponding GPU module is directly accessed for address translation. This reduces the remote memory access overhead caused by page table traversal and improves the overall performance of the MCM-GPU. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] The drawings described herein are used to provide a further understanding of the embodiments of the present invention and constitute a part of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0036] Figure 1 A schematic diagram of an existing MCM-GPU architecture is provided as an example;

[0037] Figure 2 A flowchart of a virtual-to-real address translation lookup strategy for an MCM-GPU provided by an embodiment of the present invention;

[0038] Figure 3 A schematic diagram of the MCM-GPU architecture provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0039] To make the objectives, technical solutions, and advantages of the present invention more clear, the technical solutions of the present invention will be clearly and completely described below in conjunction with specific embodiments of the present invention and corresponding drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0040] The technical solutions provided by the embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0041] refer to Figure 2-3 , an embodiment of the present invention provides a virtual-to-real address translation lookup strategy for an MCM-GPU, the strategy comprising the following steps:

[0042] Step 1: Set up a shared tracking unit (Share Tracker) in the input / output memory management unit (IOMMU) of the CPU connected to the MCM-GPU, and the shared tracking unit initially records the table entry information in the private L2 TLB of all GPU modules in the MCM-GPU;

[0043] Step 2: When a page fault occurs in the L2 TLB of a GPU module in the MCM-GPU and the request is forwarded to the input / output memory management unit, the information recorded in the shared tracking unit is queried to determine whether there is a corresponding virtual-to-real address translation table entry in other GPU modules;

[0044] Step 3: If yes, direct the request to the corresponding GPU module to access the L2 TLB of the GPU module to complete the virtual-to-real address conversion;

[0045] Step 4: If not, perform page table traversal in the input / output memory management unit to obtain the virtual-to-real address conversion result, write the virtual-to-real address conversion result information into the L2 TLB of the GPU module that sent the request, and synchronously record the write information into the shared tracking unit.

[0046] The virtual-to-real address translation lookup strategy for an MCM-GPU provided by an embodiment of the present invention sets a shared tracking unit in an input / output memory management unit and uses the shared tracking unit to record entry information in the private L2TLBs of all GPU modules in the MCM-GPU. When a page fault occurs in the L2 TLB of a GPU module, the information recorded in the shared tracking unit can be queried to determine whether other GPU modules already have corresponding virtual-to-real address translation entries. If so, the L2 TLB of the corresponding GPU module is directly accessed for address translation. This reduces the remote memory access overhead caused by page table traversal (TLB) and improves the overall performance of the MCM-GPU.

[0047] Furthermore, in this embodiment of the present invention, considering that when the shared tracking unit directs a request to the GPU module's L2 TLB, the corresponding entry in the GPU module's L2 TLB may have been replaced and cleared, thereby causing a secondary L2 TLB failure, in which case the request is forwarded to the I / O memory management unit again. To this end, the strategy also includes the following steps:

[0048] If a page fault occurs in the L2 TLB when the request is directed to the corresponding GPU module to access the L2 TLB of the GPU module, a page table traversal is performed in the input / output memory management unit according to the request to obtain the virtual-to-real address conversion result, and the virtual-to-real address conversion result information is written into the L2 TLB of the GPU module that sent the request, and the write information is synchronously recorded in the shared tracking unit.

[0049] This configuration can avoid adding additional delay by repeatedly querying the shared tracking unit in the event of multiple failures.

[0050] Furthermore, in an embodiment of the present invention, considering that the cache space of the shared tracking unit is limited, in order to improve the utilization of the information recorded in the shared tracking unit, thereby reducing the number of page table traversals and lowering memory access latency, the strategy further includes the following steps:

[0051] When synchronously recording the write information to the shared tracking unit, if the available cache lines in the shared tracking unit are insufficient, the cache line that has not been used for the longest time is selected for information replacement.

[0052] refer to Figure 3 Furthermore, in an embodiment of the present invention, in order to facilitate the shared tracking unit to record the table entry information in the L2 TLB, the shared tracking unit adopts a 4-way set associative structure, each 4-way group includes 4 cache lines, and the cache line is used to record the table entry information in the L2 TLB. Each cache line includes a virtual page number (VPN), a GPU module list (GPU List), and a valid bit (Valid Bit).

[0053] In an embodiment of the present invention, the virtual page number in the cache line is used to record the virtual page number of the entry, the GPU module list is used to record the GPU module ID containing the entry, and the valid bit is used to mark whether the line is valid.

[0054] Furthermore, in an embodiment of the present invention, the strategy further includes:

[0055] For each 4-way group of the shared tracking unit, a corresponding 3-bit replacement array is set, and the 3-bit replacement array is used to indicate the usage time sequence of the 4 cache lines in the corresponding 4-way group.

[0056] In the embodiment of the present invention, a 3-bit replacement array is set to indicate the usage time sequence of cache lines in the shared tracking unit, so that cache lines that have not been used for a longer time can be accurately and quickly located to facilitate subsequent cache line information replacement processing.

[0057] Furthermore, in an embodiment of the present invention, the three bits in the 3-bit replacement array are respectively used to indicate the relative usage time sequence of the first cache line and the second cache line, the relative usage time sequence of the third cache line and the fourth cache line, and the relative usage time sequence of the first pair of cache lines consisting of the first cache line and the second cache line and the second pair of cache lines consisting of the third cache line and the fourth cache line.

[0058] In the embodiment of the present invention, by adopting the above-set indication method, the cache line that has not been used for a longer period of time among the four cache lines in the 4-way group can be quickly determined.

[0059] In an optional implementation of an embodiment of the present invention, the first bit in the 3-bit replacement array is used to indicate the relative usage time sequence of the first cache line and the second cache line, the second bit is used to indicate the relative usage time sequence of the third cache line and the fourth cache line, and the third bit is used to indicate the relative usage time sequence of a first pair of cache lines consisting of the first cache line and the second cache line and a second pair of cache lines consisting of the third cache line and the fourth cache line.

[0060] Furthermore, in an optional implementation of the embodiment of the present invention, considering that the value of each bit in the 3-bit replacement array can be 0 or 1, based on the indication method of the 3-bit replacement array specifically defined above, when the first bit in the 3-bit replacement array is 0, it indicates that the usage time of the first cache line is earlier than the usage time of the second cache line; when the first bit in the 3-bit replacement array is 1, it indicates that the usage time of the first cache line is later than the usage time of the second cache line; when the second bit in the 3-bit replacement array is 0, it indicates that the usage time of the third cache line is earlier than the usage time of the fourth cache line; when the second bit in the 3-bit replacement array is 1, it indicates that the usage time of the third cache line is later than the usage time of the fourth cache line; when the third bit in the 3-bit replacement array is 0, it indicates that the usage time of the first pair of cache lines is earlier than the usage time of the second pair of cache lines; when the first bit in the 3-bit replacement array is 1, it indicates that the usage time of the first pair of cache lines is later than the usage time of the second pair of cache lines.

[0061] Furthermore, in an embodiment of the present invention, based on the above setting for the 3-bit replacement array, each time a cache line is accessed, the 3-bit replacement array is synchronously updated in the following manner:

[0062] When accessing the first cache line, set the first bit of the 3-bit replacement array to 1 and the third bit to 1;

[0063] When accessing the second cache line, set the first bit of the 3-bit replacement array to 0 and the third bit to 1;

[0064] When accessing the third cache line, the second bit of the 3-bit replacement array is set to 1 and the third bit is set to 0;

[0065] When the fourth cache line is accessed, the second bit of the 3-bit replacement array is set to 0 and the third bit is set to 0.

[0066] In this way, it is ensured that after each access to a cache line, a cache line that has not been used for a longer period of time can still be accurately located according to the 3-bit replacement array, so as to facilitate subsequent cache line information replacement processing.

[0067] Furthermore, in an embodiment of the present invention, based on the above setting for the 3-bit replacement array, the cache line that has not been used for a longer period of time is determined according to the 3-bit replacement array in the following manner:

[0068] Query the value of the third bit in the 3-bit replacement array. If the value of the third bit is 0, further query the value of the first bit. If the value of the third bit is 1, further query the value of the second bit.

[0069] If the value of the first bit is 0, the first cache line in the corresponding 4-way group is determined to be the cache line that has not been used for a longer time; if the value of the first bit is 1, the second cache line in the corresponding 4-way group is determined to be the cache line that has not been used for a longer time;

[0070] If the value of the second bit is 0, the third cache line in the corresponding 4-way group is determined to be a cache line that has not been used for a longer time. If the value of the second bit is 1, the fourth cache line in the corresponding 4-way group is determined to be a cache line that has not been used for a longer time.

[0071] In this way, the cache line that has not been used for a longer period of time can be accurately determined based on the 3-bit replacement array, so as to facilitate subsequent cache line information replacement processing.

[0072] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device that includes a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In addition, "front", "back", "left", "right", "upper" and "lower" in this document are all referenced to the placement states shown in the accompanying drawings.

[0073] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A virtual-to-real address translation lookup strategy for MCM-GPU, characterized in that: The MCM-GPU includes multiple GPU modules, each of which also includes a multi-level TLB for address translation. In the GPU, each computing unit has a fully associative private L1 TLB, while all computing units share a larger L2 TLB. The virtual-to-real address translation lookup strategy for the MCM-GPU includes the following steps: A shared tracking unit is set in the input / output memory management unit of the CPU connected to the MCM-GPU, and the shared tracking unit initially records the entry information in the private L2 TLB of all GPU modules in the MCM-GPU; When a page fault occurs in the L2 TLB of a GPU module in the MCM-GPU and a request is forwarded to the input / output memory management unit, the information recorded in the shared tracking unit is queried to determine whether there is a corresponding virtual-to-real address translation table entry in other GPU modules; If so, the request is directed to the corresponding GPU module to access the L2 TLB of the GPU module to complete the virtual-to-real address conversion; If not, perform a page table traversal in the input / output memory management unit to obtain a virtual-to-real address translation result, write the virtual-to-real address translation result information into the L2 TLB of the GPU module that sent the request, and synchronously record the written information into the shared tracking unit; Also includes: If a page fault occurs in the L2 TLB when the request is directed to the corresponding GPU module to access the GPU module's L2 TLB, a page table traversal is performed in the input / output memory management unit according to the request to obtain a virtual-to-real address translation result, and the virtual-to-real address translation result information is written to the L2 TLB of the GPU module that sent the request. The written information is also synchronously recorded in the shared tracking unit to avoid repeated querying of the shared tracking unit in the case of multiple failures, thereby increasing additional delay; Also includes: When writing information to the shared tracking unit, if the available cache lines in the shared tracking unit are insufficient, the cache line that has not been used for the longest time is selected for information replacement.

2. The virtual-to-real address translation lookup strategy for MCM-GPU according to claim 1, characterized in that: The shared tracking unit adopts a 4-way set associative structure. Each 4-way set includes 4 cache lines. The cache line is used to record the table entry information in the L2 TLB. Each cache line includes a virtual page number, a GPU module list and a valid bit.

3. The virtual-to-real address translation lookup strategy for MCM-GPU according to claim 2, characterized in that: Also includes: For each 4-way group of the shared tracking unit, a corresponding 3-bit replacement array is set, and the 3-bit replacement array is used to indicate the usage time sequence of the 4 cache lines in the corresponding 4-way group.

4. The virtual-to-real address translation lookup strategy for MCM-GPU according to claim 3, characterized in that: The three bits in the 3-bit replacement array are used to indicate the relative usage time sequence of the first cache line and the second cache line, the relative usage time sequence of the third cache line and the fourth cache line, and the relative usage time sequence of the first pair of cache lines consisting of the first cache line and the second cache line and the second pair of cache lines consisting of the third cache line and the fourth cache line, respectively.

5. The virtual-to-real address translation lookup strategy for MCM-GPU according to claim 4, characterized in that: The first bit in the 3-bit replacement array is used to indicate the relative usage time sequence of the first cache line and the second cache line, the second bit is used to indicate the relative usage time sequence of the third cache line and the fourth cache line, and the third bit is used to indicate the relative usage time sequence of the first pair of cache lines consisting of the first cache line and the second cache line and the second pair of cache lines consisting of the third cache line and the fourth cache line.

6. The virtual-to-real address translation lookup strategy for MCM-GPU according to claim 5, characterized in that: When the first bit in the 3-bit replacement array is 0, it indicates that the first cache line is used earlier than the second cache line; when the first bit in the 3-bit replacement array is 1, it indicates that the first cache line is used later than the second cache line; When the second bit in the 3-bit replacement array is 0, it indicates that the use time of the third cache line is earlier than the use time of the fourth cache line; when the second bit in the 3-bit replacement array is 1, it indicates that the use time of the third cache line is later than the use time of the fourth cache line; When the third bit in the 3-bit replacement array is 0, it indicates that the first pair of cache lines is used earlier than the second pair of cache lines. When the third bit in the 3-bit replacement array is 1, it indicates that the first pair of cache lines is used later than the second pair of cache lines.

7. The virtual-to-real address translation lookup strategy for MCM-GPU according to claim 6, characterized in that: On every access to a cache line, the 3-bit replacement array is synchronously updated in the following way: When accessing the first cache line, set the first bit of the 3-bit replacement array to 1 and the third bit to 1; When accessing the second cache line, set the first bit of the 3-bit replacement array to 0 and the third bit to 1; When accessing the third cache line, the second bit of the 3-bit replacement array is set to 1 and the third bit is set to 0; When the fourth cache line is accessed, the second bit of the 3-bit replacement array is set to 0 and the third bit is set to 0.

8. The virtual-to-real address translation lookup strategy for MCM-GPU according to claim 7, characterized in that: Based on the 3-bit replacement array, the cache line that has not been used for the longest time is determined as follows: Query the value of the third bit in the 3-bit replacement array. If the value of the third bit is 0, further query the value of the first bit. If the value of the third bit is 1, further query the value of the second bit. If the value of the first bit is 0, the first cache line in the corresponding 4-way group is determined to be the cache line that has not been used for a longer time; if the value of the first bit is 1, the second cache line in the corresponding 4-way group is determined to be the cache line that has not been used for a longer time; If the value of the second bit is 0, the third cache line in the corresponding 4-way group is determined to be a cache line that has not been used for a longer time. If the value of the second bit is 1, the fourth cache line in the corresponding 4-way group is determined to be a cache line that has not been used for a longer time.

Images