Virtual-real address translation lookup strategy for MCM-GPU

Abstract

The invention discloses a virtual-real address conversion lookup strategy for an MCM-GPU. The virtual-real address conversion lookup strategy comprises the steps that a sharing tracking unit which initially records table item information in L2TLB of all GPU modules is arranged in an input / output memory management unit; when an L2TLB of a certain GPU module has a missing page and a request is forwarded to an input / output memory management unit, querying a sharing tracking unit, and determining whether corresponding virtual-real address conversion table entries exist in other GPU modules; if yes, the request is guided to a corresponding GPU module to access the L2TLB, and virtual-real address conversion is completed; if not, page table traversal is carried out in the input / output memory management unit, a virtual-real address conversion result is obtained, conversion result information is written into the L2TLB of the GPU module sending the request, and the written information is synchronously recorded into the sharing tracking unit. According to the method, the access overhead of the far-end memory caused by page table traversal can be reduced, and the overall performance is improved.

Description

Technical Field

[0001] The present invention relates to the field of GPU technology, and in particular to a virtual-to-real address conversion lookup strategy for MCM-GPU. Background Art

[0002] With the continuous development of GPU (Graphics Processing Unit) technology, the number of GPU cores has been increasing, and the computing power has been increasing. For example, from Fermi to Volta architecture, the number of streaming multi-processors (SM) in GPU has increased from 14 to 80. Due to the limitations of manufacturing process and chip size, it is becoming increasingly difficult to integrate a large number of SMs on a single chip. In order to further improve the performance of GPU and avoid the constraints of chip size, a GPU architecture designed using a multi-chip module (MCM) method is currently proposed. The multi-chip module design is a discrete design in which the processor package contains multiple smaller cores, each of which contains only a small part of the resources of a traditional single-chip system-on-a-chip (SoC). It is easier and cheaper to manufacture a single core, and these cores are connected together through fast, high-bandwidth chip package internal interconnects to provide more computing resources. The MCM-GPU architecture is a high-performance computing architecture that improves computing performance by integrating multiple modules in one package. Compared with traditional high-throughput processor architectures, MCM-GPU has higher computing density, lower power consumption and better scalability. MCM-GPU improves computing performance by increasing the number of modules and can use more efficient interconnection technology without increasing the system footprint.

[0003] refer to Figure 1 , Figure 1An architecture diagram of an existing MCM-GPU is provided as an example. The existing MCM-GPU includes multiple GPU modules, and each GPU also includes a multi-level TLB (Translation Lookaside Buffer) for address translation. Specifically, in the GPU, each computing unit (CU) has a fully associative private L1 TLB, and all computing units share a larger L2 TLB. In the MCM-GPU, all GPUs are connected together through a high-speed bandwidth connection (such as InfiniBand) and connected to the CPU. On the CPU side, a shared input / output memory management unit (IOMMU) is used to process requests generated by all GPUs. Each GPU has its local memory, but the page table is concentrated in the CPU memory and controlled by the CPU. Therefore, if a page fault occurs in the L2 TLB of the GPU, the request needs to be forwarded to the input / output memory management unit for remote page table traversal (Page Table Walker, PTW).

[0004] However, since the page table of the input / output memory management unit is located in the CPU-side memory, accessing this remote memory resource not only increases the data transmission latency, but also significantly increases the address translation overhead of the GPU. Especially in high-concurrency access mode, the L2 TLBs of most GPU modules in the MCM-GPU fail pages one after another in a short period of time, further aggravating the burden of the IOMMU, thereby having a serious negative impact on the overall system performance. Summary of the invention

[0005] In order to solve some or all of the technical problems existing in the above-mentioned prior art, the present invention provides a virtual-to-real address conversion lookup strategy for MCM-GPU.

[0006] The technical solution of the present invention is as follows:

[0007] A virtual-to-real address translation lookup strategy for MCM-GPU is provided, comprising the following steps:

[0008] A shared tracking unit is set in the input / output memory management unit of the CPU connected to the MCM-GPU, and the shared tracking unit initially records the table entry information in the private L2 TLB of all GPU modules in the MCM-GPU;

[0009] When a page fault occurs in the L2 TLB of a GPU module in the MCM-GPU and a request is forwarded to the input / output memory management unit, the information recorded in the shared tracking unit is queried to determine whether there is a corresponding virtual-to-real address translation table entry in other GPU modules;

[0010] If so, the request is directed to the corresponding GPU module to access the L2 TLB of the GPU module to complete the virtual-to-real address conversion;

[0011] If not, perform page table traversal in the input / output memory management unit to obtain the virtual-to-real address conversion result, write the virtual-to-real address conversion result information into the L2 TLB of the GPU module that sends the request, and synchronously record the write information into the shared tracking unit.

[0012] In some optional implementations, the strategy further includes:

[0013] If a page fault occurs in the L2 TLB when the request is directed to the corresponding GPU module to access the L2 TLB of the GPU module, a page table traversal is performed in the input / output memory management unit according to the request to obtain the virtual-to-real address conversion result, and the virtual-to-real address conversion result information is written into the L2 TLB of the GPU module that sent the request, and the write information is synchronously recorded in the shared tracking unit.

[0014] In some optional implementations, the strategy further includes:

[0015] When the write information is synchronously recorded to the shared tracking unit, if the available cache lines in the shared tracking unit are insufficient, the cache line that has not been used for the longest time is selected for information replacement.

[0016] In some optional embodiments, the shared tracking unit adopts a 4-way group associative structure, each 4-way group includes 4 cache lines, the cache line is used to record the table entry information in the L2 TLB, and each cache line includes a virtual page number, a GPU module list and a valid bit.

[0017] In some optional implementations, the strategy further includes:

[0018] For each 4-way group of the shared tracking unit, a corresponding 3-bit replacement array is set, and the 3-bit replacement array is used to indicate the usage time sequence of the 4 cache lines in the corresponding 4-way group.

[0019] In some optional embodiments, the three bits in the three-bit replacement array are used to indicate the relative usage time order of the first cache line and the second cache line, the relative usage time order of the third cache line and the fourth cache line, and the relative usage time order of a first pair of cache lines consisting of the first cache line and the second cache line and a second pair of cache lines consisting of the third cache line and the fourth cache line, respectively.

[0020] In some optional embodiments, the first bit in the 3-bit replacement array is used to indicate the relative usage time sequence of the first cache line and the second cache line, the second bit is used to indicate the relative usage time sequence of the third cache line and the fourth cache line, and the third bit is used to indicate the relative usage time sequence of a first pair of cache lines consisting of the first cache line and the second cache line and a second pair of cache lines consisting of the third cache line and the fourth cache line.

[0021] In some optional embodiments, when the first bit in the 3-bit replacement array is 0, it indicates that the use time of the first cache line is earlier than the use time of the second cache line, and when the first bit in the 3-bit replacement array is 1, it indicates that the use time of the first cache line is later than the use time of the second cache line;

[0022] When the second bit in the 3-bit replacement array is 0, it indicates that the use time of the third cache line is earlier than the use time of the fourth cache line, and when the second bit in the 3-bit replacement array is 1, it indicates that the use time of the third cache line is later than the use time of the fourth cache line;

[0023] When the third bit in the 3-bit replacement array is 0, it indicates that the first pair of cache lines is used earlier than the second pair of cache lines. When the first bit in the 3-bit replacement array is 1, it indicates that the first pair of cache lines is used later than the second pair of cache lines.

[0024] In some optional implementations, each time a cache line is accessed, the 3-bit replacement array is synchronously updated in the following manner:

[0025] When accessing the first cache line, set the first bit of the 3-bit replacement array to 1 and the third bit to 1;

[0026] When accessing the second cache line, the first bit of the 3-bit replacement array is set to 0 and the third bit is set to 1;

[0027] When accessing the third cache line, the second bit of the 3-bit replacement array is set to 1 and the third bit is set to 0;

[0028] When the fourth cache line is accessed, the second bit of the 3-bit replacement array is set to 0 and the third bit is set to 0.

[0029] In some optional implementations, based on the 3-bit replacement array, the cache line that has not been used for a longer period of time is determined by:

[0030] Query the value of the third bit in the 3-bit replacement array. If the value of the third bit is 0, further query the value of the first bit. If the value of the third bit is 1, further query the value of the second bit.

[0031] If the value of the first bit is 0, the first cache line in the corresponding 4-way group is determined to be the cache line that has not been used for a longer time, and if the value of the first bit is 1, the second cache line in the corresponding 4-way group is determined to be the cache line that has not been used for a longer time;

[0032] If the value of the second bit is 0, the third cache line in the corresponding 4-way group is determined to be a cache line that has not been used for a longer time. If the value of the second bit is 1, the fourth cache line in the corresponding 4-way group is determined to be a cache line that has not been used for a longer time.

[0033] The main advantages of the technical solution of the present invention are as follows:

[0034] The virtual-to-real address conversion lookup strategy for MCM-GPU of the present invention sets a shared tracking unit in an input / output memory management unit, and uses the shared tracking unit to record the table entry information in the private L2 TLB of all GPU modules in the MCM-GPU. When a page fault occurs in the L2 TLB of a certain GPU module, the information recorded in the shared tracking unit can be queried first to determine whether other GPU modules have corresponding virtual-to-real address conversion table entries. If there are corresponding table entries, the L2 TLB of the corresponding GPU module is directly accessed for address conversion, thereby reducing the remote memory access overhead caused by page table traversal and improving the overall performance of the MCM-GPU. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] The drawings described herein are used to provide a further understanding of the embodiments of the present invention and constitute a part of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0036] Figure 1 A schematic diagram of an existing MCM-GPU architecture provided as an example;

[0037] Figure 2 A flowchart of a virtual-to-real address conversion lookup strategy for MCM-GPU provided by an embodiment of the present invention;

[0038] Figure 3 A schematic diagram of the system architecture of an MCM-GPU provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0039] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the specific embodiments of the present invention and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0040] The technical solution provided by the embodiments of the present invention is described in detail below with reference to the accompanying drawings.

[0041] refer to Figure 2-3 , an embodiment of the present invention provides a virtual-to-real address conversion lookup strategy for MCM-GPU, the strategy comprising the following steps:

[0042] Step 1, setting a shared tracking unit (Share Tracker) in the input / output memory management unit (IOMMU) of the CPU connected to the MCM-GPU, and the shared tracking unit initially records the table entry information in the private L2 TLB of all GPU modules in the MCM-GPU;

[0043] Step 2, when a page fault occurs in the L2 TLB of a GPU module in the MCM-GPU and the request is forwarded to the input / output memory management unit, the information recorded in the shared tracking unit is queried to determine whether there is a corresponding virtual-to-real address translation table entry in other GPU modules;

[0044] Step 3: If yes, direct the request to the corresponding GPU module to access the L2 TLB of the GPU module to complete the virtual-to-real address conversion;

[0045] Step 4: If not, perform page table traversal in the input / output memory management unit, obtain the virtual-to-real address conversion result, write the virtual-to-real address conversion result information into the L2 TLB of the GPU module that sent the request, and synchronously record the write information into the shared tracking unit.

[0046] The virtual-to-real address translation lookup strategy for MCM-GPU provided by the embodiment of the present invention sets a shared tracking unit in the input / output memory management unit, and uses the shared tracking unit to record the table entry information in the private L2TLB of all GPU modules in the MCM-GPU. When a page fault occurs in the L2 TLB of a certain GPU module, the information recorded in the shared tracking unit can be queried first to determine whether other GPU modules have corresponding virtual-to-real address translation table entries. If there are corresponding table entries, the L2 TLB of the corresponding GPU module is directly accessed for address translation, thereby reducing the remote memory access overhead caused by page table traversal (TLB) and improving the overall performance of the MCM-GPU.

[0047] Furthermore, in the embodiment of the present invention, considering that when the shared tracking unit directs the request to the L2TLB of the GPU module, the corresponding table entry in the L2 TLB of the GPU module may have been replaced and cleared, thereby causing the secondary L2 TLB to fail, in this case, the request will be forwarded to the input / output memory management unit again. To this end, the strategy also includes the following steps:

[0048] If a page fault occurs in the L2 TLB when the request is directed to the corresponding GPU module to access the L2 TLB of the GPU module, a page table traversal is performed in the input / output memory management unit according to the request to obtain the virtual-to-real address conversion result, and the virtual-to-real address conversion result information is written into the L2 TLB of the GPU module that sent the request, and the write information is synchronously recorded in the shared tracking unit.

[0049] This configuration can avoid adding additional delays by repeatedly querying the shared tracking unit in the event of multiple failures.

[0050] Furthermore, in the embodiment of the present invention, considering that the cache space of the shared tracking unit is limited, in order to improve the utilization rate of the information recorded in the shared tracking unit, thereby reducing the number of page table traversals and reducing memory access latency, the strategy also includes the following steps:

[0051] When synchronously recording the write information to the shared tracking unit, if the available cache lines in the shared tracking unit are insufficient, the cache line that has not been used for the longest time is selected for information replacement.

[0052] refer to Figure 3 Furthermore, in an embodiment of the present invention, in order to facilitate the shared tracking unit to record the table entry information in the L2 TLB, the shared tracking unit adopts a 4-way group associative structure, each 4-way group includes 4 cache lines, the cache line is used to record the table entry information in the L2 TLB, and each cache line includes a virtual page number (VPN), a GPU module list (GPU List) and a valid bit (Valid Bit).

[0053] In an embodiment of the present invention, the virtual page number in the cache line is used to record the virtual page number of the entry, the GPU module list is used to record the GPU module ID containing the entry, and the valid bit is used to mark whether the line is valid.

[0054] Furthermore, in the embodiment of the present invention, the strategy also includes:

[0055] For each 4-way group of the shared tracking unit, a corresponding 3-bit replacement array is set, and the 3-bit replacement array is used to indicate the usage time sequence of the 4 cache lines in the corresponding 4-way group.

[0056] In the embodiment of the present invention, by setting a 3-bit replacement array, the 3-bit replacement array is used to indicate the usage time sequence of cache lines in the shared tracking unit, so that cache lines that have not been used for a longer time can be accurately and quickly located to facilitate subsequent cache line information replacement processing.

[0057] Further, in an embodiment of the present invention, the three bits in the three-bit replacement array are respectively used to indicate the relative usage time sequence of the first cache line and the second cache line, the relative usage time sequence of the third cache line and the fourth cache line, and the relative usage time sequence of a first pair of cache lines consisting of the first cache line and the second cache line and a second pair of cache lines consisting of the third cache line and the fourth cache line.

[0058] In the embodiment of the present invention, by adopting the above-set indication method, the cache line that has not been used for a longer time among the four cache lines in the 4-way group can be quickly determined.

[0059] In an optional implementation of an embodiment of the present invention, the first bit in the 3-bit replacement array is used to indicate the relative usage time sequence of the first cache line and the second cache line, the second bit is used to indicate the relative usage time sequence of the third cache line and the fourth cache line, and the third bit is used to indicate the relative usage time sequence of a first pair of cache lines consisting of the first cache line and the second cache line and a second pair of cache lines consisting of the third cache line and the fourth cache line.

[0060] Further, in an optional implementation of the embodiment of the present invention, considering that the value of each bit in the 3-bit replacement array can be 0 or 1, based on the indication method of the 3-bit replacement array specifically defined above, when the first bit in the 3-bit replacement array is 0, it indicates that the usage time of the first cache line is earlier than the usage time of the second cache line, and when the first bit in the 3-bit replacement array is 1, it indicates that the usage time of the first cache line is later than the usage time of the second cache line; when the second bit in the 3-bit replacement array is 0, it indicates that the usage time of the third cache line is earlier than the usage time of the fourth cache line, and when the second bit in the 3-bit replacement array is 1, it indicates that the usage time of the third cache line is later than the usage time of the fourth cache line; when the third bit in the 3-bit replacement array is 0, it indicates that the usage time of the first pair of cache lines is earlier than the usage time of the second pair of cache lines, and when the first bit in the 3-bit replacement array is 1, it indicates that the usage time of the first pair of cache lines is later than the usage time of the second pair of cache lines.

[0061] Further, in the embodiment of the present invention, based on the above setting for the 3-bit replacement array, each time a cache line is accessed, the 3-bit replacement array is synchronously updated in the following manner:

[0062] When accessing the first cache line, set the first bit of the 3-bit replacement array to 1 and the third bit to 1;

[0063] When accessing the second cache line, the first bit of the 3-bit replacement array is set to 0 and the third bit is set to 1;

[0064] When accessing the third cache line, the second bit of the 3-bit replacement array is set to 1 and the third bit is set to 0;

[0065] When the fourth cache line is accessed, the second bit of the 3-bit replacement array is set to 0 and the third bit is set to 0.

[0066] In this way, it can be ensured that after each access to the cache line, the cache line that has not been used for a longer period of time can still be accurately located according to the 3-bit replacement array, so as to facilitate the subsequent cache line information replacement processing.

[0067] Further, in the embodiment of the present invention, based on the above setting for the 3-bit replacement array, the cache line that has not been used for a longer period of time is determined according to the 3-bit replacement array in the following manner:

[0068] Query the value of the third bit in the 3-bit replacement array. If the value of the third bit is 0, further query the value of the first bit. If the value of the third bit is 1, further query the value of the second bit.

[0069] If the value of the first bit is 0, the first cache line in the corresponding 4-way group is determined to be the cache line that has not been used for a longer time, and if the value of the first bit is 1, the second cache line in the corresponding 4-way group is determined to be the cache line that has not been used for a longer time;

[0070] If the value of the second bit is 0, the third cache line in the corresponding 4-way group is determined to be a cache line that has not been used for a longer time. If the value of the second bit is 1, the fourth cache line in the corresponding 4-way group is determined to be a cache line that has not been used for a longer time.

[0071] In this way, the cache lines that have not been used for a longer period of time can be accurately determined according to the 3-bit replacement array, so as to facilitate the subsequent information replacement processing of the cache lines.

[0072] It should be noted that, in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In addition, "front", "back", "left", "right", "upper" and "lower" in this article are all referenced to the placement state shown in the accompanying drawings.

[0073] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A virtual-to-real address conversion lookup strategy for MCM-GPU, characterized in that: The following steps are involved: A shared tracking unit is set in the input / output memory management unit of the CPU connected to the MCM-GPU, and the shared tracking unit initially records the table entry information in the private L2 TLB of all GPU modules in the MCM-GPU; When a page fault occurs in the L2 TLB of a GPU module in the MCM-GPU and a request is forwarded to the input / output memory management unit, the information recorded in the shared tracking unit is queried to determine whether there is a corresponding virtual-to-real address translation table entry in other GPU modules; If so, the request is directed to the corresponding GPU module to access the L2 TLB of the GPU module to complete the virtual-to-real address conversion; If not, perform page table traversal in the input / output memory management unit to obtain the virtual-to-real address conversion result, write the virtual-to-real address conversion result information into the L2 TLB of the GPU module that sends the request, and synchronously record the write information into the shared tracking unit.

2. The virtual-to-real address conversion lookup strategy for MCM-GPU according to claim 1, characterized in that: Also includes: If a page fault occurs in the L2 TLB when the request is directed to the corresponding GPU module to access the L2 TLB of the GPU module, a page table traversal is performed in the input / output memory management unit according to the request to obtain the virtual-to-real address conversion result, and the virtual-to-real address conversion result information is written into the L2 TLB of the GPU module that sent the request, and the write information is synchronously recorded in the shared tracking unit.

3. The virtual-to-real address conversion lookup strategy for MCM-GPU according to claim 1, characterized in that: Also includes: When the write information is synchronously recorded to the shared tracking unit, if the available cache lines in the shared tracking unit are insufficient, the cache line that has not been used for the longest time is selected for information replacement.

4. The virtual-to-real address conversion lookup strategy for MCM-GPU according to claim 3, characterized in that: The shared tracking unit adopts a 4-way set associative structure, each 4-way set includes 4 cache lines, the cache line is used to record the table entry information in the L2 TLB, and each cache line includes a virtual page number, a GPU module list and a valid bit.

5. The virtual-to-real address conversion lookup strategy for MCM-GPU according to claim 4, characterized in that: Also includes: For each 4-way group of the shared tracking unit, a corresponding 3-bit replacement array is set, and the 3-bit replacement array is used to indicate the usage time sequence of the 4 cache lines in the corresponding 4-way group.

6. The virtual-to-real address conversion lookup strategy for MCM-GPU according to claim 5, characterized in that: The three bits in the 3-bit replacement array are respectively used to indicate the relative usage time sequence of the first cache line and the second cache line, the relative usage time sequence of the third cache line and the fourth cache line, and the relative usage time sequence of a first pair of cache lines consisting of the first cache line and the second cache line and a second pair of cache lines consisting of the third cache line and the fourth cache line.

7. The virtual-to-real address conversion lookup strategy for MCM-GPU according to claim 6, characterized in that: The first bit in the 3-bit replacement array is used to indicate the relative usage time sequence of the first cache line and the second cache line, the second bit is used to indicate the relative usage time sequence of the third cache line and the fourth cache line, and the third bit is used to indicate the relative usage time sequence of a first pair of cache lines consisting of the first cache line and the second cache line and a second pair of cache lines consisting of the third cache line and the fourth cache line.

8. The virtual-to-real address conversion lookup strategy for MCM-GPU according to claim 7, characterized in that: When the first bit in the 3-bit replacement array is 0, it indicates that the use time of the first cache line is earlier than the use time of the second cache line, and when the first bit in the 3-bit replacement array is 1, it indicates that the use time of the first cache line is later than the use time of the second cache line; When the second bit in the 3-bit replacement array is 0, it indicates that the use time of the third cache line is earlier than the use time of the fourth cache line, and when the second bit in the 3-bit replacement array is 1, it indicates that the use time of the third cache line is later than the use time of the fourth cache line; When the third bit in the 3-bit replacement array is 0, it indicates that the first pair of cache lines is used earlier than the second pair of cache lines. When the first bit in the 3-bit replacement array is 1, it indicates that the first pair of cache lines is used later than the second pair of cache lines.

9. The virtual-to-real address conversion lookup strategy for MCM-GPU according to claim 8, characterized in that: On every access to a cache line, the 3-bit replacement array is synchronously updated in the following way: When accessing the first cache line, set the first bit of the 3-bit replacement array to 1 and the third bit to 1; When accessing the second cache line, the first bit of the 3-bit replacement array is set to 0 and the third bit is set to 1; When accessing the third cache line, the second bit of the 3-bit replacement array is set to 1 and the third bit is set to 0; When the fourth cache line is accessed, the second bit of the 3-bit replacement array is set to 0 and the third bit is set to 0.

10. The virtual-to-real address conversion lookup strategy for MCM-GPU according to claim 9, characterized in that: Based on the 3-bit replacement array, the cache line that has not been used for the longest time is determined by: Query the value of the third bit in the 3-bit replacement array. If the value of the third bit is 0, further query the value of the first bit. If the value of the third bit is 1, further query the value of the second bit. If the value of the first bit is 0, the first cache line in the corresponding 4-way group is determined to be the cache line that has not been used for a longer time, and if the value of the first bit is 1, the second cache line in the corresponding 4-way group is determined to be the cache line that has not been used for a longer time; If the value of the second bit is 0, the third cache line in the corresponding 4-way group is determined to be a cache line that has not been used for a longer time. If the value of the second bit is 1, the fourth cache line in the corresponding 4-way group is determined to be a cache line that has not been used for a longer time.

Images