Virtual and real address conversion searching method for MCM-GPU

Abstract

The technical scheme of the invention provides a virtual-real address conversion searching method for an MCM-GPU, and the method comprises the steps: each CU in the GPU corresponds to a respective rotation arbiter, the rotation arbiter receives a virtual-real address conversion request in a clock period, carries out the virtual-real address conversion in the CU in the same GPU according to the type of the virtual-real address conversion request, or carries out the virtual-real address conversion in the CU in the same GPU according to the type of the virtual-real address conversion request. And searching L2 TLBs of other GUPs to carry out virtual and real address conversion. The method has the advantages that the provided TLB detection mechanism can access the private L1 TLB of the adjacent CU in the same GPU to check whether virtual and real address translation hit exists when the local L1 TLB fails, and if the virtual and real address translation hit exists, a real memory page number is returned. If the private L1 TLB close to the CU is not hit, the virtual and real address conversion request is sent to the inter-chip interconnection network, and the corresponding shared L2 TLB is accessed through the inter-chip interconnection network.

Description

Technical Field

[0001] The present invention relates to the field of virtual-to-real address conversion, and in particular to a virtual-to-real address conversion search method for MCM-GPU. Background Art

[0002] As a discrete design, Multi-Chip Module GPU (MCM-GPU) integrates multiple GPU modules (GPU Modules, GPM) in a single package. Each chip only contains part of the resources in the traditional monolithic system-on-chip (SoC), so as to achieve continuous performance expansion.

[0003] With the widespread application of MCM-GPU in parallel computing and high-performance computing, how to effectively manage virtual memory has become the key to managing GPU storage space. In traditional memory access, when L1 TL misses, the virtual address needs to access the remote L2 TLB through the inter-chip interconnect network, resulting in a large address translation delay.

[0004] In the prior art, the virtual memory management architecture in the MCM-GPU system consists of multiple GPU chips and a CPU, and the conversion of virtual to physical addresses is managed collaboratively by the IOMMU (input and output memory management unit) in the CPU. Inside each GPU chip, there are multiple computing units (CUs), each of which is connected to a private L1 TLB. The L1 TLB is used to cache the most recently used virtual to physical address mappings. When the CU needs to access the memory, the L1 TLB can directly provide the physical address, thereby reducing access latency. When the L1 TLB fails to hit, the system will search in the globally shared L2 TLB according to the virtual address. The L2TLB is shared by multiple GPU chips, so the request can be routed to a specific GPU with the relevant mapping for conversion operations, further improving the hit rate of address conversion and the resource utilization of the system. When the address conversion fails to hit in the TLB in the GPU, the virtual address request will be sent to the IOMMU in the CPU. The IOMMU contains multiple page table walkers (PageTable Walker), which can search the physical address level by level from the memory to complete the conversion of the virtual to physical address.

[0005] In MCM-GPU, if a CU in one of the GPU modules has an L1 TLB failure, since the L2 TLB is shared, the virtual-to-real address conversion request of the L1 TLB failure will be sent to the L2 TLB of a GPU module according to the corresponding address mapping algorithm. Compared with a single GPU, the inter-chip interconnect network in MCM-GPU has higher latency and limited bandwidth, so the request to access the remote L2TLB has extremely high latency, which will significantly affect the overall performance of the MCM-GPU and significantly slow down the system operation.

[0006] In order to solve the above problems, the prior art MCM-GPU sets a corresponding L2 TLB for each GPU. Compared with the L2 TLB shared by multiple GPUs, each GPU private L2 TLB shows different advantages and limitations in performance, access latency and resource utilization. Each GPU does not need to make cross-chip requests when accessing its own L2 TLB. Direct access to the local L2 TLB can significantly reduce access latency, which is particularly beneficial for applications that require frequent virtual address conversion. However, since the L2TLBs of each GPU work independently, it is inevitable that the L2 TLBs of different GPUs will store the same virtual address mapping multiple times, resulting in a waste of memory resources, especially in the scenario of high-frequency cross-GPU data sharing, the overhead of this repeated storage is particularly obvious. In addition, the private L2 TLB lacks centralized management capabilities, and different GPUs cannot share address conversion results. In the process of data interaction and collaborative processing between different GPUs, the utilization efficiency of the private L2 TLB is low, which will increase the load of the IOMMU, resulting in a decrease in the virtual-to-real address conversion efficiency of the overall system. In contrast, the aforementioned L2TLB shared by multiple GPUs can effectively optimize resource utilization. Through a shared mechanism, the virtual address mapping table is centrally stored and managed, which avoids repeated storage and update operations of the same address in multiple L2 TLBs, significantly improving resource utilization. This centralized management method also has the convenience of cross-GPU access, which can improve the overall hit rate of L2 TLB, reduce the number of L1TLB misses, and reduce the burden on IOMMU, thereby improving the virtual memory management efficiency of the entire system.

[0007] These two methods have their own advantages and disadvantages. The first method can make full use of resources but is prone to access delays. The second method has low conversion efficiency and is prone to waste of resources but has fast access.

[0008] Therefore, how to provide a virtual-to-real address conversion and search method that not only has high resource utilization but also can quickly convert virtual-to-real addresses efficiently has become a problem that needs to be solved urgently. Summary of the invention

[0009] The present invention provides a virtual-to-real address conversion and search method for MCM-GPU, which is used to solve the problem in the prior art that the MCM-GPU can quickly perform virtual-to-real address conversion while reducing resource waste.

[0010] In order to achieve the above-mentioned purpose, the technical solution of the present invention provides a virtual-to-real address conversion search method for MCM-GPU, including: each CU in the GPU corresponds to its own round-robin arbiter, the round-robin arbiter receives a virtual-to-real address conversion request within a clock cycle, and performs virtual-to-real address conversion in the CU in the same GPU according to the type of the virtual-to-real address conversion request, or searches the L2 TLB of other GPUs for virtual-to-real address conversion.

[0011] As a preferred embodiment of the above technical solution, preferably, the round-robin arbiter simultaneously receives different types of virtual-to-real address conversion requests sent simultaneously by the cross switch and the load storage unit, and sends the corresponding virtual-to-real address conversion request to the L1 TLB of its corresponding CU according to the clock cycle.

[0012] As a preferred embodiment of the above technical solution, preferably, the round-robin arbiter receives a virtual-to-real address conversion request within a clock cycle, including: the round-robin arbiter simultaneously receives a local virtual-to-real address conversion request sent by its corresponding load storage unit and a remote virtual-to-real address conversion request from other CUs sent by a cross switch in the GPU within the clock cycle.

[0013] As a preferred embodiment of the above technical solution, preferably, the clock cycle includes a local clock cycle and a remote clock cycle, and the two clock cycles rotate alternately; if the current cycle is a local clock cycle, the local virtual-to-real address conversion request sent by the loading and storage unit is given priority; if the current cycle is a remote clock cycle, the remote virtual-to-real address conversion request sent by the cross switch from other CUs in this GPU is given priority.

[0014] As a preferred embodiment of the above technical solution, preferably, when the request received by the L1 TLB in the CU is the local virtual-to-real address conversion request: if the request is successful, the local virtual-to-real address conversion request is returned to the load storage unit for memory access operation; if the request fails, the idle CU is queried and the local virtual-to-real request is forwarded to the round-robin arbiter of the idle CU for request.

[0015] As a preferred embodiment of the above technical solution, preferably, an idle CU is queried, and the local virtual-to-real address request is forwarded to the round-robin arbiter of the idle CU for request, including that the L1 TLB that fails to access forwards the local virtual-to-real address conversion request to the cross network switch in the current GPU; the routing arbiter queries the idle CU in the current GPU, and if found, the local virtual-to-real address conversion request is sent to the idle CU.

[0016] As a preferred embodiment of the above technical solution, preferably, when the request received by the L1 TLB in the CU is the remote virtual-to-real address conversion request: if the request is successful, the request result is fed back to the CU that initiated the remote virtual-to-real address conversion request through the cross switch; if the request fails, the remote virtual-to-real address conversion request is sent to the inter-chip interconnection network through the cross switch to find the L2 TLB of other GUPs for virtual-to-real address conversion.

[0017] As a preferred embodiment of the above technical solution, preferably, the request is successful, which means that the crossbar network switch traverses the L1 TLBs of all CUs in the current GPU until the current remote virtual-to-real address conversion request is successful.

[0018] As a preferred embodiment of the above technical solution, preferably, the request failure means that after traversing the L1TLBs of all CUs in the current GPU, no L1 TLB can accept the remote virtual-to-real address conversion request.

[0019] The technical solution of the present invention provides a virtual-to-real address conversion search method for MCM-GPU, comprising: each CU in the GPU corresponds to its own round-robin arbiter, the round-robin arbiter receives a virtual-to-real address conversion request within a clock cycle, performs virtual-to-real address conversion in the CU in the same GPU according to the type of the virtual-to-real address conversion request, or searches for the L2 TLB of other GPUs to perform virtual-to-real address conversion.

[0020] The advantage of the present invention is that it provides a TLB detection mechanism, which can access the private L1 TLB of the adjacent CU in the same GPU to see if there is a hit in the virtual-to-real address conversion when the local L1 TLB fails. If there is a hit, the real memory page number is returned. If the private L1 TLB of the adjacent CU does not hit, the virtual-to-real address conversion request is sent to the inter-chip interconnection network, and the corresponding shared L2 TLB is accessed through the inter-chip interconnection network. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, a brief introduction is given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0022] Figure 1 A flowchart of a virtual-to-real address conversion lookup method for MCM-GPU provided by the present invention.

[0023] Figure 2A system architecture diagram of a virtual-to-real address conversion lookup method for MCM-GPU provided by the present invention.

[0024] Figure 3 A schematic diagram of the architecture of a round-robin arbiter corresponding to each CU in a virtual-to-real address conversion lookup method for MCM-GPU provided by the present invention.

[0025] Figure 4 A schematic diagram of searching for idle CU units in a virtual-to-real address conversion search method for MCM-GPU provided by the present invention. DETAILED DESCRIPTION

[0026] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0027] First, the process of the technical solution of the present invention is briefly described. Figure 1 As shown, Figure 1 A schematic diagram of a process provided for an embodiment of the present invention: Step 101: The round-robin arbiter simultaneously receives different types of virtual-to-real address conversion requests sent by the crossbar switch and the load-storage unit, and the types of the virtual-to-real address conversion requests enter different processing flows.

[0028] Specifically, each CU in each GPU corresponds to its own round-robin arbiter, which continuously receives local virtual-to-real address conversion requests sent by its corresponding load storage unit and remote virtual-to-real address conversion requests from other CUs sent by the cross switch in the GPU within a clock cycle.

[0029] Specifically, the clock cycle includes a local clock cycle and a remote clock cycle. Normally, the arbiter forwards the virtual address conversion request corresponding to the corresponding clock cycle in different clock cycles. However, if there is no corresponding virtual-to-real address conversion request in the current cycle and there is another type of virtual address conversion request, the other type of virtual address conversion request is processed.

[0030] For local virtual-to-real address translation requests: Step 102: Check whether the L1 TLB of the current CU can process the request. If yes, execute step 103; otherwise, execute step 104.

[0031] Step 103: Return the local virtual-to-real address conversion request to the load storage unit LSU for memory access operation.

[0032] Step 104: Query and send idle CUs.

[0033] Specifically, the local virtual-to-real request is forwarded to the round-robin arbiter of the idle CU for request, including the routing arbiter querying the idle CU in the current GPU. If found, the local virtual-to-real address conversion request is sent to the idle CU. The L1 TLB that cannot process the local virtual-to-real address conversion forwards the local virtual-to-real address conversion request to the cross-network switch in the current GPU, and then to the round-robin arbiter of the idle CU via the cross-network switch.

[0034] For remote virtual-to-real address translation requests: Step 105 : Check whether the L1 TLB of the current CU can process the request. If yes, execute step 106 ; otherwise, execute step 107 .

[0035] The crossbar network switch traverses the L1 TLBs of all CUs in the current GPU until one L1 TLB receives the remote virtual-to-real address translation request. The current remote virtual-to-real address translation request is successful, and step 106 is executed. If no L1 TLB can accept the remote virtual-to-real address translation request after traversing the L1 TLBs of all CUs in the current GPU, the request fails and step 107 is executed.

[0036] Step 106: The crossbar switch feeds back a result.

[0037] Specifically, the crossbar switch feeds back the request result to the CU that initiated the remote virtual-to-real address conversion request.

[0038] Step 107: Find the L2 TLB of other GPUs to perform virtual-to-real address conversion.

[0039] Specifically, the crossbar switch sends a remote virtual-to-real address translation request to the inter-chip interconnection network to find the L2TLB of other GUPs to perform virtual-to-real address translation.

[0040] Now, a specific embodiment is used to describe the technical solution of the present invention. Figure 2 The architecture shown is as follows: Figure 3 The structure shown is a schematic diagram of the architecture of a round-robin arbiter corresponding to a CU unit receiving virtual and real address requests. In actual electronic components, each CU in a GPU corresponds to the following Figure 3 In other words, there are multiple parallel GPUs in the GPU. Figure 3 The architecture shown.

[0041] Specifically, the round-robin arbiter receives the remote virtual-to-real address translation request queue forwarded by the crossbar network switch and the local virtual-to-real address translation request queue received from the load storage unit LSU, and sends them to the corresponding L1 TLB in a round-robin manner according to different clock cycles. The local virtual-to-real address translation request refers to the virtual-to-real address translation request sent by the CU unit corresponding to the current private L1 TLB via the LSU; the remote virtual-to-real address translation request refers to the virtual-to-real address translation request sent by other CU units in the same GPU as the aforementioned CU unit.

[0042] Among them, the clock cycle is two alternating clock cycles, including local clock cycles and remote clock cycles. If the current cycle is a local clock cycle, the local virtual-to-real address conversion request sent by the load storage unit LSU is given priority; if the current cycle is a remote clock cycle, the remote virtual-to-real address conversion request sent by the cross switch from other CUs in this GPU is given priority. Among them, if there is no local virtual-to-real conversion request in the local clock cycle, but there is a remote virtual-to-real address conversion request, the round-robin arbiter can also forward and process the remote virtual-to-real conversion request in the local clock cycle; if there is no remote virtual-to-real conversion request in the remote clock cycle, but there is a local virtual-to-real address conversion request, the round-robin arbiter can also forward and process the local virtual-to-real conversion request in the remote clock cycle, so as to make full use of resources.

[0043] The above is a round-robin mechanism for realizing the utilization rate increase of the private L1 TLB of each CU in each GPU in the present invention. The virtual-to-real address translation request sent by other CUs is inserted into the remote virtual-to-real address translation request queue after passing through the crossbar network switch. The round-robin arbiter selects a virtual-to-real address translation request from the local virtual-to-real address translation request queue and the remote virtual-to-real address translation request queue in a round-robin manner to access the L1 TLB.

[0044] If the local virtual-to-real address translation request issued by the current CU hits the private L1 TLB of the current CU (the local virtual-to-real address request is successful), it will directly return to its corresponding LSU for memory access operation; if this L1 TLB fails, which is equivalent to the failure of this local virtual-to-real address translation request, then this request will return to the cross-network switch to access the L1 TLB of other CUs.

[0045] Specifically, when the L1 TLB of a CU fails, the idle CU is queried and the local virtual request is forwarded to the idle CU. Figure 2As shown, for example: when a local virtual-to-real address translation request is sent to the L1 TLB of CU1 but the L1 TLB invalidation request fails, the L1 TLB sends the local virtual-to-real address translation request to the crossbar network switch, which routes the local virtual-to-real address translation request to a CU in the GPU to access its private L1 TLB.

[0046] At this time, the routing arbitrator Switch allocator in the GPU queries the idle CU (output port) according to the request received by the current router. For example, if the output port of CU0 is idle, the routing arbitrator will send the virtual-to-real address conversion request to CU0 to access its L1TLB. If successful, CU0 will receive this local virtual-to-real address conversion request as a remote virtual-to-real address conversion request, and CU0's L1 TLB will process this request, and the processing result will be fed back to CU0 through the cross network switch. If the L1 TLBs of all CUs in the current GPU do not receive this request, it is considered that this virtual-to-real address conversion request is invalid in the L1TLB, and it is forwarded to the local L2 TLB. If it is also invalid, it is sent to the inter-chip interconnection network through the crossbar network switch, routed to the remote L2 TLB of other GPUs for access and address conversion, and then fed back to the CU that initially sent this virtual-to-real address conversion request (CU1 in this embodiment) after success.

[0047] Finally, it should be noted that in the present invention, for CU, the local virtual-to-real address translation request refers to the request sent by the CU in the current GPU, and the remote virtual-to-real address translation request refers to the request sent by other CUs in the same GPU; for L2TLB, the remote L2 TLB is the L2 TLB of other different GPUs.

[0048] For the above-mentioned query of idle CU, if the request can access the L1 TLB of any adjacent CU, it is very easy to cause conflicts between different CUs. For example, when the L1 TLBs of multiple CUs fail and the L1 TLBs of the same CU are accessed at the same time, different virtual-to-real address conversion requests will compete for the same output port of the Crossbar, resulting in conflicts. In order to solve this problem, the present invention further proposes the above-mentioned idle port access mechanism architecture as follows: Figure 4 shown.

[0049] Based on the traditional MCM-GPU, the present invention uses the crossbar network switch interconnection network between the L1 TLB and L2 TLB of different CUs to complete the access to the L1 TLB of the CU in the same GPU module. When a certain L1 TLB of the virtual-to-real address conversion request fails, the virtual-to-real address conversion request is sent to the crossbar network switch interconnection network to access the L1 TLB of the adjacent CU. If the adjacent L1 TLB hits, it returns to the current CU through the Crossbar interconnection network to complete the virtual-to-real address conversion. If all L1 TLBs do not hit, the virtual-to-real address conversion request is sent to the inter-chip interconnection network through the Crossbar interconnection network to access the L2TLB of the corresponding GPU module to complete the virtual-to-real address conversion.

[0050] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A virtual-to-real address conversion lookup method for MCM-GPU, characterized in that: The method comprises: Each CU in the GPU has its own corresponding round-robin arbiter. The round-robin arbiter receives a virtual-to-real address conversion request in a clock cycle, and performs a virtual-to-real address conversion in a CU in the same GPU according to the type of the virtual-to-real address conversion request, or searches for the L2 TLB of another GPU to perform a virtual-to-real address conversion. Among them, CU is the Compute Unit computing unit, and L2 TLB is the Level2 Translation Lookaside Buffer secondary address translation cache.

2. The method according to claim 1, characterized in that The round-robin arbiter simultaneously receives different types of virtual-to-real address conversion requests sent simultaneously by the cross switch and the load-storage unit, and sends the corresponding virtual-to-real address conversion request to the L1 TLB of its corresponding CU according to the clock cycle; wherein the L1 TLB is a Level1 Translation Lookaside Buffer first-level address translation high-speed cache.

3. The method according to claim 1, characterized in that The round-robin arbiter receives a virtual-to-real address conversion request within a clock cycle, including: The round-robin arbiter simultaneously receives a local virtual-to-real address conversion request sent by its corresponding load storage unit and a remote virtual-to-real address conversion request from other CUs sent by a cross switch in the GPU within a clock cycle.

4. The method according to claim 3, characterized in that The clock cycle includes a local clock cycle and a remote clock cycle, and the two clock cycles rotate alternately; If the current cycle is a local clock cycle, the local virtual-to-real address conversion request sent by the load storage unit is processed first; if the current cycle is a remote clock cycle, the remote virtual-to-real address conversion request sent by the cross switch from other CUs in this GPU is processed first.

5. The method according to claim 3, characterized in that: When the request received by the L1 TLB in the CU is the local virtual-to-real address translation request: if the request is successful, the local virtual-to-real address translation request is returned to the load storage unit for memory access operation; If the request fails, an idle CU is queried, and the local virtual-real request is forwarded to the round-robin arbiter of the idle CU for requesting; wherein L1 TLB is a Level1 Translation Lookaside Buffer first-level address translation cache.

6. The method according to claim 5, characterized in that The querying of the idle CU and forwarding the local virtual-real request to the round-robin arbiter of the idle CU for requesting includes: The L1 TLB that fails to access forwards the local virtual-to-real address translation request to the crossbar network switch in the current GPU; The routing arbiter queries the idle CU in the current GPU, and if found, sends the local virtual-to-real address conversion request to the idle CU.

7. The method according to claim 3, characterized in that When the request received by the L1 TLB in the CU is the remote virtual-to-real address conversion request: if the request is successful, the request result is fed back to the CU that initiated the remote virtual-to-real address conversion request through the cross switch; if the request fails, the remote virtual-to-real address conversion request is sent to the inter-chip interconnection network through the cross switch to find the L2 TLB of other GUPs for virtual-to-real address conversion; wherein, L1 TLB is a Level1 TranslationLookaside Buffer first-level address translation cache.

8. The method according to claim 7, characterized in that The request is successful, which means that the crossbar network switch traverses the L1 TLBs of all CUs in the current GPU until the current remote virtual-to-real address conversion request is successful.

9. The method according to claim 7, characterized in that: The request failure means that after traversing the L1 TLBs of all CUs in the current GPU, no L1 TLB can accept the remote virtual-to-real address conversion request.

Images