Virtual memory architecture design method and system for MCM GPU, medium and equipment

Abstract

The invention provides a virtual memory architecture design method and system for an MCM GPU, a medium and equipment, and the method comprises the following steps: receiving and recognizing a virtual address conversion request, and orienting the virtual address conversion request to a corresponding first-stage conversion buffer region based on a recognition result of a virtual address so as to query whether a corresponding page table item exists or not; if yes, address translation operation is executed; if the virtual address translation request does not exist, querying the virtual address translation request by using a hardware lookup table constructed by a uckoo filter, and orienting the virtual address translation request to a corresponding secondary translation buffer area based on a hardware query result so as to query whether a corresponding page table item exists or not; if yes, address translation operation is executed; and if not, executing a page table traversal operation on the virtual address translation request until the corresponding page table item is obtained and the address translation operation is completed. By optimizing data access and cache utilization between chips and improving L2TLB management, the overall performance and efficiency of the GPU are improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of computer hardware and storage technology, and in particular to a virtual memory architecture design method, system, medium and device for MCM GPU. Background Art

[0002] The evolution of GPU architecture from a single GPU to a multi-GPU system, and then to the latest MCM GPU (Multi-chip-Module multi-chip modular GPU) is to cope with the slowdown in transistor scaling and the huge demand for computing power in the era of artificial intelligence. MCM achieves higher performance and larger memory capacity by packaging multiple small chips (Chiplet) together. Chiplet is a technology that packages multiple bare chips that meet specific functions together through die-to-die internal interconnect technology. MCM GPUs form a logically unified "monolithic" GPU by stitching multiple chiplets together, but the physical resources are distributed across multiple chiplets. MCM GPUs bring about the problem of non-uniform memory access (NUMA) due to accessing resources across chiplets. Remote memory access is slower than accessing data residing in the memory of its own chiplet, which affects performance.

[0003] like Figure 1 As shown, Figure 1 The figure shows the general translation process of virtual addresses in a single GPU. In an MCM GPU, each CU (Compute Unit) contains a private first-level translation buffer (L1 TLB), while all CUs share a second-level translation buffer (L2 TLB). The address translation request first queries the L1 TLB, and if it fails, it queries the L2 TLB, and finally performs page table translation through the page walker.

[0004] like Figure 2 As shown, Figure 2 Shown is the typical architecture of an MCM GPU. An MCM GPU consists of multiple chiplets, each of which contains the typical compute units and memory system of a monolithic GPU, but in smaller quantities. Each chiplet contains multiple CUs, each of which consists of multiple SIMD units. The SIMD units in each CU share the L1 cache and L1 TLB. Each chiplet has a L2 cache that is shared by all CUs in the chiplet. Each chiplet is also connected to an HBM or GDDR.

[0005] Currently, some applications are limited by L2 TLB capacity, while others are limited by page table accesses. Some applications perform better with a private TLB, as local L2 TLB hits are faster. Other applications perform better with a shared L2 TLB design because the total L2 TLB capacity is larger and the L2 TLB miss rate is lower. In addition, if an application does not speed up using a shared TLB design over a private TLB, it does not necessarily mean that the application does not benefit from the aggregated TLB capacity. It may be that the data pages and thread block (CTA) locations used by the program are not on the same chiplet. In regular GPU applications, different CTAs typically compute different parts of a data set. If the CTAs and the data they access are mapped to the same chiplet, then the L2 TLB capacity of the chiplet can be well utilized even with a private L2 TLB.

[0006] Based on this, how to meet the challenges of modern technology requirements through multi-chip modular design has become a technical problem that technical personnel in this field urgently need to solve. Summary of the invention

[0007] The object of the present invention is to provide a virtual memory architecture design method, system, medium and device for MCM GPU, which can improve the overall performance and efficiency of GPU by optimizing data access and cache utilization between chiplets and improving the management of L2 TLB.

[0008] In a first aspect, the present invention provides a virtual memory architecture design method for an MCM GPU, the method comprising the following steps: receiving a virtual address translation request and identifying it to obtain a virtual address identification result; based on the virtual address identification result, directing the virtual address translation request to a corresponding L1 TLB to query whether a corresponding page table entry exists; if so, performing an address translation operation; if not, querying the virtual address translation request using a hardware lookup table constructed by a cuckoo filter to obtain a hardware query result; based on the hardware query result, directing the virtual address translation request to a corresponding L2 TLB to query whether a corresponding page table entry exists; if so, performing an address translation operation; if not, performing a page table traversal operation on the virtual address translation request until the corresponding page table entry is obtained to complete the address translation operation.

[0009] In this implementation, by intelligently mapping data and corresponding page table entries (PTEs) to the same chip or adjacent chips, the need for cross-chip data access is reduced, thereby reducing data access latency. The L2TLB caches on multiple chiplets are logically aggregated to form a larger virtual cache, which can cache more PTEs and reduce the TLB miss rate. At the same time, the management method of L2 TLB is improved to reduce the dependence on remote L2 TLB. The performance and efficiency of MCM GPUs in processing complex tasks are significantly improved.

[0010] In an implementation of the first aspect, the method further includes L1 TLB grouping design: grouping a preset number of adjacent computing units into one group, and allocating different address translation request ranges to the L1 TLBs in the computing units, and distinguishing them by corresponding identification numbers.

[0011] In this implementation, the design of grouped TLB reduces duplication between different CU L1 TLBs, improves the TLB hit rate by expanding the TLB capacity available to each CU, and significantly improves the performance and efficiency of the GPU.

[0012] In an implementation of the first aspect, the method further includes an address space-aware buffer replacement control strategy: adding a field to the page table entry structure of the L1 TLB to distinguish between local page table entries and remote page table entries; dynamically adjusting the cache mode of the L1 TLB by monitoring the ratio of accesses to the local page table entries and the remote page table entries and comparing it with a preset threshold.

[0013] In this implementation, the address space-aware buffer replacement control strategy not only further improves the LITLB hit rate, cache efficiency and system responsiveness, but also reduces memory access latency and cross-chip communication overhead, thereby significantly improving the performance of GPU virtual memory management.

[0014] In an implementation of the first aspect, the method also includes a data page allocation mechanism based on first access page migration: when a thread block first accesses a specific data page, the data page is allocated to the dynamic random access memory inside the chiplet where the corresponding thread block is located, and the page table entry corresponding to the data page is also migrated to the page table storage structure inside the chiplet where the corresponding thread block is located.

[0015] In this implementation, by introducing a data page allocation mechanism for first access page migration, the data transmission requirements across chiplets when subsequently accessing the data page are reduced, thereby reducing the communication overhead between chiplets and improving the locality of data access, which helps to reduce memory access latency and improve cache hit rates.

[0016] In an implementation of the first aspect, by monitoring the ratio of accesses to the local page table entries and the remote page table entries and comparing with a preset threshold, dynamically adjusting the cache mode of the L1 TLB includes the following steps: when the access ratio of the local page table entries in the L1 TLB is greater than a first threshold, increasing the capacity of the local page table entries in the L1 TLB; when the access ratio of the local page table entries in the L1 TLB is less than a second threshold, increasing the capacity of the remote page table entries in the L1 TLB; when the L1 TLB needs to replace a page table entry, giving priority to replacing a page table entry type that exceeds the preset threshold.

[0017] In this implementation, by dynamically adjusting the cache capacity according to the access ratio of local page table entries and remote page table entries, TLB resources can be used more effectively and the cache strategy can be optimized.

[0018] In an implementation of the first aspect, based on the hardware query result, the virtual address translation request is directed to the corresponding L2 TLB to query whether there is a corresponding page table entry; if so, performing the address translation operation includes the following steps: querying the virtual address translation request through the hardware lookup table constructed by the cuckoo filter, and determining the type of the virtual address translation request; if the virtual address translation request belongs to a local page table entry, forwarding the virtual address translation request to the local L2 TLB to implement the address translation operation; if the virtual address translation request belongs to a remote page table entry, forwarding the virtual address translation request to the remote L2 TLB through a remote transmission unit to implement the address translation operation.

[0019] In this implementation, by using cuckoo filters at the L2 TLB level to query and process address translation requests, not only the efficiency of address translation and cache hit rate are improved, but also the cross-chip communication overhead and memory access latency are reduced, thereby significantly improving the performance of GPU virtual memory management and the overall efficiency of the system.

[0020] In an implementation of the first aspect, the method further includes a buffer consistency maintenance mechanism: when any of the L2TLBs updates a page table entry, an update message is broadcast to ensure that a hardware lookup table constructed by the cuckoo filter is synchronously updated or replaced.

[0021] In this implementation, the buffer consistency maintenance mechanism not only maintains cache consistency but also improves data access accuracy and system performance, while enhancing system reliability by ensuring synchronous update of page table entries in the TLB.

[0022] In a second aspect, the present invention provides a virtual memory architecture design system for an MCM GPU, the system comprising:

[0023] A receiving module, used for receiving and identifying a virtual address conversion request, and obtaining an identification result of the virtual address;

[0024] A first-level module, for directing the virtual address translation request to the corresponding L1 TLB based on the recognition result of the virtual address to query whether a corresponding page table entry exists; if so, performing an address translation operation;

[0025] A secondary module is used for querying the virtual address translation request using a hardware lookup table constructed by a cuckoo filter to obtain a hardware query result if there is no corresponding page table entry in the L1 TLB; based on the hardware query result, directing the virtual address translation request to the corresponding L2 TLB to query whether there is a corresponding page table entry; if so, performing an address translation operation;

[0026] The traversal module is used to perform a page table traversal operation on the virtual address translation request until the corresponding page table entry is obtained to complete the address translation operation if the L2 TLB does not have a corresponding page table entry.

[0027] In a third aspect, the present invention provides an electronic device, the electronic device comprising: a processor and a memory;

[0028] The memory is used to store computer programs;

[0029] The processor is used to execute the computer program stored in the memory, so that the electronic device executes the above-mentioned virtual memory architecture design method for MCM GPU.

[0030] In a fourth aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by an electronic device, implements the above-mentioned virtual memory architecture design method for MCM GPU. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 Shown is a diagram of the general conversion process of virtual addresses in a single GPU;

[0032] Figure 2 Shown is a typical architecture diagram of an MCM GPU;

[0033] Figure 3 Shown is a flow chart of a virtual memory architecture design method for an MCM GPU in one embodiment of the present invention;

[0034] Figure 4 It is a diagram showing a grouped L1TLB architecture of a virtual memory architecture design method for an MCM GPU in one embodiment of the present invention;

[0035] Figure 5 It is a diagram illustrating the modification of page table entries in an embodiment of the virtual memory architecture design method for MCM GPU of the present invention;

[0036] Figure 6 ART microarchitecture details shown as cuckoo filter based;

[0037] Figure 7 Shown is a flowchart of an address translation request based on ART in one embodiment of a virtual memory architecture design method for an MCM GPU of the present invention;

[0038] Figure 8 Shown is a virtual address conversion flow chart of a virtual memory architecture design method for MCM GPU in one embodiment of the present invention;

[0039] Fig. 9 Shown is a schematic diagram of the structure of an electronic device of the present invention in one embodiment;

[0040] Fig.10 FIG. 4 is a schematic diagram showing the structure of a virtual memory architecture design system for MCM GPU in one embodiment of the present invention. DETAILED DESCRIPTION

[0041] The following describes the embodiments of the present invention by specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the following embodiments and features in the embodiments can be combined with each other without conflict.

[0042] It should be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present invention, and thus the drawings only show components related to the present invention rather than being drawn according to the number, shape and size of components in actual implementation. In actual implementation, the type, quantity and proportion of each component may be changed arbitrarily, and the component layout may also be more complicated.

[0043] The technical solutions in the embodiments of the present invention will be described in detail below in conjunction with the accompanying drawings in the embodiments of the present invention.

[0044] like Figure 3 As shown, in one embodiment, the virtual memory architecture design method for MCM GPU of the present invention includes steps S11 to S14.

[0045] Step S11: Receive and identify a virtual address translation request and a corresponding address space, and obtain an identification result of the virtual address.

[0046] Specifically, in computer systems, especially hardware accelerators such as GPUs, programs usually use virtual addresses to access memory. These virtual addresses need to be converted into physical addresses so that the actual hardware of the CPU or GPU can access the correct memory location. When the GPU performs an operation that requires access to memory, it initiates a virtual address translation request. This request contains virtual address information, which is a logical representation of the memory location that the program is trying to access. In a multitasking operating system, each process has its own virtual address space, which is isolated to protect data between processes from being accessed incorrectly. The process of identifying the address space involves determining the process or task that initiated the virtual address translation request and mapping it to its corresponding virtual address space. This is accomplished through the collaborative work of the operating system and hardware.

[0047] Once a virtual address translation request is received and the address space to which the request belongs is determined, the system needs to identify the specific information of the virtual address, including the page number and offset within the page. The identification result usually includes the page number (Page Number) and the offset within the page (Offset) of the virtual address. The page number is used to look up in the page table, and the offset within the page is used to determine the specific location within the physical page.

[0048] Through this process, the system is prepared for the subsequent address translation operation, ensuring that the virtual address can be correctly and efficiently converted to the physical address.

[0049] Step S12: Based on the recognition result of the virtual address, direct the virtual address translation request to the corresponding L1 TLB to query whether a corresponding page table entry exists; if so, perform an address translation operation.

[0050] Further, such as Figure 4 As shown, the method further includes L1 TLB grouping architecture design: grouping a preset number of adjacent computing units into one group, and allocating different address translation request ranges to the L1 TLBs in the computing units, and distinguishing them by corresponding identification numbers.

[0051] Specifically, the L1 TLB is private to each compute unit (CU), but due to limited capacity, many applications have a low L1TLB hit rate. Adjacent CUs may access the same data page, resulting in duplicates between L1 TLBs, which indicates the existence of a sharing feature. The four physically close CUs and their L1 TLBs are grouped together to take advantage of this sharing feature. The four L1 TLBs in each group serve different address translation request ranges, which are distinguished by the lower two bits of the virtual page number. As shown in Table 1, the address translation request range served by each L1 TLB is different, which can evenly distribute the address range within the group, reduce duplicates, and improve the hit rate.

[0052] L1 TLB number within the group Address range for address translation requests The lower two digits of the corresponding virtual page number 0 4a 00 1 4a+1 01 2 4a+2 10 3 4a+3 11

[0053] Table 1 L1 TLB address division within a group

[0054] Furthermore, the method also includes an address space-aware buffer replacement control strategy: adding a field to the page table entry structure of the L1 TLB to distinguish between local page table entries and remote page table entries; dynamically adjusting the cache mode of the L1 TLB by monitoring the ratio of access to the local page table entries and the remote page table entries and comparing it with a preset threshold.

[0055] Specifically, by dynamically monitoring the ratio of accesses to local page table entries and remote page table entries, the replacement strategy of L1 TLB is optimized to further improve the hit rate. Figure 5 As shown in the figure, the page table entry structure in the L1 TLB is modified, and a bit (Local field) is added to distinguish between local pages and remote pages. The ratio of access to local pages and remote pages (L / T) is monitored and compared with the preset thresholds (T1, T2). According to the access ratio, the capacity of local and remote page table entries in the L1 TLB is dynamically adjusted. When the number of remote page table entries in the L1 TLB exceeds the set value, the remote page table entries are replaced first; otherwise, the local page table entries are replaced first.

[0056] This strategy reduces the probability of corresponding type page table entries being replaced, and improves the hit rate of L1 TLB based on the principles of temporal locality and spatial locality.

[0057] In one embodiment, during initial allocation, ways in the L1 TLB are evenly allocated to local pages and remote pages. Assuming that the L1 TLB has a total of N ways, then initially, local pages and remote pages are each allocated to N / 2 ways. The system monitors and records the total number of address translation requests T and the number of local page address translation requests L in a statistical cycle. The ratio of local page address translation requests to the total number of address translations L / T is calculated.

[0058] Set the first threshold T1 and the second threshold T2, which are used to determine when to adjust the allocation of ways. The values of T1 and T2 need to be determined according to the specific requirements and performance goals of the system.

[0059] If L / T < T2: This means that the access frequency of local pages is low, and the importance of local pages decreases relative to remote pages. Therefore, the algorithm will reduce the number of ways allocated to local pages (local page way--), while increasing the number of ways allocated to remote pages (remote page way++).

[0060] If L / T > T1: This means that the access frequency of local pages is high, and the importance of local pages increases relative to remote pages. Therefore, the algorithm will increase the number of ways allocated to local pages (local page way++), while reducing the number of ways allocated to remote pages (remote page way--).

[0061] If T1 ≤ L / T ≤ T2: This means that the access frequencies of local and remote pages are in a balanced state without obvious preference. Therefore, the algorithm keeps the current cache line allocation unchanged.

[0062] The above address space-aware L1 TLB allocation algorithm aims to optimize the hit rate of the L1 TLB by dynamically adjusting the allocation of ways. When more frequently accessed page table entries are retained in the TLB, future address translation requests are more likely to directly find the corresponding mapping in the L1 TLB, thus reducing the need to access the slower L2 TLB or main memory. During the execution of a program, its memory access pattern may change. This algorithm can dynamically adjust the TLB allocation according to the change in the access pattern to maintain the best performance. Through this dynamic adjustment strategy, the system can better adapt to different workloads and access patterns, thereby improving the overall memory access efficiency.

[0063] In summary, through the grouped L1 TLB architecture and the address space-aware TLB replacement control strategy, duplicates can be reduced and the replacement strategy can be optimized, resulting in an increase in the hit rate of the L1 TLB. An increase in the L1 TLB hit rate means that fewer requests are forwarded to the L2 TLB, thus reducing the burden on the L2 TLB. By making better use of the L1 TLB, the memory access efficiency of the GPU is improved, thereby enhancing the overall performance.

[0064] Furthermore, the method also includes a data page allocation mechanism based on first access page migration: when a thread block first accesses a specific data page, the data page is allocated to the dynamic random access memory inside the chiplet where the corresponding thread block is located, and the page table entry corresponding to the data page is also migrated to the page table storage structure inside the chiplet where the corresponding thread block is located.

[0065] Specifically, when a data page is accessed for the first time, it is allocated to the DRAM of the chiplet where the thread block (CTA) that initiated the access is located. This strategy is based on the computing characteristics of the GPU, that is, when a data page is accessed for the first time by a CTA, it is likely that the CTA is mainly responsible for the calculation. Through this mechanism, data communication between chiplets is reduced because each chiplet mainly accesses its own DRAM.

[0066] Step S13: If it does not exist, query the virtual address translation request using the hardware lookup table constructed by the cuckoo filter to obtain the hardware query result; based on the hardware query result, direct the virtual address translation request to the corresponding secondary translation buffer to query whether there is a corresponding page table entry; if it exists, perform the address translation operation.

[0067] Specifically, when a virtual address translation request occurs, it is first queried through the cuckoo filter. If the cuckoo filter indicates that the request may exist, it provides a hardware query result, which tells the GPU which secondary translation buffer should be queried. Based on the hardware query result of the cuckoo filter, the virtual address translation request is directed to the corresponding secondary translation buffer. Each chiplet has its own secondary translation buffer for storing local page table entries. Query whether there is a corresponding page table entry in the specified secondary translation buffer. If the page table entry exists, the address translation operation can be performed directly to convert the virtual address into a physical address.

[0068] like Figure 6 As shown in Figure 1, cuckoo filter is an efficient data structure used to quickly check whether an element belongs to a set without storing the entire element. In multi-chiplet GPU systems, Cuckoo filter is used to support logical sharing of L2TLB.

[0069] The cuckoo filter consists of a series of buckets, each of which can store multiple fixed-length fingerprints. Each ART (Address Request Table) contains 128 buckets, each with 4 fingerprints, so theoretically 512 page table entries can be supported. The fingerprint is a short representation of the hash value of an element (in this scenario, the virtual page number VPN). VPN is converted into a fixed-length fingerprint through a hash function (such as MetroHash). The cuckoo filter uses two hash functions (h1 and h2) to generate two possible storage locations (alternative buckets) for each element. When a new element is inserted, its fingerprint is attempted to be inserted into one of these two locations.

[0070] Insertion operation: If both candidate buckets have space, the fingerprint is inserted into one of the buckets. If both buckets are full, the cuckoo filter will select a bucket, kick out an existing fingerprint, and reinsert the "victim" fingerprint into its candidate position. This process may cause a series of "kick-out" operations, called "Cuckooing".

[0071] Search operation: When searching, the cuckoo filter calculates the fingerprint of the element and the position of the two alternative buckets. If a matching fingerprint is found in any bucket, true is returned, indicating that the element exists; otherwise, false is returned.

[0072] Delete operation: The delete operation removes the matching fingerprint from any candidate bucket.

[0073] In one embodiment, when a virtual-to-real mapping is inserted into the L2 TLB of a chiplet, the fingerprint of the corresponding VPN will also be recorded in the ART. In this way, when it is necessary to check whether a virtual page number exists in the L2TLB of a chiplet, it can be quickly searched through the ART. When the virtual-to-real mapping in the L2 TLB is updated (inserted or deleted), the ART also needs to be updated accordingly to maintain data consistency. If a mapping is removed from the L2 TLB, its corresponding fingerprint must also be deleted from the ART. Since the ART contains eight comparators, all candidate fingerprints in the two buckets can be checked in parallel, which greatly improves the search efficiency.

[0074] In this way, cuckoo filters play a key role in the GPU's multi-chiplet architecture, enabling the L2 TLB between different chiplets to be logically shared while maintaining efficient address translation performance.

[0075] In one embodiment, when a miss occurs in the L1 TLB, the address translation request first queries the corresponding ART. If the ART indicates that the request exists in the L2 TLB of a chiplet, the request is forwarded to the corresponding L2 TLB. If it is a local chiplet, it is directly forwarded to the local L2 TLB; if it is a remote chiplet, it is forwarded through the RTU. When the L2 TLB is updated, the ART on all chiplets also needs to be updated to reflect the latest virtual-to-real mapping information.

[0076] like Figure 7 As shown, when the L1 TLB cannot find the corresponding page table entry (PTE), a miss occurs and the address translation request needs to be further processed. The address request is first sent to the corresponding ART (e.g., [ART]x, where x is the chiplet number). The ART records the L2 TLB information of each chiplet, including which chiplet is responsible for processing requests for a specific address range. Look up the address request in [ART]x. If found, determine whether the request is local or remote. If the request is local, forward it directly to the L2 TLB of the local chiplet. If the request is remote, forward the request to the remote chiplet through the remote transmission unit (RTU). Look up the address translation request in the L2 TLB of the local chiplet. If the L2 TLB hits, the address translation request is directly returned to the corresponding L1 TLB to complete the address translation. If a miss occurs in the local L2 TLB, a page walk operation is required to obtain the page table entry. While updating the local L2 TLB, the ART on all chiplets also needs to be updated to ensure that subsequent requests can be routed correctly. If the request is forwarded to the L2 TLB of the remote chiplet, the remote L2 TLB queries the address translation request. If the remote L2 TLB hits, the address translation response is returned to the original L1TLB through the RTU. If a miss occurs in the remote L2 TLB, the page table walker on the remote chiplet performs a page traversal operation. The page table entry is inserted into the remote L2 TLB to process the current request. At the same time, the remote L2 TLB needs to broadcast the L2 TLB update information to all chiplets to ensure that the ART information is up to date.

[0077] Whether it is a local or remote L2 TLB update, it is necessary to ensure that the ART information on all chiplets is up to date in order to correctly route future address requests. The key to this process is to ensure that address requests can be quickly and accurately routed to the correct L2 TLB, and find the corresponding page table entry in the L2 TLB to complete the address translation. By maintaining ART and updating it in a timely manner, the system can effectively handle memory access requests across chiplets and improve overall memory access efficiency.

[0078] In one embodiment, if Figure 8 As shown, the CU generates a virtual address request and first queries the virtual address in the CU's L1 TLB. If the L1 TLB fails, the chiplet number corresponding to the request is queried through the address request table, and then the request is sent to the L2 TLB of the corresponding chiplet; if the L2 TLB of the chiplet fails, the request is sent to the page walker, and the page walker searches the corresponding memory according to the information in the page table and updates the local page table cache. The obtained PTE is updated to the L2 TLB of the corresponding chiplet so that the mapping information can be found directly in the TLB in the future. The virtual address is converted to the physical address using the PTE, and the memory request is processed. During the virtual-to-real conversion process, if the page table entry is updated or inserted, the corresponding TLB and the ART (address request table) based on the cuckoo filter also need to be updated.

[0079] In this way, the MCM GPU can more efficiently utilize its aggregated L2 TLB capacity, reduce memory access latency across chiplets, and optimize overall memory access performance. This architecture is particularly suitable for applications that are sensitive to memory access latency and those that require a lot of memory bandwidth and cache.

[0080] Furthermore, the total cost of all ART hardware on an MCM GPU can be calculated using the following formula:

[0081] n=Num c ×Num s ×Num e ×Num b

[0082] Where n represents the number of bits, Num c Indicates the number of chiplets, Num s Indicates the number of lookup tables for each chiplet, Num e Indicates the number of page table entries in each lookup table, Num b The number of bits that represent the fingerprint corresponding to each page table entry.

[0083] The protection scope of the virtual memory architecture design method for MCM GPU described in the embodiment of the present invention is not limited to the execution order of the steps listed in this embodiment. All solutions implemented by adding, reducing, or replacing steps in the prior art based on the principles of the present invention are included in the protection scope of the present invention.

[0084] like Fig. 9 As shown, an embodiment of the present invention further provides an electronic device for designing a virtual memory architecture for an MCM GPU. The electronic device includes: at least one processor 901, a memory 902, at least one network interface 903, and a user interface 905. The various components in the device are coupled together through a bus system 904. It can be understood that the bus system 904 is used to achieve connection and communication between these components. In addition to the data bus, the bus system 904 also includes a power bus, a control bus, and a status signal bus. However, for the sake of clarity, Fig. 9 In the specification, various buses are labeled as bus systems.

[0085] The user interface 905 may include a display, a keyboard, a mouse, a trackball, a click gun, keys, buttons, a touch pad or a touch screen.

[0086] It is understood that the memory 902 can be a volatile memory or a non-volatile memory, and can also include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), which is used as an external cache. By way of example but not limitation, many forms of RAM are available, such as static random access memory (SRAM), synchronous static random access memory (SSRAM). The memory described in the embodiments of the present invention is intended to include but is not limited to these and any other suitable categories of memory.

[0087] The memory 902 in the embodiment of the present invention is used to store various categories of data to support the operation of the electronic terminal 900. Examples of these data include: any executable program for operating on the electronic terminal 900, such as an operating system 9021 and an application 9022; the operating system 9021 includes various system programs, such as a framework layer, a core library layer, a driver layer, etc., for implementing various basic services and processing hardware-based tasks. The application 9022 may include various applications, such as a media player (Media Player), a browser (Browser), etc., for implementing various application services. The virtual memory architecture design method for an MCM GPU provided in an embodiment of the present invention may be included in the application 9022.

[0088] The method disclosed in the above embodiment of the present invention can be applied to the processor 901, or implemented by the processor 901. The processor 901 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit in the processor 901 or the instruction in the form of software. The above processor 901 may be a general processor, a digital signal processor (DSP, Digital Signal Processor), or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The processor 901 can implement or execute the various methods, steps and logic block diagrams disclosed in the embodiment of the present invention. The general processor 901 may be a microprocessor or any conventional processor, etc. In combination with the steps of the accessory optimization method provided in the embodiment of the present invention, it can be directly embodied as a hardware decoding processor to execute, or it can be executed by a combination of hardware and software modules in the decoding processor. The software module can be located in a storage medium, which is located in a memory, and the processor reads the information in the memory and completes the steps of the above method in combination with its hardware.

[0089] In an exemplary embodiment, the electronic terminal 900 may be implemented by one or more application specific integrated circuits (ASIC), DSP, programmable logic device (PLD), complex programmable logic device (CPLD) to execute the aforementioned method.

[0090] An embodiment of the present invention further provides a virtual memory architecture design system for an MCM GPU. The virtual memory architecture design system for an MCM GPU can implement the virtual memory architecture design method for an MCM GPU described in the present invention. However, the implementation device of the virtual memory architecture design system for an MCM GPU described in the present invention includes but is not limited to the structure of the virtual memory architecture design system for an MCM GPU listed in this embodiment. All structural deformations and replacements of the prior art made according to the principles of the present invention are included in the protection scope of the present invention.

[0091] like Fig.10 As shown, in one embodiment, the virtual memory architecture design system for MCM GPU of the present invention includes:

[0092] The receiving module 101 is used to receive and identify a virtual address translation request, and obtain an identification result of the virtual address;

[0093] The first-level module 102 is connected to the receiving module 101 and is used to direct the virtual address translation request to the corresponding first-level translation buffer based on the recognition result of the virtual address to query whether there is a corresponding page table entry; if so, perform the address translation operation;

[0094] The secondary module 103 is connected to the primary module 102 and is used for querying the virtual address translation request using a hardware lookup table constructed by a cuckoo filter to obtain a hardware query result if the corresponding page table entry does not exist in the primary translation buffer; based on the hardware query result, directing the virtual address translation request to the corresponding secondary translation buffer to query whether the corresponding page table entry exists; if so, performing the address translation operation;

[0095] The traversal module 104 is connected to the secondary module 103 and is used to perform a page table traversal operation on the virtual address translation request until the corresponding page table entry is obtained to complete the address translation operation if there is no corresponding page table entry in the secondary translation buffer.

[0096] In the several embodiments provided by the present invention, it should be understood that the disclosed system, device or method can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of modules / units is only a logical function division, and there may be other division methods in actual implementation, such as multiple modules or units can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or modules or units, which can be electrical, mechanical or other forms.

[0097] The modules / units described as separate components may or may not be physically separated, and the components displayed as modules / units may or may not be physical modules, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules / units may be selected according to actual needs to achieve the purpose of the embodiments of the present invention. For example, the functional modules / units in the various embodiments of the present invention may be integrated into one processing module, or each module / unit may exist physically separately, or two or more modules / units may be integrated into one module / unit.

[0098] Those of ordinary skill in the art should further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0099] The embodiment of the present invention also provides a computer-readable storage medium. A person of ordinary skill in the art can understand that all or part of the steps in the method for implementing the above embodiment can be completed by instructing the processor through a program, and the program can be stored in a computer-readable storage medium, and the storage medium is a non-transitory medium, such as a random access memory, a read-only memory, a flash memory, a hard disk, a solid-state hard disk, a magnetic tape, a floppy disk, an optical disc, and any combination thereof. The above storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or a data center that includes one or more available media. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a digital video disc (DVD)), or a semiconductor medium (e.g., a solid-state disk (SSD)), etc.

Claims

1. A virtual memory architecture design method for MCM GPU, characterized in that: The method comprises the following steps: Receive and identify a virtual address translation request, and obtain an identification result of the virtual address; Based on the recognition result of the virtual address, direct the virtual address translation request to the corresponding first-level translation buffer to query whether a corresponding page table entry exists; if so, perform an address translation operation; If it does not exist, query the virtual address translation request using the hardware lookup table constructed by the cuckoo filter to obtain the hardware query result; Based on the hardware query result, directing the virtual address translation request to the corresponding secondary translation buffer to query whether a corresponding page table entry exists; if so, performing an address translation operation; If not present, a page table traversal operation is performed on the virtual address translation request until the corresponding page table entry is obtained to complete the address translation operation.

2. The method for designing a virtual memory architecture for an MCM GPU according to claim 1, wherein: The method also includes a first-level conversion buffer grouping design: grouping a preset number of adjacent computing units into one group, and allocating different address conversion request ranges to the first-level conversion buffers in the computing units, and distinguishing them by corresponding identification numbers.

3. The method for designing a virtual memory architecture for an MCM GPU according to claim 1, wherein: The method also includes an address space aware buffer replacement control strategy: Adding a field in the page table entry structure of the first-level translation buffer to distinguish between local page table entries and remote page table entries; The cache mode of the first-level translation buffer is dynamically adjusted by monitoring the ratio of access to the local page table entry and the remote page table entry and comparing the ratio with a preset threshold.

4. The method for designing a virtual memory architecture for an MCM GPU according to claim 1, wherein: The method also includes a data page allocation mechanism based on first access page migration: When a thread block accesses a specific data page for the first time, the data page is allocated to the dynamic random access memory inside the core where the corresponding thread block is located, and the page table entry corresponding to the data page is also migrated to the page table storage structure inside the core where the corresponding thread block is located.

5. The method for designing a virtual memory architecture for an MCM GPU according to claim 3, wherein: By monitoring the ratio of access to the local page table entry and the remote page table entry and comparing it with a preset threshold, dynamically adjusting the cache mode of the first-level translation buffer comprises the following steps: When the access ratio of the local page table entries in the first-level translation buffer is greater than a first threshold, increasing the capacity of the local page table entries in the first-level translation buffer; When the access ratio of the local page table entries in the first-level translation buffer is less than a second threshold, increasing the capacity of the remote page table entries in the first-level translation buffer; When the first-level translation buffer needs to replace page table entries, page table entry types exceeding the preset threshold are replaced first.

6. The method for designing a virtual memory architecture for an MCM GPU according to claim 3, wherein: Based on the hardware query result, directing the virtual address translation request to the corresponding secondary translation buffer to query whether a corresponding page table entry exists; If it exists, performing the address translation operation includes the following steps: Querying the virtual address translation request through a hardware lookup table constructed by the cuckoo filter, and determining the type of the virtual address translation request; If the virtual address translation request belongs to a local page table entry, forwarding the virtual address translation request to a local secondary translation buffer to implement an address translation operation; If the virtual address translation request belongs to a remote page table entry, the virtual address translation request is forwarded to a remote secondary translation buffer via a remote transmission unit to implement an address translation operation.

7. The method for designing a virtual memory architecture for an MCM GPU according to claim 1, wherein: The method also includes a buffer consistency maintenance mechanism: When any of the secondary translation buffers updates a page table entry, an update message is broadcast to ensure that the hardware lookup table constructed by the cuckoo filter is updated or replaced synchronously.

8. A virtual memory architecture design system for MCM GPU, characterized in that: The system comprises: A receiving module, used for receiving and identifying a virtual address conversion request, and obtaining an identification result of the virtual address; A first-level module, for directing the virtual address translation request to the corresponding first-level translation buffer based on the recognition result of the virtual address, to query whether a corresponding page table entry exists; if so, performing an address translation operation; A secondary module is used for querying the virtual address translation request using a hardware lookup table constructed by a cuckoo filter to obtain a hardware query result if there is no corresponding page table entry in the primary translation buffer; based on the hardware query result, directing the virtual address translation request to the corresponding secondary translation buffer to query whether there is a corresponding page table entry; if so, performing an address translation operation; The traversal module is used to perform a page table traversal operation on the virtual address conversion request if there is no corresponding page table entry in the secondary conversion buffer until the corresponding page table entry is obtained to complete the address conversion operation.

9. An electronic device, characterized in that: The electronic device comprises: a processor and a memory; The memory is used to store computer programs; The processor is used to execute the computer program stored in the memory, so that the electronic device executes the virtual memory architecture design method for MCM GPU according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by an electronic device, the virtual memory architecture design method for MCM GPU described in any one of claims 1 to 7 is implemented.

Images