Virtual memory architecture design methods, systems, media, and devices for MCM GPUs

Abstract

This invention provides a virtual memory architecture design method, system, medium, and device for MCM GPUs. The method includes the following steps: receiving and identifying a virtual address translation request; based on the identification result of the virtual address, directing the virtual address translation request to the corresponding first-level translation buffer to check if a corresponding page table entry exists; if it exists, performing an address translation operation; if it does not exist, querying the virtual address translation request using a hardware lookup table constructed with a Cuckoo filter; based on the hardware lookup result, directing the virtual address translation request to the corresponding second-level translation buffer to check if a corresponding page table entry exists; if it exists, performing an address translation operation; if it does not exist, performing a page table traversal operation on the virtual address translation request until the corresponding page table entry is obtained to complete the address translation operation. This invention improves the overall performance and efficiency of the GPU by optimizing data access and cache utilization between chips and improving L2TLB management.

Description

Technical Field

[0001] This invention belongs to the technical field of computer hardware and storage technology, and in particular relates to a virtual memory architecture design method, system, medium and device for MCM GPU. Background Technology

[0002] The evolution of GPU architecture, from single GPUs to multi-GPU systems, and now to the latest MCM GPUs (Multi-chip-Module GPUs), is a response to the slowdown in transistor scaling and the enormous computing power demands of the artificial intelligence era. MCM achieves higher performance and larger memory capacity by packaging multiple chiplets together. A chiplet is a technology that packages multiple dies performing specific functions together using die-to-die interconnect technology. While MCM GPUs stitch multiple chiplets together to form a logically unified "monolithic" GPU, physical resources are distributed across multiple chiplets. MCM GPUs introduce Non-Uniform Memory Access (NUMA) issues due to cross-chiplet resource access. Remote memory access is slower than accessing data residing in the memory of its own chiplet, impacting performance.

[0003] like Figure 1 As shown, Figure 1 This illustrates the general translation process for virtual addresses in a single GPU. In an MCM GPU, each CU (Compute Unit) contains a private Level 1 Translation Buffer (L1 TLB), while all CUs share a Level 2 Translation Buffer (L2 TLB). Address translation requests first query the L1 TLB; if it fails, they query the L2 TLB, and finally, page table translation is performed via a page roamer.

[0004] like Figure 2 As shown, Figure 2 This illustrates a typical architecture of an MCM GPU. An MCM GPU consists of multiple chiplets, each containing a small number of typical compute units and memory systems found in a single GPU. Each chiplet contains multiple CUs (Compute Units), and each CU comprises multiple SIMD units. The SIMD units within each CU share an L1 cache and an L1 TLB. Each chiplet has an L2 cache, shared by all CUs within that chiplet. Each chiplet is also connected to an HBM (Hardware Baseband) or GDDR (Glass Memory).

[0005] Currently, some applications are limited by L2 TLB capacity, while others are limited by page table access. Local L2 TLB hits are faster, and some applications perform better when using a private TLB. Other applications perform better with a shared L2 TLB design because the total L2 TLB capacity is larger and the L2 TLB miss rate is lower. Furthermore, if an application using a shared TLB design does not perform faster than a private TLB, this does not necessarily mean that the application is not benefiting from the aggregated TLB capacity; it could be because the data pages and thread blocks (CTAs) used by the program are not located on the same chiplet. In typical GPU applications, different CTAs often compute different parts of the dataset. If the CTAs and the data they access are mapped to the same chiplet, then even with a private L2 TLB, the L2 TLB capacity of that chiplet can be well utilized.

[0006] Therefore, how to address the challenges of modern technological needs through multi-chip modular design has become a technical problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0007] The purpose of this invention is to provide a virtual memory architecture design method, system, medium, and device for MCM GPUs, which improves the overall performance and efficiency of GPUs by optimizing data access and cache utilization between chiplets and improving L2 TLB management.

[0008] In a first aspect, the present invention provides a virtual memory architecture design method for MCM GPUs, the method comprising the following steps: receiving and identifying a virtual address translation request, and obtaining an identification result of the virtual address; based on the identification result of the virtual address, directing the virtual address translation request to the corresponding L1 TLB to query whether a corresponding page table entry exists; if it exists, performing an address translation operation; if it does not exist, querying the virtual address translation request using a hardware lookup table constructed using a Cuckoo filter to obtain a hardware lookup result; based on the hardware lookup result, directing the virtual address translation request to the corresponding L2 TLB to query whether a corresponding page table entry exists; if it exists, performing an address translation operation; if it does not exist, performing a page table traversal operation on the virtual address translation request until a corresponding page table entry is obtained to complete the address translation operation.

[0009] This implementation intelligently maps data and corresponding page table entries (PTEs) to the same or adjacent chips, reducing cross-chip data access requirements and thus lowering data access latency. It logically aggregates L2TLB caches on multiple chiplets to form a larger virtual cache, which can cache more PTEs and reduce TLB miss rate. Simultaneously, it improves L2 TLB management, reducing reliance on remote L2 TLBs. This significantly improves the performance and efficiency of the MCM GPU when handling complex tasks.

[0010] In one implementation of the first aspect, the method further includes L1 TLB grouping design: grouping adjacent preset number of calculation units into a group, allocating different address translation request ranges for the L1 TLBs within the calculation units, and distinguishing them by corresponding identifiers.

[0011] In this implementation, the grouped TLB design reduces duplicates between different CU L1 TLBs, and increases the TLB hit rate by expanding the TLB capacity available to each CU, thus significantly improving GPU performance and efficiency.

[0012] In one implementation of the first aspect, the method further includes an address space-aware buffer replacement control strategy: adding a field to the page table entry structure of the L1 TLB to distinguish between local page table entries and remote page table entries; dynamically adjusting the caching method of the L1 TLB by monitoring the ratio of access to the local page table entries and the remote page table entries and comparing it with a preset threshold.

[0013] In this implementation, the address space-aware buffer replacement control strategy not only further improves the LITLB hit rate, cache efficiency, and system responsiveness, but also reduces memory access latency and cross-chip communication overhead, thereby significantly improving the performance of GPU virtual memory management.

[0014] In one implementation of the first aspect, the method further includes a data page allocation mechanism based on the first access page migration: when a thread block accesses a specific data page for the first time, the data page is allocated to the dynamic random access memory inside the chiplet where the corresponding thread block is located, and the page table entry corresponding to the data page is also migrated to the page table storage structure inside the chiplet where the corresponding thread block is located.

[0015] In this implementation, the data page allocation mechanism for the first access page migration reduces the data transfer requirements across chiplets when accessing the data page subsequently, thereby reducing the communication overhead between chiplets and improving the locality of data access. This helps to reduce memory access latency and improve cache hit rate.

[0016] In one implementation of the first aspect, dynamically adjusting the caching method of the L1 TLB by monitoring the ratio of access to the local page table entries and the remote page table entries and comparing it with a preset threshold includes the following steps: when the access ratio of local page table entries in the L1 TLB is greater than a first threshold, the capacity of local page table entries in the L1 TLB is increased; when the access ratio of local page table entries in the L1 TLB is less than a second threshold, the capacity of remote page table entries in the L1 TLB is increased; when the L1 TLB needs to replace page table entries, page table entry types that exceed the preset threshold are replaced first.

[0017] In this implementation, by dynamically adjusting the cache capacity based on the access ratio of local page table entries and remote page table entries, TLB resources can be utilized more effectively and the caching strategy can be optimized.

[0018] In one implementation of the first aspect, based on the hardware lookup result, the virtual address translation request is directed to the corresponding L2 TLB to check if a corresponding page table entry exists; if it exists, the address translation operation includes the following steps: querying the virtual address translation request through the hardware lookup table constructed by the Cuckoo filter, and determining the type of the virtual address translation request; if the virtual address translation request belongs to a local page table entry, forwarding the virtual address translation request to the local L2 TLB to implement the address translation operation; if the virtual address translation request belongs to a remote page table entry, forwarding the virtual address translation request to the remote L2 TLB through a remote transmission unit to implement the address translation operation.

[0019] In this implementation, by using a cuckoo filter at the L2 TLB level to query and process address translation requests, not only is the efficiency of address translation and cache hit rate improved, but cross-chip communication overhead and memory access latency are also reduced, thereby significantly improving the performance of GPU virtual memory management and the overall efficiency of the system.

[0020] In one implementation of the first aspect, the method further includes a buffer consistency maintenance mechanism: when any L2TLB updates a page table entry, an update message is broadcast to ensure that the hardware lookup table constructed by the cuckoo filter is synchronously updated or replaced.

[0021] In this implementation, the buffer consistency maintenance mechanism ensures the synchronous updating of page table entries in the TLB, which not only maintains cache consistency but also improves the accuracy of data access and system performance, while enhancing system reliability.

[0022] Secondly, the present invention provides a virtual memory architecture design system for MCM GPUs, the system comprising:

[0023] The receiving module is used to receive and identify virtual address translation requests, and obtain the identification results of the virtual address;

[0024] The first-level module is used to direct the virtual address translation request to the corresponding L1 TLB based on the identification result of the virtual address to check whether there is a corresponding page table entry; if there is, the address translation operation is performed.

[0025] The second-level module is used to query the virtual address translation request using a hardware lookup table constructed with a Cuckoo filter if the corresponding page table entry does not exist in the L1 TLB, and obtain the hardware query result; based on the hardware query result, the virtual address translation request is directed to the corresponding L2 TLB to query whether the corresponding page table entry exists; if it exists, the address translation operation is performed.

[0026] The traversal module is used to perform a page table traversal operation on the virtual address translation request if the L2 TLB does not have a corresponding page table entry until the corresponding page table entry is obtained to complete the address translation operation.

[0027] Thirdly, the present invention provides an electronic device, the electronic device comprising: a processor and a memory;

[0028] The memory is used to store computer programs;

[0029] The processor is used to execute the computer program stored in the memory, so that the electronic device performs the virtual memory architecture design method for MCM GPU described above.

[0030] Fourthly, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by an electronic device, implements the above-described virtual memory architecture design method for MCM GPUs. Attached Figure Description

[0031] Figure 1 This diagram illustrates the general translation process of virtual addresses in a single GPU.

[0032] Figure 2 This is a typical architecture diagram of an MCM GPU;

[0033] Figure 3 The flowchart shown is an embodiment of the virtual memory architecture design method for MCM GPU of the present invention;

[0034] Figure 4 The diagram shows a grouped L1TLB architecture diagram in one embodiment of the virtual memory architecture design method for MCM GPUs of the present invention.

[0035] Figure 5 The diagram illustrates a modification of page table entries in one embodiment of the virtual memory architecture design method for MCM GPUs according to the present invention.

[0036] Figure 6 Shown as a detailed diagram of the ART microarchitecture based on the Cuckoo filter;

[0037] Figure 7 The diagram shows an ART-based address translation request flowchart of one embodiment of the virtual memory architecture design method for MCM GPUs according to the present invention.

[0038] Figure 8 The diagram shows a virtual address translation flowchart in one embodiment of the virtual memory architecture design method for MCM GPUs according to the present invention.

[0039] Figure 9 The diagram shown is a structural schematic of the electronic device of the present invention in one embodiment;

[0040] Figure 10 The diagram shown is a structural schematic of one embodiment of the virtual memory architecture design system for MCM GPUs according to the present invention. Detailed Implementation

[0041] The following specific examples illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that, unless otherwise specified, the following embodiments and features described therein can be combined with each other.

[0042] It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. Therefore, the drawings only show the components related to the present invention and are not drawn according to the actual number, shape and size of the components in the actual implementation. In the actual implementation, the form, quantity and proportion of each component can be arbitrarily changed, and the layout of the components may also be more complex.

[0043] The technical solutions of the present invention will now be described in detail with reference to the accompanying drawings.

[0044] like Figure 3 As shown, in one embodiment, the virtual memory architecture design method for MCM GPU of the present invention includes steps S11 to S14.

[0045] Step S11: Receive and identify the virtual address translation request and the corresponding address space, and obtain the identification result of the virtual address.

[0046] Specifically, in computer systems, especially in hardware accelerators such as GPUs, programs typically use virtual addresses to access memory. These virtual addresses need to be translated into physical addresses so that the actual hardware of the CPU or GPU can access the correct memory location. When the GPU needs to access memory to perform an operation, it initiates a virtual address translation request. This request contains virtual address information, which is a logical representation of the memory location the program is attempting to access. In a multitasking operating system, each process has its own virtual address space, and these address spaces are isolated to protect data between processes from incorrect access. The process of identifying address spaces involves determining the process or task that initiated the virtual address translation request and mapping it to its corresponding virtual address space. This is accomplished through the collaborative work of the operating system and hardware.

[0047] Once a virtual address translation request is received and the address space to which the request belongs is determined, the system needs to identify the specific information of the virtual address, including its page number and offset within the page. The identification result typically includes the virtual address's page number and offset. The page number is used for lookup in the page table, while the offset within the page is used to determine its specific location within the physical page.

[0048] Through this process, the system prepares for the subsequent address translation operation, ensuring that virtual addresses can be correctly and efficiently translated into physical addresses.

[0049] Step S12: Based on the identification result of the virtual address, the virtual address translation request is directed to the corresponding L1 TLB to check if there is a corresponding page table entry; if there is, the address translation operation is performed.

[0050] Furthermore, such as Figure 4 As shown, the method also includes an L1 TLB grouping architecture design: a preset number of adjacent calculation units are grouped together, and different address translation request ranges are allocated to the L1 TLBs within the calculation units, and they are distinguished by corresponding identifiers.

[0051] Specifically, the L1 TLB is private to each compute unit (CU), but due to limited capacity, many applications experience low L1 TLB hit rates. Adjacent CUs may access the same data pages, resulting in duplicates in the L1 TLB, indicating a shared characteristic. This shared characteristic is utilized by grouping four physically close CUs and their L1 TLBs together. The address translation request ranges served by the four L1 TLBs within each group are different, distinguished by the lower two bits of the virtual page number. As shown in Table 1, the address translation request ranges served by each L1 TLB are different, which evenly distributes the address ranges within the group, reducing duplicates and improving the hit rate.

[0052] L1 TLB number within the group Address range for address translation requests The lower two digits of the corresponding virtual page number 0 4a 00 1 4a+1 01 2 4a+2 10 3 4a+3 11

[0053] Table 1 shows the L1 TLB address allocation within each group.

[0054] Furthermore, the method also includes an address space-aware buffer replacement control strategy: adding a field to the page table entry structure of the L1 TLB to distinguish between local page table entries and remote page table entries; dynamically adjusting the caching method of the L1 TLB by monitoring the ratio of access to the local page table entries and the remote page table entries and comparing it with a preset threshold.

[0055] Specifically, by dynamically monitoring the ratio of accesses to local page table entries to remote page table entries, the L1 TLB replacement strategy can be optimized to further improve the hit rate. For example... Figure 5 As shown, the page table entry structure in the L1 TLB is modified by adding a bit (Local field) to distinguish between local and remote pages. The ratio of accesses to local and remote pages (L / T) is monitored and compared with preset thresholds (T1, T2). Based on the access ratio, the capacity of local and remote page table entries in the L1 TLB is dynamically adjusted. When the number of remote page table entries in the L1 TLB exceeds the set value, remote page table entries are replaced first; conversely, local page table entries are replaced first.

[0056] This strategy reduces the probability of page table entries of the corresponding type being replaced, and improves the hit rate of L1 TLB based on the principles of temporal and spatial locality.

[0057] In one embodiment, during initial allocation, the ways in the L1 TLB are evenly distributed between local pages and remote pages. Assuming the L1 TLB has a total of N ways, initially, local pages and remote pages are each allocated N / 2 ways. The system monitors and records the total number of address translation requests T and the number of local page address translation requests L within a statistical period. It then calculates the proportion L / T of local page address translation requests to the total number of address translations.

[0058] Set the first threshold T1 and the second threshold T2, which are used to determine when to adjust the allocation of ways. The values of T1 and T2 need to be determined according to the specific requirements and performance goals of the system.

[0059] If L / T < T2: This means that the access frequency of local pages is relatively low, and the importance of local pages decreases relative to remote pages. Therefore, the algorithm will reduce the number of ways allocated to local pages (local page way--), while increasing the number of ways allocated to remote pages (remote page way++).

[0060] If L / T > T1: This means that the access frequency of local pages is relatively high, and the importance of local pages increases relative to remote pages. Therefore, the algorithm will increase the number of ways allocated to local pages (local page way++), while reducing the number of ways allocated to remote pages (remote page way--).

[0061] If T1 ≤ L / T ≤ T2: This means that the access frequencies of local and remote pages are in a balanced state without obvious preference. Therefore, the algorithm keeps the current cache line allocation unchanged.

[0062] The above address space-aware L1 TLB allocation algorithm aims to optimize the hit rate of the L1 TLB by dynamically adjusting the allocation of ways. When more frequently accessed page table entries are retained in the TLB, future address translation requests are more likely to directly find the corresponding mapping in the L1 TLB, thus reducing the need to access the slower L2 TLB or main memory. During the execution of a program, its memory access pattern may change. This algorithm can dynamically adjust the TLB allocation according to the change of the access pattern to maintain the best performance. Through this dynamic adjustment strategy, the system can better adapt to different workloads and access patterns, thereby improving the overall memory access efficiency.

[0063] In summary, through the grouped L1 TLB architecture and the address space-aware TLB replacement control strategy, duplicates can be reduced and the replacement strategy can be optimized, and the hit rate of the L1 TLB is improved. The improvement of the L1 TLB hit rate means that fewer requests are forwarded to the L2 TLB, thus reducing the burden on the L2 TLB. By making better use of the L1 TLB, the memory access efficiency of the GPU is improved, thereby enhancing the overall performance.

[0064] Furthermore, the method also includes a data page allocation mechanism based on the first access page migration: when a thread block accesses a specific data page for the first time, the data page is allocated to the dynamic random access memory inside the chiplet where the corresponding thread block is located, and the page table entry corresponding to the data page is also migrated to the page table storage structure inside the chiplet where the corresponding thread block is located.

[0065] Specifically, when a data page is accessed for the first time, it is allocated to the DRAM of the chiplet containing the thread block (CTA) that initiated the access. This strategy is based on the computing characteristics of the GPU, namely that once a data page is first accessed by a CTA, it is likely that the CTA will be primarily responsible for the computation. This mechanism reduces data communication between chiplets, as each chiplet mainly accesses its own DRAM.

[0066] Step S13: If it does not exist, the virtual address translation request is queried using the hardware lookup table constructed by the Cuckoo filter to obtain the hardware lookup result; based on the hardware lookup result, the virtual address translation request is directed to the corresponding secondary translation buffer to check if there is a corresponding page table entry; if it exists, the address translation operation is performed.

[0067] Specifically, when a virtual address translation request occurs, it is first queried through a Cuckoo filter. If the Cuckoo filter indicates that the request may exist, it provides a hardware query result, which tells the GPU which secondary translation buffer to query. Based on the hardware query result of the Cuckoo filter, the virtual address translation request is directed to the corresponding secondary translation buffer. Each chiplet has its own secondary translation buffer for storing local page table entries. The specified secondary translation buffer is checked to see if a corresponding page table entry exists. If the page table entry exists, the address translation operation can be performed directly, converting the virtual address to a physical address.

[0068] like Figure 6 As shown, a Cuckoo filter is an efficient data structure used to quickly check whether an element belongs to a set without storing the entire element. In multi-chiplet GPU systems, Cuckoo filters are used to support logical sharing of L2TLBs.

[0069] The Cuckoo filter consists of a series of buckets, each capable of storing multiple fixed-length fingerprints. Each ART (Address Request Table) contains 128 buckets, each with 4 fingerprints, thus theoretically supporting 512 page table entries. A fingerprint is a short representation of the hash value of an element (in this scenario, the virtual page number VPN). Using a hash function (such as MetroHash), the VPN is transformed into a fixed-length fingerprint. The Cuckoo filter uses two hash functions (h1 and h2) to generate two possible storage locations (alternate buckets) for each element. When a new element is inserted, its fingerprint is attempted to be inserted into one of these two locations.

[0070] Insertion process: If both candidate bins are full, the fingerprint is inserted into one of them. If both bins are full, the cuckoo filter selects a bin, kicks out an existing fingerprint, and reinserts this "victim" fingerprint into its candidate position. This process can trigger a series of "kickout" operations, known as "Cuckooing."

[0071] Search operation: During the search, the Cuckoo filter calculates the element's fingerprint and the positions of two candidate buckets. If a matching fingerprint is found in either bucket, it returns true, indicating that the element exists; otherwise, it returns false.

[0072] Deletion operation: The deletion operation removes the matching fingerprint from any candidate bucket.

[0073] In one embodiment, when a virtual-to-physical mapping is inserted into the L2 TLB of a chiplet, the fingerprint of the corresponding VPN is also recorded in the ART. This allows for a quick lookup via the ART when checking if a virtual page number exists in the L2 TLB of a chiplet. When the virtual-to-physical mapping in the L2 TLB is updated (inserted or deleted), the ART also needs to be updated accordingly to maintain data consistency. If a mapping is removed from the L2 TLB, its corresponding fingerprint must also be deleted from the ART. Since the ART contains eight comparators, it can check all candidate fingerprints in both buckets in parallel, which greatly improves lookup efficiency.

[0074] In this way, the Cuckoo filter plays a key role in the multi-chiplet architecture of the GPU, enabling logical sharing of L2 TLB between different chiplets while maintaining efficient address translation performance.

[0075] In one embodiment, when an L1 TLB miss occurs, the address translation request first queries the corresponding ART. If the ART indicates that the request exists in the L2 TLB of a certain chiplet, the request is forwarded to the corresponding L2 TLB. If it is a local chiplet, it is forwarded directly to the local L2 TLB; if it is a remote chiplet, it is forwarded via the RTU. When the L2 TLB is updated, the ARTs on all chiplets also need to be updated to reflect the latest virtual-to-physical mapping information.

[0076] like Figure 7 As shown, when the L1 TLB cannot find the corresponding Page Table Entry (PTE), a miss occurs, and the address translation request needs further processing. The address request is first sent to the corresponding ART (e.g., [ART]x, where x is the chiplet number). The ART records the L2 TLB information for each chiplet, including which chiplet is responsible for handling requests for a specific address range. The address request is searched in [ART]x. If found, it is determined whether the request is local or remote. If the request is local, it is forwarded directly to the local chiplet's L2 TLB. If the request is remote, it is forwarded to the remote chiplet via a Remote Transmission Unit (RTU). The address translation request is then queried in the local chiplet's L2 TLB. If the L2 TLB hits, the address translation request is directly returned to the corresponding L1 TLB, completing the address translation. If the local L2 TLB misses, a page walk operation is needed to obtain the page table entry. Simultaneously with updating the local L2 TLB, the ARTs on all chiplets also need to be updated to ensure that subsequent requests can be correctly routed. If the request is forwarded to the remote chiplet's L2 TLB, the remote L2 TLB performs an address translation request lookup. If the remote L2 TLB hits, the address translation response is returned to the original L1 TLB via the RTU. If the remote L2 TLB misses, the page table roamer on the remote chiplet performs a page traversal. Page table entries are inserted into the remote L2 TLB to handle the current request. Simultaneously, the remote L2 TLB needs to broadcast L2 TLB update information to all chiplets to ensure the ART information is up-to-date.

[0077] Whether updating the L2 TLB locally or remotely, it's crucial to ensure that the ART information on all chiplets is up-to-date to correctly route future address requests. The key to this process is ensuring that address requests are quickly and accurately routed to the correct L2 TLB and that the corresponding page table entry is found within the L2 TLB to complete the address translation. By maintaining and promptly updating the ART, the system can effectively handle cross-chiplet memory access requests, improving overall memory access efficiency.

[0078] In one embodiment, such as Figure 8 As shown, the CU generates a virtual address request. First, it queries the virtual address in the CU's L1 TLB. If the L1 TLB is invalid, it looks up the corresponding chiplet number in the address request table and then sends the request to the corresponding chiplet's L2 TLB. If the chiplet's L2 TLB is invalid, the request is sent to the page rovers. The page rovers search for the corresponding memory location based on the information in the page table and update their local page table cache. The obtained PTE is updated in the corresponding chiplet's L2 TLB so that the mapping information can be directly found in the TLB in the future. The PTE is used to translate the virtual address into a physical address and process the memory request. During the virtual-to-physical translation process, if page table entries are updated or inserted, the corresponding TLB and the Cuckoo filter-based ART (Address Request Table) also need to be updated.

[0079] In this way, MCM GPUs can more efficiently utilize their aggregated L2 TLB capacity, reduce cross-chiplet memory access latency, and optimize overall memory access performance. This architecture is particularly suitable for applications that are sensitive to memory access latency, as well as those that require significant memory bandwidth and caching.

[0080] Furthermore, the total ART hardware overhead on an MCM GPU can be calculated using the following formula:

[0081] n = Num c ×Num s ×Num e ×Num b

[0082] Where n represents the number of bits, Num c Num indicates the number of chiplets. s Num represents the number of lookup tables for each chiplet. e Num represents the number of entries per page in each lookup table. b This indicates the number of bits in the fingerprint corresponding to each page table entry.

[0083] The scope of protection of the virtual memory architecture design method for MCM GPU described in this embodiment is not limited to the execution order of the steps listed in this embodiment. Any solution implemented by adding, subtracting, or replacing steps in the prior art based on the principles of this invention is included within the scope of protection of this invention.

[0084] like Figure 9 As shown, this embodiment of the invention also provides an electronic device for designing a virtual memory architecture for an MCM GPU. The electronic device includes at least one processor 901, a memory 902, at least one network interface 903, and a user interface 905. The various components in the device are coupled together via a bus system 904. It is understood that the bus system 904 is used to implement communication between these components. In addition to a data bus, the bus system 904 also includes a power bus, a control bus, and a status signal bus. However, for clarity, in… Figure 9 The general will label all buses as bus systems.

[0085] The user interface 905 may include a monitor, keyboard, mouse, trackball, clicker, button, touchpad, or touch screen.

[0086] It is understood that memory 902 can be volatile memory or non-volatile memory, or both. Non-volatile memory can be read-only memory (ROM) or programmable read-only memory (PROM), used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as static random access memory (SRAM) and synchronous static random access memory (SSRAM). The memories described in the embodiments of this invention are intended to include, but are not limited to, these and any other suitable categories of memory.

[0087] In this embodiment of the invention, the memory 902 is used to store various types of data to support the operation of the electronic terminal 900. Examples of this data include: any executable program for operation on the electronic terminal 900, such as the operating system 9021 and application programs 9022; the operating system 9021 contains various system programs, such as the framework layer, core library layer, driver layer, etc., for implementing various basic services and handling hardware-based tasks. The application program 9022 may contain various applications, such as a media player, browser, etc., for implementing various application services. The virtual memory architecture design method for MCM GPU provided in this embodiment of the invention can be included in the application program 9022.

[0088] The methods disclosed in the above embodiments of the present invention can be applied to or implemented by processor 901. Processor 901 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the integrated logic circuit of the hardware in processor 901 or by instructions in software form. The processor 901 may be a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Processor 901 can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of the present invention. General-purpose processor 901 may be a microprocessor or any conventional processor, etc. The steps of the accessory optimization method provided in the embodiments of the present invention can be directly reflected as being executed by a hardware decoding processor, or being executed by a combination of hardware and software modules in the decoding processor. The software module may be located in a storage medium, which is located in memory. The processor reads the information in the memory and combines it with its hardware to complete the steps of the aforementioned method.

[0089] In an exemplary embodiment, the electronic terminal 900 may be used by one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), or complex programmable logic devices (CPLDs) to execute the aforementioned method.

[0090] This invention also provides a virtual memory architecture design system for MCM GPUs. The virtual memory architecture design system for MCM GPUs can implement the virtual memory architecture design method for MCM GPUs described in this invention. However, the implementation device of the virtual memory architecture design system for MCM GPUs described in this invention includes, but is not limited to, the structure of the virtual memory architecture design system for MCM GPUs listed in this embodiment. All structural modifications and substitutions of the prior art made according to the principles of this invention are included within the protection scope of this invention.

[0091] like Figure 10 As shown, in one embodiment, the virtual memory architecture design system for MCM GPUs of the present invention includes:

[0092] The receiving module 101 is used to receive and identify virtual address translation requests, and obtain the identification result of the virtual address;

[0093] The first-level module 102, connected to the receiving module 101, is used to direct the virtual address translation request to the corresponding first-level translation buffer based on the identification result of the virtual address, in order to check whether there is a corresponding page table entry; if there is, the address translation operation is performed.

[0094] The secondary module 103, connected to the primary module 102, is used to query the virtual address translation request using a hardware lookup table constructed by the Cuckoo filter if the corresponding page table entry does not exist in the primary translation buffer, and obtain the hardware query result; based on the hardware query result, the virtual address translation request is directed to the corresponding secondary translation buffer to query whether the corresponding page table entry exists; if it exists, the address translation operation is performed.

[0095] The traversal module 104, connected to the secondary module 103, is used to perform a page table traversal operation on the virtual address translation request until the corresponding page table entry is obtained and the address translation operation is completed if the secondary translation buffer does not have a corresponding page table entry.

[0096] In the embodiments provided by this invention, it should be understood that the disclosed systems, apparatuses, or methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For instance, the division of modules / units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules or units may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection of apparatuses or modules or units may be electrical, mechanical, or other forms.

[0097] The modules / units described as separate components may or may not be physically separate. The components shown as modules / units may or may not be physical modules; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules / units can be selected to achieve the objectives of the embodiments of the present invention, depending on actual needs. For example, the functional modules / units in the various embodiments of the present invention may be integrated into one processing module, or each module / unit may exist physically separately, or two or more modules / units may be integrated into one module / unit.

[0098] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0099] This invention also provides a computer-readable storage medium. Those skilled in the art will understand that all or part of the steps in the methods of the above embodiments can be implemented by a program instructing a processor. The program can be stored in a computer-readable storage medium, which is a non-transitory medium, such as random access memory, read-only memory, flash memory, hard disk, solid-state drive, magnetic tape, floppy disk, optical disk, and any combination thereof. The storage medium can be any available medium accessible to a computer or a data storage device such as a server or data center that integrates one or more available media. This available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., digital video disc (DVD)), or a semiconductor medium (e.g., solid-state drive (SSD)).

Claims

1. A virtual memory architecture design method for MCM GPUs, characterized in that, The method includes the following steps: Receive and identify virtual address translation requests, and obtain the identification results of virtual addresses; Based on the identification result of the virtual address, the virtual address translation request is directed to the corresponding first-level translation buffer to check if a corresponding page table entry exists; if it exists, the address translation operation is performed. If it does not exist, the virtual address translation request is queried using the hardware lookup table built with the Cuckoo filter to obtain the hardware lookup result; Based on the hardware query results, the virtual address translation request is directed to the corresponding secondary translation buffer to check if a corresponding page table entry exists; if it exists, the address translation operation is performed. If it does not exist, then perform a page table traversal operation on the virtual address translation request until the corresponding page table entry is obtained to complete the address translation operation; The method also includes an address space-aware buffer replacement control strategy; adding a field to the page table entry structure of the first-level translation buffer to distinguish between local page table entries and remote page table entries; and dynamically adjusting the caching method of the first-level translation buffer by monitoring the ratio of access to the local page table entries and the remote page table entries and comparing it with a preset threshold. The method also includes a buffer consistency maintenance mechanism: when any of the secondary transformation buffers updates a page table entry, an update message is broadcast to ensure that the hardware lookup table built by the cuckoo filter is synchronously updated or replaced; Based on the hardware query results, the virtual address translation request is directed to the corresponding secondary translation buffer to check if a corresponding page table entry exists; if it exists, the address translation operation includes the following steps: The virtual address translation request is queried through the hardware lookup table constructed by the cuckoo filter, and the type of the virtual address translation request is determined. If the virtual address translation request belongs to a local page table entry, the virtual address translation request is forwarded to the local secondary translation buffer to perform the address translation operation; If the virtual address translation request belongs to a remote page table entry, the virtual address translation request is forwarded to the remote secondary translation buffer through the remote transmission unit to realize the address translation operation.

2. The virtual memory architecture design method for MCM GPU according to claim 1, characterized in that, The method also includes a first-level translation buffer grouping design: a preset number of adjacent calculation units are grouped together, and different address translation request ranges are allocated to the first-level translation buffers within the calculation units, and they are distinguished by corresponding identifiers.

3. The virtual memory architecture design method for MCM GPU according to claim 1, characterized in that, The method also includes a data page allocation mechanism based on first-access page migration: When a thread block accesses a specific data page for the first time, the data page is allocated to the dynamic random access memory inside the core where the corresponding thread block is located, and the page table entry corresponding to the data page is also migrated to the page table storage structure inside the core where the corresponding thread block is located.

4. The virtual memory architecture design method for MCM GPU according to claim 1, characterized in that, The dynamic adjustment of the caching method of the first-level translation buffer by monitoring the ratio of access to the local page table entries and the remote page table entries, and comparing it with a preset threshold, includes the following steps: When the access ratio of local page table entries in the first-level translation buffer is greater than the first threshold, the capacity of local page table entries in the first-level translation buffer is increased. When the access ratio of local page table entries in the first-level translation buffer is less than the second threshold, the capacity of remote page table entries in the first-level translation buffer is increased. When the first-level conversion buffer needs to replace page table entries, page table entry types that exceed the preset threshold are replaced first.

5. A virtual memory architecture design system for MCM GPUs, characterized in that, The system includes: The receiving module is used to receive and identify virtual address translation requests, and obtain the identification results of the virtual address; The first-level module is used to direct the virtual address translation request to the corresponding first-level translation buffer based on the identification result of the virtual address, in order to check whether there is a corresponding page table entry; if there is, the address translation operation is performed. The secondary module is used to query the virtual address translation request using a hardware lookup table constructed with a Cuckoo filter if the corresponding page table entry does not exist in the primary translation buffer, and obtain the hardware query result; based on the hardware query result, the virtual address translation request is directed to the corresponding secondary translation buffer to query whether the corresponding page table entry exists; if it exists, the address translation operation is performed. The traversal module is used to perform a page table traversal operation on the virtual address translation request until the corresponding page table entry is obtained and the address translation operation is completed if the corresponding page table entry does not exist in the secondary translation buffer. The system also includes an address space-aware buffer replacement control strategy; adding a field to the page table entry structure of the first-level translation buffer to distinguish between local page table entries and remote page table entries; and dynamically adjusting the caching method of the first-level translation buffer by monitoring the ratio of access to the local page table entries and the remote page table entries and comparing it with a preset threshold. The system also includes a buffer consistency maintenance mechanism: when any of the secondary transformation buffers updates a page table entry, an update message is broadcast to ensure that the hardware lookup table built by the cuckoo filter is synchronously updated or replaced; Based on the hardware query results, the virtual address translation request is directed to the corresponding secondary translation buffer to check if a corresponding page table entry exists; if it exists, the address translation operation includes the following steps: The virtual address translation request is queried through the hardware lookup table constructed by the cuckoo filter, and the type of the virtual address translation request is determined. If the virtual address translation request belongs to a local page table entry, the virtual address translation request is forwarded to the local secondary translation buffer to perform the address translation operation; If the virtual address translation request belongs to a remote page table entry, the virtual address translation request is forwarded to the remote secondary translation buffer through the remote transmission unit to realize the address translation operation.

6. An electronic device, characterized in that, The electronic device includes: a processor and a memory; The memory is used to store computer programs; The processor is used to execute the computer program stored in the memory to cause the electronic device to perform the virtual memory architecture design method for MCM GPU as described in any one of claims 1 to 4.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by an electronic device, the program implements the virtual memory architecture design method for MCM GPU as described in any one of claims 1 to 4.

Images