A GPU memory access simulation system, method, storage medium, and electronic device

Abstract

This invention relates to a GPU memory access simulation system, method, storage medium, and electronic device. The GPU memory access simulation system includes: a GPU multi-threaded simulation module, a page computation module, a shared lock dynamic allocator, and a GPU memory module. The page computation module is configured to respond to a memory access request initiated by any thread in the GPU multi-threaded simulation module, and determine the memory page to be accessed based on the memory access request. The shared lock dynamic allocator is configured to allocate one or more locks corresponding to the memory pages accessed by each thread based on the memory pages accessed by each thread's memory access request, so that all threads initiating memory access requests can access the GPU memory module according to their respective allocated locks. This invention improves the simulation efficiency of GPU memory access simulation by using a shared lock dynamic allocation mechanism to allocate one or more locks corresponding to the memory pages accessed by each thread based on the memory pages accessed by each thread's memory access request.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer, and in particular, to a GPU memory access simulation system and method, a storage medium and an electronic device. BACKGROUND

[0002] In the early stage of GPU design, software modeling is essential for verifying the functionality and performance of the GPU. This process not only allows for a preliminary evaluation of the potential performance of the GPU, but also provides software developers with an early development platform through a simulated environment. This approach promotes parallel development across multiple departments and accelerates the progress of the entire project.

[0003] In previous GPU architecture model designs, ESL modeling usually uses C++ frameworks such as SystemC to abstract the interaction between hardware modules (such as computing units, memory controllers, buses) through software-level "transactions" (such as data transfer requests, operation commands), rather than accurately describing circuit logic. The core of the commonly used hardware design-based modeling method is to "accurately simulate hardware details" (such as clock synchronization, signal timing, protocol interaction). In order to ensure the accuracy of hardware characteristics, the speed of software simulation is often sacrificed. For example, in order to simulate the replacement strategy of Cache or the arbitration mechanism of the bus, a large number of detailed logic needs to be introduced, resulting in a long simulation process.

[0004] For GPU memory access-related modeling, the full link from "thread memory access request -> L1 Cache -> L2 Cache -> global memory (or high bandwidth memory HBM)" and the communication protocol and bus interaction in between are simulated in ESL modeling. This full link simulation further increases the complexity of the model, exacerbating the problem of simulation efficiency. In the process of multi-threaded memory access, in order to ensure data consistency for concurrent access operations to the same memory page, the memory interface is usually locked, allowing only one thread (or thread bundle) to access the memory interface at the same time, and other threads need to wait, which can cause a significant performance delay. Especially in "memory-intensive tasks" such as deep learning and large-scale parallel computing, memory access is the core operation, and the serialization caused by the lock mechanism completely negates the parallel capabilities of multi-threading.

[0005] That is, in related technologies, transaction-level software models are used to accurately simulate hardware characteristics (such as memory access links, hardware parameters), but due to excessive attention to hardware details and the lack of optimization of the global lock mechanism for multi-threaded memory access, the simulation efficiency is low, especially in the multi-threaded memory access scenario of the GPU, lock contention can seriously offset the parallel advantage, becoming an obstacle to architecture design verification. SUMMARY

[0006] Therefore, the present application provides a GPU memory access simulation system, method, storage medium and electronic device to improve the simulation efficiency of GPU memory access simulation.

[0007] Specifically, the present application is implemented by the following technical solutions:

[0008] According to a first aspect of the present application, a GPU memory access simulation system is provided, which comprises a GPU multi-thread simulation module, a page calculation module, a shared lock dynamic allocator and a GPU memory module.

[0009] The page calculation module is configured to determine the accessed memory page based on a memory access request initiated by any one thread in the GPU multi-thread simulation module.

[0010] The shared lock dynamic allocator is configured to allocate one or more locks corresponding to the accessed memory page to each thread based on the accessed memory page corresponding to the memory access request of each thread, so that all threads initiating the memory access request can access the GPU memory module according to the locks allocated to each thread.

[0011] In some embodiments, the number of locks allocated to the thread is the same as the number of pages of the accessed memory page determined based on the memory access request of the thread.

[0012] In some embodiments, the multiple threads allocated to the locks can perform the operation of accessing the GPU memory module according to the locks in parallel.

[0013] In some embodiments, the memory access request carries the following information: a memory access initial virtual address and a memory access required length.

[0014] The page calculation module is configured to:

[0015] calculate the start page code corresponding to the accessed memory page according to the memory access initial virtual address;

[0016] calculate the total number of pages corresponding to the accessed memory page according to the memory access required length;

[0017] calculate the page codes of each page corresponding to the accessed memory page according to the start page code and the total number of pages.

[0018] The shared lock dynamic allocator is configured to:

[0019] allocate one or more locks corresponding to the accessed memory page to each thread based on the page codes of each page of the accessed memory page corresponding to the memory access request of each thread.

[0020] In some embodiments, the shared lock dynamic allocator comprises an object pool storing a mapping relationship between page numbers of memory pages and locks; wherein, in the mapping relationship, at least one of the following situations exists: one lock corresponds to one page number of memory pages, or one lock corresponds to multiple page numbers of memory pages.

[0021] The shared lock dynamic allocator is configured to:

[0022] For each thread initiating a memory access request, based on the mapping relationship, determine target locks corresponding to page numbers of the accessed memory pages corresponding to the memory access request of the thread.

[0023] In some embodiments, the shared lock dynamic allocator is configured to:

[0024] In the mapping relationship, query whether there is a lock corresponding to each page number of the accessed memory pages corresponding to the memory access request;

[0025] In response to querying a lock corresponding to a first page number in the page numbers, determine the queried one or more locks as target locks corresponding to the first page number; the first page number is a page number of the accessed memory pages corresponding to the memory access request.

[0026] In some embodiments, the shared lock dynamic allocator is further configured to:

[0027] In response to failing to query a lock corresponding to the first page number in the page numbers, generate a new lock corresponding to the first page number;

[0028] Register a relationship between the first page number and the new lock in the object pool, and update the mapping relationship in the mapping relationship after successful registration;

[0029] Based on the updated mapping relationship, determine the new lock corresponding to the first page number as a target lock corresponding to the first page number.

[0030] In some embodiments, the shared lock dynamic allocator is further configured to:

[0031] Simultaneously manage states of multiple locks; the states of the locks include used and unused;

[0032] In response to a state of a target lock corresponding to any page number of a memory page accessed by a thread initiating a memory access request being unused, allocate the unused target lock to the thread, so that the thread accesses the corresponding memory page according to the unused target lock.

[0033] In some embodiments, the mapping relationship is stored based on a hash table, a key of the hash table is a page number, and a value is a pointer of a lock.

[0034] In some embodiments, the shared lock dynamic allocator is further configured to:

[0035] In response to the number of objects in the object pool exceeding a preset threshold, destroying target objects by a preset strategy, the target objects including a key-value pair and an object instance pointed by a pointer of a lock determined based on the key-value pair, the object instance being an address space allocated in a GPU memory module.

[0036] According to a second aspect of the present application, a GPU memory access simulation method is provided, the method comprising:

[0037] In response to any one of the threads in the GPU multi-thread simulation module initiating a memory access request, determining an accessed memory page based on the memory access request;

[0038] Based on the accessed memory page corresponding to the memory access request of each thread, allocating one or more locks corresponding to the accessed memory page to each thread respectively, so that all threads initiating the memory access request can access the GPU memory module according to the locks allocated respectively.

[0039] In some embodiments, the memory access request carries the following information: a memory access initial virtual address and a memory access required length;

[0040] The determining of the accessed memory page based on the memory access request comprises:

[0041] Calculating a start page code corresponding to the accessed memory page according to the memory access initial virtual address;

[0042] Calculating a total page number corresponding to the accessed memory page according to the memory access required length;

[0043] Calculating each page page code corresponding to the accessed memory page according to the start page code and the total page number;

[0044] The allocating one or more locks corresponding to the accessed memory page to each thread respectively based on the accessed memory page corresponding to the memory access request of each thread comprises:

[0045] Allocating one or more locks corresponding to the accessed memory page to each thread respectively based on each page page code corresponding to the accessed memory page corresponding to the memory access request of each thread.

[0046] In some embodiments, the number of locks allocated to a thread is the same as the number of pages of the accessed memory page determined based on the memory access request of the thread.

[0047] In some embodiments, the multiple threads allocated to the locks can perform the operation of accessing the GPU memory module according to the locks in parallel.

[0048] In some embodiments, the shared lock dynamic allocator includes an object pool that stores a mapping relationship between page numbers of memory pages and locks; wherein, in the mapping relationship, at least one of the following conditions exists: one lock corresponds to one page number of a memory page, or one lock corresponds to multiple page numbers of a memory page;

[0049] The memory pages accessed based on the memory access requests of each thread are allocated one or more locks corresponding to the accessed memory pages for each thread, including:

[0050] For each thread that initiates a memory access request, based on the mapping relationship, the target lock corresponding to the page number of each page of the memory page accessed by the thread is determined.

[0051] In some embodiments, determining the target lock corresponding to each page number of the memory page accessed by the thread based on the mapping relationship includes:

[0052] In the mapping relationship, check if there is a lock corresponding to each page number of the memory page accessed by the memory access request;

[0053] In response to finding the lock corresponding to the first page number among all page numbers, one or more of the found locks are identified as the target lock corresponding to the first page number; the first page number is the page number of the memory page accessed corresponding to the memory access request.

[0054] In some embodiments, determining the target lock corresponding to each page number of the memory page accessed by the thread based on the mapping relationship further includes:

[0055] If the lock corresponding to the first page number cannot be found in the page numbers of each page, a new lock corresponding to the first page number is generated.

[0056] Register the relationship between the first page number and the new lock in the object pool, and update the storage in the mapping relationship after successful registration;

[0057] Based on the updated mapping, the new lock corresponding to the first page number is determined as the target lock corresponding to the first page number.

[0058] In some embodiments, the method further includes:

[0059] Simultaneously manage the states of multiple locks; the states of the locks include used and unused.

[0060] In response to the fact that the target lock corresponding to any page number in the memory page accessed by the thread that initiated the memory access request is in an unused state, the unused target lock is allocated to the thread so that the thread can access the corresponding memory page based on the unused target lock.

[0061] In some embodiments, the mapping relationship is stored based on a hash table, where the key is the page number and the value is a pointer to the lock.

[0062] In some embodiments, the method further includes:

[0063] In response to the number of objects in the object pool exceeding a preset threshold, a target object is destroyed according to a preset strategy. The target object includes a key-value pair and an object instance pointed to by a pointer to a lock determined based on the key-value pair. The object instance is an address space allocated in the GPU memory module.

[0064] According to a third aspect of the invention, a storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps of the GPU memory access simulation method in the second aspect or any possible implementation thereof.

[0065] According to a fourth aspect of the present invention, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the GPU memory access simulation method in the second aspect or any possible implementation thereof.

[0066] According to a fifth aspect of the present invention, a computer program product is provided, comprising a computer program that, when executed by a processor, implements the steps of the GPU memory access simulation method in the second aspect or any possible implementation thereof.

[0067] The technical solution provided by this invention brings at least the following beneficial effects:

[0068] In the GPU memory access simulation system provided by this invention, when any thread in the GPU multi-threaded simulation module initiates a memory access request, the page calculation module first determines the memory page to be accessed based on the memory access request. Then, a shared lock dynamic allocator allocates one or more locks corresponding to the memory pages accessed by each thread based on the memory pages accessed by each thread's memory access request. Finally, all threads that initiated memory access requests can access the GPU memory module according to their respective allocated locks. This invention, through a shared lock dynamic allocation mechanism, allocates one or more locks corresponding to the memory pages accessed by each thread based on the memory pages accessed by each thread's memory access request. While ensuring "consistency in memory access within the same page," memory accesses to different pages can be executed in parallel, which significantly improves the parallelism of multi-threaded memory access and thus improves the simulation efficiency of GPU memory access simulation. Attached Figure Description

[0069] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0070] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, those skilled in the art can obtain other drawings based on these drawings without creative effort.

[0071] Figure 1 This invention provides a diagram of a GPU memory access simulation system architecture.

[0072] Figure 2 A GPU memory access simulation system architecture diagram provided in this embodiment of the invention;

[0073] Figure 3 A flowchart of the first GPU memory access simulation method provided in the embodiments of the present invention;

[0074] Figure 4 A flowchart of the second GPU memory access simulation method provided in this embodiment of the invention;

[0075] Figure 5 A flowchart of the third GPU memory access simulation method provided in this embodiment of the invention;

[0076] Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0077] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0078] In the GPU design process, software modeling of the GPU is required in the early stages of the project. This allows for early verification of the GPU's functionality and performance, and also provides a development platform to software developers in advance through virtual machines, facilitating parallel development by multiple departments.

[0079] Currently, in the modeling process of GPU memory access, the GPU memory access simulation systems used are such as... Figure 1 As shown, Figure 1The diagram illustrates the architecture of a GPU memory access simulation system provided by this invention. This system includes multiple threads 101, a global lock 102, an address translation module 103, and a GPU memory module 104. The address translation module 103 converts virtual addresses (VA) to physical addresses (PA). Multiple threads 101 share a single global lock 102. In multi-threaded memory access scenarios, multiple threads compete for the same global lock 102. Only the thread that acquires the lock can perform the VA-to-PA conversion operation, and this lock-grabbing process causes significant performance degradation. In other words, in the software modeling of GPU memory access, the introduction of the global lock 102 forces the VA-to-PA conversion operation from a "parallel behavior" to a "serialized behavior," resulting in a dramatic increase in latency across the entire memory access chain, leading to low simulation efficiency. This completely contradicts the characteristics of parallel GPU memory access.

[0080] To address the problem of low simulation efficiency caused by the global locking mechanism of GPU multi-threaded memory access, which forces "parallel behavior" to "serialized behavior," this invention provides a GPU memory access simulation system, method, storage medium, and electronic device. When a thread initiates a memory access request, the page calculation module first determines the memory page to be accessed based on the memory access request. Then, a shared lock dynamic allocator allocates one or more locks corresponding to the memory pages accessed by each thread based on the memory pages accessed by each thread's memory access request. Finally, all threads initiating memory access requests can access the GPU memory module according to their allocated locks. Through the shared lock dynamic allocation mechanism, one or more locks corresponding to the memory pages accessed by each thread are allocated to each thread based on the memory pages accessed by each thread's memory access request. While ensuring "consistency of memory access within the same page," memory access to different pages can be executed in parallel, which significantly improves the parallelism of multi-threaded memory access and thus improves the simulation efficiency of GPU memory access simulation.

[0081] See Figure 2 This invention provides a GPU memory access simulation system, which includes a GPU multi-threaded simulation module 201, a page calculation module 202, a shared lock dynamic allocator 203, and a GPU memory module 104.

[0082] The GPU multi-threaded simulation module 201 includes multiple threads 101, such as thread 1, thread 2, ..., thread k. Each thread 101 is a pipeline, and these multiple threads 101 are used to handle memory access transactions initiated by the GPU core. Alternatively, different threads 101 in the GPU multi-threaded simulation module 201 can represent different GPU cores.

[0083] Page calculation module 202 is configured to respond to a memory access request initiated by any thread 101 in the GPU multi-threaded simulation module 201, and determine the memory page to be accessed based on the memory access request. Here, the memory page refers to a memory page in the GPU memory module 104.

[0084] In some embodiments, the memory access request carries the following information: the initial virtual address for memory access and the length required for memory access;

[0085] Page calculation module 202 is configured as follows:

[0086] The starting page number corresponding to the memory page being accessed is calculated based on the initial virtual address of the memory access.

[0087] Calculate the total number of pages corresponding to the memory pages accessed based on the required memory access length;

[0088] The page numbers corresponding to the accessed memory pages are calculated based on the starting page number and the total number of pages.

[0089] In some embodiments, the memory access request may also carry the operation type (read / write).

[0090] In this embodiment of the invention, the starting page number corresponding to the accessed memory page can be calculated based on the initial virtual address of the memory access using the following formula:

[0091] Starting page number = initial virtual address for memory access / page size.

[0092] For example, if the initial virtual address for memory access is VA, and the page size of the memory page is 1024 bytes, the calculated starting page number is VA / 1024, where VA is a byte-aligned address.

[0093] In this embodiment of the invention, the total number of memory pages accessed can be calculated based on the required memory access length using the following formula:

[0094] Total number of pages = (length required for memory access + page size - 1) / page size.

[0095] Of course, other calculation formulas can also be used to calculate the starting page number and total number of pages corresponding to the memory page accessed by the thread's memory access request. This embodiment of the invention does not limit this.

[0096] The shared lock dynamic allocator 203 is configured to allocate one or more locks to each thread corresponding to the memory page accessed based on the memory page accessed by each thread's memory access request, so that all threads that initiate memory access requests can access the GPU memory module 104 according to their respective allocated locks.

[0097] In some embodiments, the shared lock dynamic allocator 203 is configured to allocate one or more locks corresponding to the memory pages accessed by each thread based on the page numbers of the memory pages accessed by each thread's memory access requests.

[0098] In this embodiment of the invention, multiple threads allocated to a lock can perform operations to access the GPU memory module according to the lock in parallel.

[0099] This invention employs a shared lock dynamic allocation mechanism, dynamically allocating locks based on page numbers. While ensuring "consistency in memory access within the same page," it allows for parallel execution of memory accesses to different pages, significantly improving the parallelism of multi-threaded memory access and thus enhancing the simulation efficiency of GPU memory access simulation. This invention addresses the performance degradation caused by lock contention when multiple threads access different pages; however, lock contention still occurs when multiple threads access the same page. But due to the unique nature of GPU Single Instruction Multiple Threads (SIMT) / Single Instruction Multiple Data (SIMD) programming models, memory access tasks on the same page are typically issued sequentially by the same thread, thus the impact is negligible.

[0100] In some embodiments, the shared lock dynamic allocator 203 includes an object pool that maintains a mapping relationship between page numbers of memory pages and locks; wherein, in the mapping relationship, at least one of the following situations exists: one lock corresponds to one page number of a memory page, and one lock corresponds to multiple page numbers of a memory page. That is, the following situations may exist: (1) in the mapping relationship, one lock corresponds to one page number of a memory page, in which case one memory page uses one lock, and the number of locks allocated to a thread is the same as the number of memory pages accessed based on the memory access request of the thread; (2) in the mapping relationship, one lock corresponds to multiple page numbers of a memory page, in which case multiple memory pages share one lock; (3) in the mapping relationship, there is both one lock corresponding to one page number of a memory page and one lock corresponding to multiple page numbers of a memory page, in which case one memory page uses one lock and multiple memory pages share one lock.

[0101] The shared lock dynamic allocator 203 is configured to: for each thread that initiates a memory access request, determine the target lock corresponding to the page number of each page of the memory page accessed by the thread based on the mapping relationship.

[0102] In some embodiments, the mapping relationship is stored based on a hash table, where the key is the page number and the value is a pointer to the lock. The lock pointer points to an object instance, which is an address space allocated in the GPU memory module.

[0103] In some embodiments, the shared lock dynamic allocator 203 is further configured to:

[0104] Simultaneously manage the states of multiple locks; the states of the locks include used and unused.

[0105] In response to the fact that the target lock corresponding to any page number in the memory page accessed by the thread that initiated the memory access request is in an unused state, the unused target lock is allocated to the thread so that the thread can access the corresponding memory page based on the unused target lock.

[0106] In some embodiments, the shared lock dynamic allocator 203 is further configured to:

[0107] The Least Recently Used (LRU) algorithm is used for lock state management; the lock state includes used and unused.

[0108] Update the lock state in the LRU algorithm's hash table after allocating the lock.

[0109] In other embodiments, the shared lock dynamic allocator 203 is also configured to:

[0110] The lock state management is performed using the Approximate Least Recently Used (PLRU) algorithm; the lock state includes used and unused.

[0111] Update the lock state in the hash table of the PLRU algorithm after allocating the lock.

[0112] In this embodiment of the invention, the PLRU (Pseudo-LRU) algorithm is an approximate LRU cache replacement strategy based on a binary tree structure, which achieves efficient hardware resource utilization by reducing state storage overhead.

[0113] It should be noted that, in addition to the LRU or PLRU algorithm, other algorithms can be used to manage the lock state. The LRU or PLRU algorithm here is mainly used to mark the most recently used address. Any algorithm with this function can be used, and this embodiment of the invention does not limit it.

[0114] In some embodiments, the shared lock dynamic allocator 203 is further configured to:

[0115] In response to the number of objects in the object pool exceeding a preset threshold, a target object is destroyed according to a preset strategy. The target object includes a key-value pair and an object instance pointed to by a pointer to a lock determined based on the key-value pair. The object instance is an address space allocated in the GPU memory module.

[0116] The preset strategies include recent usage and access frequency. In this embodiment of the invention, in the recycling process, in addition to destroying the key-value pair, the object instance pointed to by the value in the key-value pair (i.e., the lock pointer) is also destroyed. The object instance refers to the address space allocated in memory.

[0117] For example, after memory access is completed, if the number of objects in the object pool exceeds a preset threshold, the least recently used target object can be destroyed using the LRU or PLRU algorithm. The lock is automatically released after memory access. During the lock reclamation phase, the object can only be destroyed after the lock on the object being destroyed is released. In the reclamation phase, the time complexity for destroying an object using the LRU or PLRU algorithm is O(1).

[0118] In some embodiments, the shared lock dynamic allocator 203 is configured as follows:

[0119] In the mapping relationship, check if there is a lock corresponding to each page number of the memory page accessed by the memory access request;

[0120] In response to finding the lock corresponding to the first page number among all page numbers, one or more of the found locks are identified as the target lock corresponding to the first page number; the first page number is the page number of the memory page accessed corresponding to the memory access request.

[0121] In some embodiments, the shared lock dynamic allocator is further configured to:

[0122] If the lock corresponding to the first page number cannot be found in the page numbers of each page, a new lock corresponding to the first page number is generated.

[0123] Register the relationship between the first page number and the new lock in the object pool, and update the storage in the mapping relationship after successful registration;

[0124] Based on the updated mapping, the new lock corresponding to the first page number is determined as the target lock corresponding to the first page number.

[0125] The first page number generally refers to the page number of the memory page that the memory access request determines. It may be one page or multiple pages.

[0126] To maximize the parallelism of memory access across different pages and improve the simulation efficiency of GPU memory access simulation, in some embodiments, the object pool contains a hash table where the key is the page number and the value is a pointer to a lock; one pointer to a lock in the hash table corresponds to only one page number of a memory page; the pointer to the lock points to the address space allocated in the GPU memory module.

[0127] The shared lock dynamic allocator 203 is configured as follows:

[0128] The hash table is searched to see if pointers to the locks corresponding to the page numbers of the memory pages accessed by the memory access request exist.

[0129] In response to the query of the page number of the accessed memory page and the corresponding lock pointer, the lock pointer corresponding to the page number of the accessed memory page is determined as the target lock pointer corresponding to the page number of the accessed memory page.

[0130] In response to the fact that no corresponding lock pointer is found for at least one page number in the accessed memory page, a new lock pointer is generated for each of the at least one page number, and the relationship between all the page numbers in the at least one page number and their respective new lock pointers is registered in the object pool. After successful registration, the relationship is updated and stored in the hash table. Based on the updated hash table, the new lock pointers corresponding to each page number in the at least one page number are determined as the pointers to the target locks corresponding to each page number in the at least one page number.

[0131] The pointers to the target locks corresponding to the page numbers of the accessed memory pages are assigned to the thread that initiated the memory access request, so that the thread can access the corresponding memory pages according to the pointers to the target locks.

[0132] It is understood that in this embodiment, the number of lock pointers allocated to a thread is the same as the total number of memory pages accessed based on the thread's memory access request; for example, 5 pages require 5 pointers.

[0133] In this embodiment of the invention, when the LRU algorithm is used for lock state management, the lock state in the LRU algorithm's hash table is updated after lock allocation. The complexity of each update is O(N). Since multiple locks may be updated simultaneously, the update complexity of the LRU algorithm is O(N×M), where N is the object pool size and M is the current number of pages required. When the PLRU algorithm is used for lock state management, the lock state in the PLRU algorithm's hash table is updated after lock allocation. The complexity of each update is O(N). Since multiple locks may be updated simultaneously, the update complexity of the PLRU algorithm is O(N×M), where N is the object pool size and M is the current number of pages required.

[0134] This invention uses a shared lock dynamic allocation mechanism, where each lock in the maintained hash table corresponds to only one page number. Based on the page number, locks are dynamically allocated, ensuring "consistency of memory access within the same page" while maximizing "parallelism of memory access across different pages", thereby improving the simulation efficiency of GPU memory access simulation.

[0135] This invention utilizes a shared lock dynamic allocation mechanism, dynamically allocating locks based on page numbers. This allows for parallel execution of memory accesses to different pages, thereby improving the simulation efficiency of GPU memory access simulations. However, if each lock corresponds to only one page number, it may result in an excessive number of locks and high lock management overhead, thus reducing the performance of multi-threaded memory access. Therefore, a balance needs to be found between lock contention and lock management overhead. If, when the same thread accesses contiguous memory addresses, multiple memory pages corresponding to those contiguous addresses can share a single lock, this reduces the number of locks without affecting multi-threaded access to different pages. Therefore, this invention provides another embodiment.

[0136] In other embodiments, the object pool contains a hash table where the key is a page number and the value is a pointer to a lock. In this hash table, one lock corresponds to at least one page number of a memory page. There is a special case where the accessed memory pages are multiple pages. In this case, one pointer to a lock in the hash table corresponds to multiple page numbers of the memory pages, that is, multiple memory pages share one lock.

[0137] The shared lock dynamic allocator is configured as follows:

[0138] The hash table is searched to see if pointers to the locks corresponding to the page numbers of the memory pages accessed by the memory access request exist.

[0139] In response to the query of the page number of the accessed memory page and the corresponding lock pointer, the lock pointer corresponding to the page number of the accessed memory page is determined as the target lock pointer corresponding to the page number of the accessed memory page.

[0140] In response to the fact that no corresponding lock pointer is found for at least one page number in the accessed memory page, a new shared lock pointer is generated for the at least one page number, and the relationship between all page numbers in the at least one page number and the pointer of the new shared lock is registered in the object pool. After successful registration, the hash table is updated and stored. Based on the updated hash table, the pointer of the new shared lock is determined as the pointer of the target lock corresponding to each page number in the at least one page number.

[0141] The pointers to the target locks corresponding to the page numbers of the accessed memory pages are assigned to the thread that initiated the memory access request, so that the thread can access the corresponding memory pages according to the pointers to the target locks.

[0142] It is understood that in this embodiment, if the accessed memory page is a multi-page, the number of lock pointers allocated to the thread is different from the total number of memory pages accessed based on the thread's memory access request. The number of pointers is related to the mapping relationship stored in the hash table. Multiple memory pages can share a lock pointer. For example, memory page 1, memory page 2, and memory page 3 share the lock pointer 1.

[0143] In another embodiment of the present invention, when the LRU algorithm is used for lock state management, the update complexity of updating the lock state in the hash table of the LRU algorithm after allocating the lock is O(1); when the PLRU algorithm is used for lock state management, the update complexity of updating the lock state in the hash table of the PLRU algorithm after allocating the lock is O(1).

[0144] The GPU memory access simulation system embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of the present invention according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0145] Based on the same inventive concept, such as Figure 3 As shown, this embodiment of the invention also provides a GPU memory access simulation method, which is applied to the above-described GPU memory access simulation system architecture. The method may include the following steps:

[0146] S101. In response to any thread in the GPU multi-threaded simulation module initiating a memory access request, determine the memory page to be accessed based on the memory access request;

[0147] S102. Based on the memory pages accessed by each thread's memory access request, allocate one or more locks corresponding to the memory pages accessed by each thread, so that all threads that initiate memory access requests can access the GPU memory module according to the locks they are allocated.

[0148] In some embodiments, the number of locks allocated to a thread is the same as the number of memory pages accessed, determined based on the thread's memory access request.

[0149] In some embodiments, multiple threads allocated to a lock can perform operations that access the GPU memory module based on the lock in parallel.

[0150] In some embodiments, the memory access request carries the following information: the initial virtual address for memory access and the length required for memory access;

[0151] Step S101, which determines the memory page to be accessed based on the memory access request, includes:

[0152] The starting page number corresponding to the memory page being accessed is calculated based on the initial virtual address of the memory access.

[0153] Calculate the total number of pages corresponding to the memory pages accessed based on the required memory access length;

[0154] Calculate the page number of each page corresponding to the accessed memory page based on the starting page number and the total number of pages;

[0155] In step S102, based on the memory pages accessed by each thread's memory access request, one or more locks are allocated to each thread corresponding to the accessed memory pages, including:

[0156] Based on the page number of the memory page accessed by each thread, one or more locks corresponding to the memory page accessed are allocated to each thread.

[0157] In some embodiments, the shared lock dynamic allocator includes an object pool that stores a mapping relationship between page numbers of memory pages and locks; wherein, in the mapping relationship, at least one of the following conditions exists: one lock corresponds to one page number of a memory page, or one lock corresponds to multiple page numbers of a memory page;

[0158] The memory pages accessed based on the memory access requests of each thread are allocated one or more locks corresponding to the accessed memory pages for each thread, including:

[0159] For each thread that initiates a memory access request, based on the mapping relationship, the target lock corresponding to the page number of each page of the memory page accessed by the thread is determined.

[0160] In some embodiments, determining the target lock corresponding to each page number of the memory page accessed by the thread based on the mapping relationship includes:

[0161] In the mapping relationship, check if there is a lock corresponding to each page number of the memory page accessed by the memory access request;

[0162] In response to finding the lock corresponding to the first page number among all page numbers, one or more of the found locks are identified as the target lock corresponding to the first page number; the first page number is the page number of the memory page accessed corresponding to the memory access request.

[0163] In some embodiments, determining the target lock corresponding to each page number of the memory page accessed by the thread based on the mapping relationship further includes:

[0164] If the lock corresponding to the first page number cannot be found in the page numbers of each page, a new lock corresponding to the first page number is generated.

[0165] Register the relationship between the first page number and the new lock in the object pool, and update the storage in the mapping relationship after successful registration;

[0166] Based on the updated mapping, the new lock corresponding to the first page number is determined as the target lock corresponding to the first page number.

[0167] In some embodiments, the mapping relationship is stored based on a hash table, where the key is the page number and the value is a pointer to the lock.

[0168] The following example illustrates the processing flow of a GPU memory access simulation method provided in this embodiment of the invention, using one lock per memory page as an example. Figure 4 As shown, the shared lock dynamic allocator includes an object pool containing a hash table. The keys of the hash table are page numbers, and the values ​​are pointers to locks. In this hash table, one lock pointer corresponds to only one page number of a memory page. The method may include the following steps:

[0169] S401. Any thread in the GPU multi-threaded simulation module initiates a memory access request, and the memory access request carries the following information: the initial virtual address of the memory access and the length required for the memory access.

[0170] S402, the page calculation module calculates the starting page number corresponding to the memory page to be accessed based on the initial virtual address of the memory access; calculates the total number of pages corresponding to the memory page to be accessed based on the required length of the memory access; and calculates the page number of each page corresponding to the memory page to be accessed based on the starting page number and the total number of pages.

[0171] For each thread that initiates a memory access request, perform the following steps:

[0172] S403. The shared lock dynamic allocator checks the hash table to see if there are pointers to the locks corresponding to the page numbers of the accessed memory pages;

[0173] If pointers to locks corresponding to all page numbers in the accessed memory pages are found, then step S404 is executed. If no pointer to a corresponding lock is found for at least one page number in the accessed memory pages, then steps S404 and S405 are executed.

[0174] S404. The shared lock dynamic allocator, in response to the page number of the accessed memory page, finds the corresponding lock pointer and determines the lock pointer corresponding to the page number of the accessed memory page as the pointer of the target lock corresponding to the page number of the accessed memory page.

[0175] S405. In response to the fact that no corresponding lock pointer is found for at least one page number in the accessed memory pages, the shared lock dynamic allocator generates a corresponding new lock pointer for each of the at least one page number, and registers the relationship between all the page numbers in the at least one page number and their respective new lock pointers in the object pool. After successful registration, the relationship is updated and stored in the hash table. Based on the updated hash table, the new lock pointers corresponding to each page number in the at least one page number are determined as the pointers of the target locks corresponding to each page number in the at least one page number.

[0176] S406. The shared lock dynamic allocator allocates the pointers to the target locks corresponding to the page numbers of the accessed memory pages to the thread that initiated the memory access request.

[0177] S407. The thread accesses the corresponding memory page in the GPU memory module based on the pointer of the target lock corresponding to the page number of each accessed memory page.

[0178] After performing the above operations, all threads that initiate memory access requests can access the GPU memory module according to the locks they are allocated.

[0179] The following example illustrates another GPU memory access simulation method provided in this embodiment of the invention, using the example of at least one memory page sharing a single lock. Figure 5 As shown, the shared lock dynamic allocator includes an object pool containing a hash table where the keys are page numbers and the values ​​are pointers to locks. A pointer to a lock in this hash table corresponds to at least one page number of a memory page. The method may include the following steps:

[0180] S501, Any thread in the GPU multi-threaded simulation module initiates a memory access request, and the memory access request carries the following information: the initial virtual address of the memory access and the length required for the memory access;

[0181] S502, the page calculation module calculates the starting page number corresponding to the memory page to be accessed based on the initial virtual address of the memory access; calculates the total number of pages corresponding to the memory page to be accessed based on the required length of the memory access; and calculates the page number of each page corresponding to the memory page to be accessed based on the starting page number and the total number of pages.

[0182] For each thread that initiates a memory access request, perform the following steps:

[0183] S503, the shared lock dynamic allocator checks the hash table to see if there are pointers to the locks corresponding to the page numbers of the accessed memory pages;

[0184] If pointers to locks corresponding to all page numbers in the accessed memory pages are found, then step S404 is executed. If no pointer to a corresponding lock is found for at least one page number in the accessed memory pages, then steps S404 and S405 are executed.

[0185] S504. The shared lock dynamic allocator, in response to the page number of the accessed memory page, finds the corresponding lock pointer and determines the lock pointer corresponding to the page number of the accessed memory page as the target lock pointer corresponding to the page number of the accessed memory page.

[0186] S505. In response to the fact that no corresponding lock pointer is found for at least one page number in the accessed memory page, the shared lock dynamic allocator generates a corresponding shared new lock pointer for the at least one page number, and registers the relationship between all page numbers in the at least one page number and the shared new lock pointer in the object pool. After successful registration, it updates and stores the relationship in the hash table. Based on the updated hash table, the shared new lock pointer is determined as the pointer of the target lock corresponding to each page number in the at least one page number.

[0187] S506. The shared lock dynamic allocator allocates the pointers to the target locks corresponding to the page numbers of the accessed memory pages to the thread that initiated the memory access request.

[0188] S507. This thread accesses the corresponding memory page in the GPU memory module based on the pointer of the target lock corresponding to the page number of each accessed memory page.

[0189] After performing the above operations, all threads that initiate memory access requests can access the GPU memory module according to the locks they are allocated.

[0190] In some embodiments, the method further includes:

[0191] Simultaneously manage the states of multiple locks; the states of the locks include used and unused.

[0192] In response to the fact that the target lock corresponding to any page number in the memory page accessed by the thread that initiated the memory access request is in an unused state, the unused target lock is allocated to the thread so that the thread can access the corresponding memory page based on the unused target lock.

[0193] In some embodiments, the method further includes:

[0194] In response to the number of objects in the object pool exceeding a preset threshold, a target object is destroyed according to a preset strategy. The target object includes a key-value pair and an object instance pointed to by a pointer to a lock determined based on the key-value pair. The object instance is an address space allocated in the GPU memory module.

[0195] As the method embodiments are basically corresponding to the system embodiments, the relevant parts can be referred to in the description of the system embodiments, and will not be repeated here.

[0196] Based on the same inventive concept, embodiments of the present invention also provide a storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the GPU memory access simulation method in any of the above possible implementations.

[0197] Optionally, the storage medium may be a non-transitory computer-readable storage medium, such as a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device.

[0198] Based on the same inventive concept, embodiments of the present invention also provide a computer program product, including a computer program that, when executed by a processor, implements the steps of the GPU memory access simulation method in any of the above possible implementations.

[0199] Based on the same inventive concept, see [link to inventive concept] Figure 6 This invention also provides an electronic device, including a memory 71 (e.g., non-volatile memory), a processor 72, and a computer program stored on the memory 71 and executable on the processor 72. When the processor 72 executes the program, it implements the steps of the GPU memory access simulation method in any of the above possible implementations, which is equivalent to the GPU memory access simulation system described above. Of course, the processor can also be used to process other data or perform calculations. This electronic device can be a PC, server, terminal, or other similar device.

[0200] like Figure 6 As shown, the electronic device may also include: memory 73, network interface 74, and internal bus 75. Other hardware may also be included besides these components, which will not be described in detail here.

[0201] It should be noted that the aforementioned GPU memory access simulation system can be implemented in software. As a logical system, it is formed by the processor 72 of the electronic device in which it resides reading the computer program instructions stored in the non-volatile memory into the memory 73 for execution.

[0202] The embodiments of the subject matter and functional operation described in this specification can be implemented in the following ways: digital electronic circuits, tangibly embodied computer software or firmware, computer hardware including the structures disclosed in this specification and their structural equivalents, or combinations thereof. Embodiments of the subject matter described in this specification can be implemented as one or more computer programs, i.e., one or more modules of computer program instructions encoded on a tangible, non-transitory program carrier for execution by a data processing apparatus or for controlling the operation of a data processing apparatus. Alternatively or additionally, the program instructions may be encoded on artificially generated propagation signals, such as machine-generated electrical, optical, or electromagnetic signals, which are generated to encode information and transmit it to a suitable receiving device for execution by the data processing apparatus. The computer storage medium may be a machine-readable storage device, a machine-readable storage substrate, a random or serial access memory device, or combinations thereof.

[0203] The processing and logic flow described in this specification can be executed by one or more programmable computers that execute one or more computer programs to perform corresponding functions by operating on input data and generating output. The processing and logic flow can also be executed by dedicated logic circuitry—such as FPGAs (Field-Programmable Gate Arrays) or ASICs (Application-Specific Integrated Circuits), and the device can also be implemented as dedicated logic circuitry.

[0204] Suitable computers for executing computer programs include, for example, general-purpose and / or special-purpose microprocessors, or any other type of central processing unit. Typically, the central processing unit receives instructions and data from read-only memory and / or random access memory. The basic components of a computer include a central processing unit for implementing or executing instructions and one or more memory devices for storing instructions and data. Typically, a computer will also include one or more mass storage devices for storing data, such as disks, magneto-optical disks, or optical disks, or the computer will be operatively coupled to such mass storage devices to receive data from or transfer data to them, or both. However, a computer is not required to have such devices. Furthermore, a computer can be embedded in another device, such as a mobile phone, a personal digital assistant (PDA), a mobile audio or video player, a game console, a global positioning system (GPS) receiver, or a portable storage device such as a universal serial bus (USB) flash drive, to name a few.

[0205] Computer-readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media, and memory devices, such as semiconductor memory devices (e.g., EPROM, EEPROM, and flash memory devices), magnetic disks (e.g., internal hard disks or removable disks), magneto-optical disks, and CD-ROM and DVD-ROM disks. Processors and memory may be supplemented by or incorporated into dedicated logic circuitry.

[0206] While this specification contains numerous specific implementation details, these should not be construed as limiting the scope of any invention or the scope of the claims, but rather are primarily intended to describe features of specific embodiments of a particular invention. Certain features described in the various embodiments herein may also be implemented in combination in a single embodiment. Conversely, various features described in a single embodiment may also be implemented separately in various embodiments or in any suitable sub-combination. Furthermore, while features may function in certain combinations as described above and even initially claimed in this way, one or more features from a claimed combination may be removed from that combination in some cases, and a claimed combination may refer to a sub-combination or a variation thereof.

[0207] Similarly, although the operations are depicted in a specific order in the accompanying drawings, this should not be construed as requiring these operations to be performed in the specific order shown or sequentially, or requiring all illustrated operations to be performed to achieve the desired result. In some cases, multitasking and parallel processing may be advantageous. Furthermore, the separation of various system modules and components in the above embodiments should not be construed as requiring such separation in all embodiments, and it should be understood that the described program components and systems can generally be integrated together in a single software product or packaged into multiple software products.

[0208] Thus, specific embodiments of the subject matter have been described. Other embodiments are within the scope of the appended claims. In some cases, the actions recited in the claims may be performed in a different order and still achieve the desired result. Furthermore, the processes depicted in the drawings are not necessarily shown in a specific order or sequence to achieve the desired result. In some implementations, multitasking and parallel processing may be advantageous.

[0209] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0210] The above description is merely a specific embodiment of the present invention, enabling those skilled in the art to understand or implement the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the present invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features claimed herein.

Claims

1. A GPU memory access simulation system, comprising: The system comprises: a GPU multi-thread simulation module, a page calculation module, a shared lock dynamic allocator and a GPU memory module; The page calculation module is configured to determine the accessed memory page based on a memory access request initiated by any one thread in the GPU multi-thread simulation module; The shared lock dynamic allocator is configured to allocate one or more locks corresponding to the accessed memory page to each thread based on the accessed memory page corresponding to the memory access request of each thread, so that all threads initiating the memory access request can access the GPU memory module according to the locks allocated to each thread; The memory access request carries the following information: a memory access initial virtual address and a memory access required length; The page calculation module is configured to: calculate the starting page code corresponding to the accessed memory page according to the memory access initial virtual address; calculate the total number of pages corresponding to the accessed memory page according to the memory access required length; calculate the page codes of each page corresponding to the accessed memory page according to the starting page code and the total number of pages; The shared lock dynamic allocator is configured to: allocate one or more locks corresponding to the accessed memory page to each thread based on the page codes of each page of the accessed memory page corresponding to the memory access request of each thread.

2. The system of claim 1, wherein, The number of locks allocated to the thread is the same as the number of pages of the accessed memory page determined based on the memory access request of the thread.

3. The system of claim 1, wherein, The multiple threads allocated to the locks can perform the operation of accessing the GPU memory module according to the locks in parallel.

4. The system of claim 1, wherein, The shared lock dynamic allocator comprises an object pool storing a mapping relationship between the page codes of the memory page and the locks; wherein, in the mapping relationship, at least one of the following situations exists: one lock corresponds to one page code of the memory page, and one lock corresponds to multiple page codes of the memory page. The shared lock dynamic allocator is configured to: for each thread initiating the memory access request, determine the target lock corresponding to the page codes of the accessed memory page corresponding to the memory access request of the thread based on the mapping relationship.

5. The system of claim 4, wherein, The shared lock dynamic allocator is configured to: query whether the lock corresponding to the page codes of the accessed memory page corresponding to the memory access request exists in the mapping relationship; in response to querying the lock corresponding to the first page code in the page codes, determine the queried one or more locks as the target lock corresponding to the first page code; the first page code is the page code of the accessed memory page corresponding to the memory access request.

6. The system of claim 5, wherein, The shared lock dynamic allocator is further configured to: in response to the first page code corresponding to the lock not being queried in the page codes, generate a new lock corresponding to the first page code; register the relationship between the first page code and the new lock in the object pool, and update the storage to the mapping relationship after successful registration; based on the updated mapping relationship, determine the new lock corresponding to the first page code as the target lock corresponding to the first page code.

7. The system according to any of claims 1-6, characterized in that, The shared lock dynamic allocator is further configured to: manage the states of multiple locks at the same time; the states of the locks include used and unused. In response to a state of a target lock corresponding to any page code of a memory page accessed by a thread initiating a memory access request being unused, an unused target lock is allocated to the thread, so that the thread accesses the corresponding memory page according to the unused target lock.

8. The system of claim 4, wherein, The mapping relationship is stored based on a hash table, a key of the hash table being a page code and a value being a pointer of a lock.

9. The system of claim 8, wherein, The shared lock dynamic allocator is further configured to: In response to a number of objects in the object pool exceeding a preset threshold, a target object is destroyed by a preset strategy, the target object including a key-value pair and an object instance pointed to by a pointer of a lock determined based on the key-value pair, the object instance being an address space allocated in a GPU memory module.

10. A method of GPU memory access simulation, the method comprising: The method comprises: In response to any one of threads in a GPU multi-thread simulation module initiating a memory access request, a memory page accessed based on the memory access request is determined; Based on the memory page accessed corresponding to the memory access request of each thread, one or more locks corresponding to the memory page accessed are respectively allocated to each thread, so that all threads initiating the memory access request can access the GPU memory module according to the locks respectively allocated thereto; The memory access request carries the following information: a memory initial virtual address and a memory required length; The determination of the memory page accessed based on the memory access request comprises: The start page code corresponding to the memory page accessed is calculated according to the memory initial virtual address; The total number of pages corresponding to the memory page accessed is calculated according to the memory required length; Each page code corresponding to the memory page accessed is calculated according to the start page code and the total number of pages. Based on the page code corresponding to the memory page accessed corresponding to the memory access request of each thread, one or more locks corresponding to the memory page accessed are respectively allocated to each thread. The program is executed by the processor to implement the steps of the method of claim 10.

11. A storage medium having stored thereon a computer program, characterized in that The processor executes the program to implement the steps of the method of claim 10.

12. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, ​

Images