A cache warming method when a GPU makes a memory application

Abstract

The application discloses a cache preheating method when a GPU applies for memory, relates to a memory management technology, and aims at the problem that the prior art excessively depends on an OS. The specific method is that a virtual address is sent to an MMU as request information for accessing the MMU to be converted into a physical address, then the MMU's page table traversal request is intercepted by a memory management assisting module, so that the physical address is correctly returned and a correct mapping is established. The method has the advantages that the MMU internal cache and the TLB are preheated without increasing additional overhead, and then when the GPU actually writes data, the TLB is directly hit, multiple memory access operations are avoided, and significant performance improvement is brought.

Description

Technical Field

[0001] This invention relates to memory management technology, and more particularly to a cache warm-up method when a GPU requests memory. Background Technology

[0002] As the performance of Graphics Processing Units (GPUs) continues to improve, their workload during image rendering is also increasing. Image rendering is a core technology of traditional GPUs. Due to the increasing workload during rendering, the GPU generates a large amount of intermediate data, which needs to be temporarily stored for later use. However, because the data volume is enormous, it cannot be completely stored in the GPU's internal cache. Therefore, the data needs to be written back to external memory for later retrieval. Since writing to external memory is necessary, this process requires establishing a mapping between virtual addresses and physical addresses so that the GPU can correctly access external memory.

[0003] In traditional systems, when newly generated rendering data needs to be written back to external memory, the GPU must request memory allocation through the Operating System (OS). The OS then allocates the memory and establishes the mapping relationship. This process requires communication with the OS and direct intervention from the OS, resulting in a significant performance penalty.

[0004] The current popular approach is to implement the memory allocation and mapping process through newly added hardware modules inside the GPU, reducing dependence on the OS and thus improving GPU performance, but there is still room for further optimization. Summary of the Invention

[0005] The purpose of this invention is to provide a cache preheating method during GPU memory allocation, aiming to further improve the performance of GPU internal hardware in establishing virtual address-to-physical address mapping and subsequent access. The proposed scheme can preheat the Memory Management Unit (MMU) and Translation Lookaside Buffer (TLB) caches simultaneously with GPU memory allocation, thereby ensuring cache hits in subsequent accesses and significantly improving system performance.

[0006] The cache preheating method for GPU memory allocation described in this invention sends the virtual address as a request to the MMU in advance for virtual address to physical address conversion. Then, the memory management assistance module intercepts the page table traversal request of the MMU so that the physical address is returned correctly and the correct mapping is established.

[0007] Specifically, the steps include the following:

[0008] S1. Obtain the mapping relationship establishment request of the memory management module, and parse the virtual address and physical address from the request;

[0009] S2. The resolved virtual address is sent to the MMU as a request for accessing the MMU to perform virtual address to physical address translation;

[0010] After internal cache traversal, S3.MMU determines that the TLB and MMU caches are missing. After the MMU internally determines that the caches are missing, it initiates a page table traversal request to obtain the intermediate information of each level of page table and the final mapping relationship.

[0011] S4. Since the mapping relationship corresponding to the virtual address carried by this request still exists inside the memory management assistance module, the MMU's page table traversal request will be intercepted by the memory management assistance module.

[0012] S5. After the memory management assistance module intercepts the request, it will determine whether the current page table traversal request is the last level page table. If it is the last level page table traversal request, it will return the physical address information of the mapping relationship that needs to be established this time; if it is not the last level page table traversal request, it will return the physical address sent by the aforementioned memory management module for establishing the page table; the memory allocation and the corresponding mapping relationship establishment have been completed.

[0013] S6. When the GPU has a request for a virtual address that arrives at the MMU for address translation, the address translation request will be directly hit and returned in the TLB.

[0014] In step S1, the memory management assistance module first determines whether the current mapping relationship establishment request involves the creation of a new page table. If not, it proceeds to step S2; if so, it performs the following processing:

[0015] Initiate a request to the memory management module for a physical address to be used for subsequent page table creation; wait for the memory management module to return the physical address used to create the page table.

[0016] When the system involves multi-level page tables, physical addresses are requested multiple times until there are enough to build the multi-level page tables.

[0017] In step S5, in addition to returning the corresponding physical address to the MMU, a memory access operation is also initiated to write the same physical address information.

[0018] The physical address includes the physical address used to establish the page table and the physical address used to establish the mapping relationship.

[0019] The cache preheating method for GPU memory allocation described in this invention has the advantage of preheating the MMU internal cache and TLB with almost no additional overhead. As a result, when the GPU actually writes data, it will directly hit the TLB, avoiding multiple memory access operations and bringing significant performance improvement. Attached Figure Description

[0020] Figure 1 This is a flowchart illustrating the cache preheating method described in this invention.

[0021] Figure 2 This is a schematic diagram of the system structure for applying the cache preheating method described in this invention.

[0022] Figure 3 This is a schematic diagram of the structure consisting of a four-level page table index for virtual addresses. Detailed Implementation

[0023] like Figures 1 to 2 As shown, the specific process of the cache preheating method for GPU memory allocation described in this invention includes the following steps:

[0024] ① The memory management module initiates a mapping establishment request, requesting the establishment of a mapping relationship between virtual address A` and physical address A. The page size pointed to by the virtual address and physical address is configurable, supporting common page sizes such as 4KB and 8KB.

[0025] ② The memory management assistance module parses the request and identifies whether a new page table needs to be created, and from which level to start, based on the page table indexes of the virtual address to be mapped. Each level of page table creation requires a physical address, referred to hereafter as the table creation physical address. Subsequently, the memory management assistance module requests the corresponding number of physical addresses from the memory management module. If the memory management assistance module identifies the need to create new first-level and second-level page tables, it will initiate a table creation physical address request to the memory management module.

[0026] In the currently popular four-level page table system, the virtual address is composed as follows: Figure 3As shown, the configuration is applicable to most architectures, and the virtual address width will vary depending on the system's storage space. In a four-level page table system, a completely new page table structure requires four physical addresses to build the entire page table hierarchy. The following describes how the memory management assistance module determines whether a new page table needs to be created. The logic is straightforward: starting with the index of the first-level page table, it checks if the index value of this level is all zeros. If the first-level page table index is all zeros, it indicates that a new first-level page table needs to be created. After identifying the need to create a new first-level page table, the same logic is used to determine whether a new second-level page table needs to be created, and so on, up to the highest-level page table.

[0027] ③ The memory management module returns the corresponding number of free physical addresses. In the example above, physical address B is returned to establish the first-level page table, and physical address C is returned to establish the second-level page table.

[0028] ④ After receiving the corresponding number of physical addresses for page table creation, the memory management assistance module will temporarily store them in its local register. The local register also contains the physical address A used to establish the mapping. Next, it will send an address translation request to the MMU for virtual address A'. At this point, the page table information is not directly written back to memory; this is to warm up the MMU's cache, thereby ensuring a direct hit for subsequent requests.

[0029] ⑤ For the translation of virtual address A', the MMU cache will inevitably miss, so a memory access operation for page table traversal will be issued. For cases where the first-level page table and the second-level page table need to be established, a memory access request for the higher-level page table, i.e., a traversal request for the second-level page table, will be issued first.

[0030] ⑥ Since the mapping relationship corresponding to the virtual address carried in this request still exists inside the memory management assistance module, the MMU's page table traversal request will be intercepted by the memory management assistance module. The memory access operation is intercepted by the memory management assistance module, which returns the physical address C for table creation allocated by the memory management module and stored in the registers inside the memory management assistance module, and returns the physical address A used to establish the mapping in the last memory access operation.

[0031] During each return of the physical address to the MMU, the memory management assistance module needs to simultaneously write the corresponding physical address information back to the page table in memory as a backup. The page table in memory is the complete page table system; the page table in the MMU cache is only a small part of the memory page table.

[0032] ⑦ After receiving the physical address A used to establish the mapping, the MMU responds that the operation is complete and returns the physical address A where the mapping relationship has been established.

[0033] ⑧ At approximately the same time, the GPU rendering module will initiate a memory access operation to write intermediate rendering information back to memory. This requires the MMU to help translate virtual addresses to physical addresses and then access memory. At this point, the virtual address corresponding to the request happens to be the virtual address A' that has just been mapped.

[0034] ⑨ Since the MMU cache and TLB warm-up have been implemented in the above process, the request for virtual address A' hits the TLB and directly returns the physical address A that has just been mapped.

[0035] For those skilled in the art, various other corresponding changes and modifications can be made based on the technical solutions and concepts described above, and all such changes and modifications should fall within the protection scope of the claims of this invention.

Claims

1. A cache warm-up method for GPU memory allocation, characterized in that, The virtual address is sent to the MMU in advance as a request to access the MMU for the translation from virtual address to physical address. Then, the memory management assistance module intercepts the page table traversal request of the MMU so that the physical address is returned correctly and the correct mapping is established. Specifically, the steps include the following: S1. Obtain the mapping relationship establishment request of the memory management module, and parse the virtual address and physical address from the request; S2. The resolved virtual address is sent to the MMU as a request for accessing the MMU to perform virtual address to physical address translation; After internal cache traversal, S3.MMU determines that the TLB and MMU caches are missing. After the MMU internally determines that the caches are missing, it initiates a page table traversal request to obtain the intermediate information of each level of page table and the final mapping relationship. S4. Since the mapping relationship corresponding to the virtual address carried by this request still exists inside the memory management assistance module, the MMU's page table traversal request will be intercepted by the memory management assistance module. S5. After the memory management assistance module intercepts the request, it will determine whether the current page table traversal request is the last level page table. If it is the last level page table traversal request, it will return the physical address information of the mapping relationship that needs to be established this time; if it is not the last level page table traversal request, it will return the physical address sent by the aforementioned memory management module for establishing the page table; the memory allocation and the corresponding mapping relationship establishment have been completed. S6. When the GPU has a request for a virtual address that arrives at the MMU for address translation, the address translation request will be directly hit and returned in the TLB.

2. The cache preheating method for GPU memory allocation according to claim 1, characterized in that, In step S1, the memory management assistance module first determines whether the current mapping relationship establishment request involves the creation of a new page table. If not, it proceeds to step S2; if so, it performs the following processing: Initiate a request to the memory management module for a physical address to be used for subsequent page table creation; wait for the memory management module to return the physical address used to create the page table.

3. The cache preheating method for GPU memory allocation according to claim 2, characterized in that, When the system involves multi-level page tables, physical addresses are requested multiple times until there are enough to build the multi-level page tables.

4. The cache warm-up method for GPU memory allocation according to claim 1, characterized in that, In step S5, in addition to returning the corresponding physical address to the MMU, a memory access operation is also initiated to write the same physical address information.

5. The cache preheating method for GPU memory allocation according to claim 1, characterized in that, The physical address includes the physical address used to establish the page table and the physical address used to establish the mapping relationship.

Images