Enhanced GPGPU (General Purpose Graphics Processing Unit) thread bundle cache prefetching method and device

By configuring an independent training prefetcher for each thread bundle, analyzing and dynamically adjusting the memory access mode, and using the global stride table to share information, the memory system overload problem caused by GPGPU prefetch technology is solved, and the response speed of prefetch operations is improved and GPGPU system performance.

CN118916304APending Publication Date: 2024-11-08SHANDONG INSPUR SCI RES INST CO LTD
View PDF 0 Cites 3 Cited by

Patent Information

Application Number
CN202411201219.8
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-29
Publication Date
2024-11-08

AI Technical Summary

Technical Problem

When GPGPU thread-level parallelism and data reuse are insufficient, prefetching technology may cause memory system overload, increase latency, and reduce performance.

Method used

By configuring an independent training prefetcher for each thread bundle, recording stride information, monitoring and analyzing memory access patterns, dynamically adjusting prefetch policies, and using the global stride table to share information, reducing the dependence of prefetch requests.

Benefits of technology

Improve the response speed and efficiency of prefetch operations, identify regular access patterns, optimize memory access management, and improve GPGPU system performance.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118916304A_ABST
    Figure CN118916304A_ABST
Patent Text Reader

Abstract

The invention relates to the technical field of GPGPU internal computing processing, and discloses an enhanced GPGPU thread beam cache prefetching method and device, and the method comprises the steps: carrying out the core initialization, and initializing a stride prefetcher for each processor core; configuring an independent training prefetcher for each thread bundle; each thread bundle stride table is used for recording stride information of each thread bundle, and the thread bundle stride table stores stride size, access frequency and other related parameters and provides guidance for prefetching operation; when it is monitored that the multiple thread bundles have the same stride for the same program counter, stride information is sent to a global stride table; untrained thread bundles are allowed to issue prefetch requests using entries in the global stride table. By analyzing the memory access address and stride of each thread bundle, the regular access mode can be identified, the prefetching request is generated accordingly, and the response speed of the prefetching operation is increased.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the technical field of GPGPU internal computing processing, for example, to an enhanced GPGPU warp cache prefetching method and device. Background Art

[0002] GPGPU processors improve their parallel processing capabilities by supporting large-scale fine-grained multithreading and caching mechanisms. The design of modern GPGPU processors that can support a large number of threads makes GPGPU perform well in processing tasks with significant memory-level parallelism.

[0003] However, GPGPU performance may suffer when thread-level parallelism and data reuse are insufficient. Although prefetching can improve performance in theory, in practice, if used improperly, it may increase the burden on the memory system, resulting in more outstanding memory requests, thereby increasing the latency of servicing demand memory requests. This problem is particularly serious in GPGPU systems because GPGPU applications typically involve a large number of concurrent threads and memory requests. If each thread generates additional prefetch requests, it may overload the memory system and degrade performance.

[0004] It should be noted that the information disclosed in the above background technology section is only used to enhance the understanding of the background of the present application, and therefore may include information that does not constitute the prior art known to ordinary technicians in the field. Summary of the invention

[0005] In order to provide a basic understanding of some aspects of the disclosed embodiments, a brief summary is given below. The summary is not an extensive review, nor is it intended to identify key / critical elements or delineate the scope of protection of these embodiments, but rather serves as a prelude to the detailed description that follows.

[0006] The enhanced GPGPU warp cache prefetching method provided by the embodiment of the present disclosure includes: Perform core initialization and initialize a stride prefetcher for each processor core; Configure independent training prefetchers for each warp; Using each warp stride table to record the stride information of each warp, wherein the warp stride table stores the stride size, access frequency and other related parameters to provide guidance for prefetch operations; When it is detected that multiple thread warps have the same stride for the same program counter, the stride information is sent to the global stride table; Allow untrained warps to issue prefetch requests using entries in the global stride table.

[0007] In some embodiments, the method further comprises: A least recently used replacement policy is applied in each warp stride table and the global stride table.

[0008] In some embodiments, the initialized stride prefetcher is trained based on the program counter to recognize and predict memory access patterns.

[0009] In some embodiments, each prefetcher configuration includes necessary initial parameters of stride size, prefetch distance, and prefetch degree.

[0010] In some embodiments, during the training process, an independent training prefetcher configured for each warp monitors the memory access pattern of each warp in real time, and detects a stride pattern involving analysis of memory access addresses and access frequencies to identify regularity of the access pattern.

[0011] In some embodiments, during the monitoring process, the data in each warp stride table is continuously updated to reflect the latest memory access pattern, so that the prefetcher dynamically adjusts the prefetch strategy according to the latest memory access pattern.

[0012] The enhanced GPGPU warp cache prefetching device provided by the embodiment of the present disclosure includes: An initialization module, used for core initialization, initializing a stride prefetcher for each processor core; A configuration module for configuring independent training prefetchers for each warp; A recording module, configured to record the stride information of each warp using a warp stride table, wherein the warp stride table stores the stride size, access frequency and other related parameters to provide guidance for the prefetch operation; A sending module, configured to send stride information to a global stride table when it is detected that multiple thread warps have the same stride for the same program counter; The sending module is further configured to allow untrained warps to issue pre-fetch requests using entries in the global stride table.

[0013] In some embodiments, the storage medium stores program instructions, wherein the program instructions, when running, execute the enhanced GPGPU warp cache prefetching method in the above embodiment.

[0014] The enhanced GPGPU warp cache prefetching method, device, and storage medium provided by the embodiments of the present disclosure can achieve the following technical effects: By analyzing the memory access address and stride of each warp, regular access patterns can be identified and prefetch requests can be generated accordingly. The stride information obtained by each warp training is stored in the stride (PWS) table of each warp, which is similar to the prefetch table in the traditional prefetcher, but is designed specifically for each warp, improving the pertinence and efficiency of the prefetch strategy. When multiple warps show the same stride for the same program counter (PC), the stride information is promoted to the global stride (GS) table. The promoted stride information allows untrained warps to immediately use the entries in the GS table to issue prefetch requests, reducing the dependence on the PWS table and the number of accesses, thereby improving the response speed of the prefetch operation.

[0015] The above general description and the following description are exemplary and explanatory only and are not intended to limit the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] One or more embodiments are exemplarily described by corresponding drawings, which do not limit the embodiments. Elements with the same reference numerals in the drawings are shown as similar elements, and the drawings do not constitute a scale limitation, and wherein: Figure 1 It is a flowchart of an enhanced GPGPU warp cache prefetching method provided by an embodiment of the present disclosure; Figure 2 is a GPGPU storage architecture diagram provided by an embodiment of the present disclosure; Figure 3 is a structural schematic diagram of an enhanced thread warp prefetcher provided by an embodiment of the present disclosure; Figure 4 It is a structural schematic diagram of an enhanced GPGPU warp cache prefetching device provided by an embodiment of the present disclosure; Figure 5 It is a structural schematic diagram of an enhanced GPGPU warp cache prefetching system provided by an embodiment of the present disclosure. DETAILED DESCRIPTION

[0017] In order to be able to understand the features and technical contents of the embodiments of the present disclosure in more detail, the implementation of the embodiments of the present disclosure is described in detail below in conjunction with the accompanying drawings. The attached drawings are for reference only and are not used to limit the embodiments of the present disclosure. In the following technical description, for the convenience of explanation, a full understanding of the disclosed embodiments is provided through multiple details. However, one or more embodiments can still be implemented without these details. In other cases, to simplify the drawings, well-known structures and devices can be simplified for display.

[0018] The terms "first", "second", etc. in the embodiments of the present disclosure are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the terms used in this way can be interchanged where appropriate, so as to describe the embodiments of the present disclosure described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions.

[0019] Unless otherwise stated, the term "plurality" means two or more.

[0020] In the embodiment of the present disclosure, the character " / " indicates that the preceding and following objects are in an "or" relationship. For example, A / B indicates: A or B.

[0021] The term "and / or" is a description of the association relationship between objects, indicating that three relationships can exist. For example, A and / or B means: A or B, or, A and B.

[0022] The term "correspondence" may refer to an association relationship or a binding relationship. The correspondence between A and B means that there is an association relationship or a binding relationship between A and B.

[0023] The enhanced GPGPU warp cache prefetching method, device and storage medium provided by the embodiments of the present disclosure are described below in conjunction with the accompanying drawings.

[0024] Figure 1 is a flow chart of an enhanced GPGPU warp cache prefetching method provided by an embodiment of the present disclosure, such as Figure 1 As shown, the enhanced GPGPU warp cache prefetching method may include: S01, perform core initialization, and initialize a stride prefetcher for each processor core; S02, configuring an independent training prefetcher for each thread warp; S03, using each warp stride table to record the stride information of each warp, wherein the warp stride table stores the stride size, access frequency and other related parameters to provide guidance for the prefetch operation; S04, when it is detected that multiple thread warps have the same stride for the same program counter, sending the stride information to a global stride table; S05, allows untrained warps to issue prefetch requests using entries in the global stride table.

[0025] In the present disclosure, the enhanced GPGPU warp cache prefetching method can identify regular access patterns by analyzing the memory access address and stride of each warp, and generate prefetch requests accordingly.

[0026] In some embodiments, Figure 1 The method may also include: A least recently used replacement policy is applied in each warp stride table and the global stride table.

[0027] In some embodiments, the initialized stride prefetcher is trained based on the program counter to recognize and predict memory access patterns.

[0028] In some embodiments, each prefetcher configuration includes necessary initial parameters of stride size, prefetch distance, and prefetch degree.

[0029] In some embodiments, during the training process, an independent training prefetcher configured for each warp monitors the memory access pattern of each warp in real time, and detects a stride pattern involving analysis of memory access addresses and access frequencies to identify regularity of the access pattern.

[0030] In some embodiments, during the monitoring process, the data in each warp stride table is continuously updated to reflect the latest memory access pattern, so that the prefetcher dynamically adjusts the prefetch strategy according to the latest memory access pattern.

[0031] Figure 2 is a GPGPU storage architecture diagram provided by an embodiment of the present disclosure, Figure 3 It is a structural schematic diagram of an enhanced thread warp prefetcher provided by an embodiment of the present disclosure.

[0032] Combination Figure 2 and Figure 3 ,right Figure 1 The enhanced GPGPU warp cache prefetching method is further described in detail.

[0033] 1. After the system is powered on, the core is initialized first, and a stride prefetcher is initialized for each processor core. The prefetcher will be trained based on the program counter (PC) to identify and predict memory access patterns. Make sure that each prefetcher is configured with the necessary initial parameters, such as stride size, prefetch distance, and prefetch degree.

[0034] 2. To avoid confusion between memory access patterns of different warps, each warp has an independently trained prefetcher. In this way, the prefetcher can more accurately predict the memory access needs of each warp. The prefetcher monitors the memory access pattern of each warp in real time during training, paying special attention to the stride pattern. This involves analyzing the memory access address and access frequency to identify the regularity of the access pattern.

[0035] 3. Use the PWS (stride per warp) table to record the stride information of each warp. This table will store the stride size, access frequency, and other related parameters to provide guidance for prefetch operations. During the monitoring process, the data in the PWS (stride per warp) table is continuously updated to reflect the latest memory access pattern. This ensures that the prefetcher can dynamically adjust the prefetch strategy based on the latest access pattern.

[0036] 4. When multiple warps are detected to have the same stride for the same program counter (PC), these stride information are promoted to the global stride (GS) table. In the subsequent process, untrained warps are allowed to use the entries in the global stride (GS) table to immediately issue prefetch requests, reducing the need to access the PWS (per warp stride) table.

[0037] 5. Apply a least recently used (LRU) replacement strategy in the PWS (per warp stride) and global stride GS tables to ensure that the most useful stride information is retained in the tables, optimizing space utilization and improving the responsiveness and accuracy of the prefetcher.

[0038] The enhanced GPGPU warp cache prefetching method disclosed in the present invention analyzes the memory access address and stride of each warp, and is able to identify regular access patterns and generate prefetch requests accordingly. The stride information obtained by each warp training is stored in the stride (PWS) table of each warp, which is similar to the prefetch table in a traditional prefetcher, but is designed specifically for each warp, thereby improving the pertinence and efficiency of the prefetch strategy. When multiple warps display the same stride for the same program counter (PC), the stride information is promoted to the global stride (GS) table. The promoted stride information allows untrained warps to immediately use entries in the GS table to issue prefetch requests, reducing dependence on the PWS table and the number of accesses, thereby improving the response speed of the prefetch operation.

[0039] Moreover, the mechanism disclosed in the present invention can determine when prefetching is beneficial and when it is harmful. This mechanism can dynamically adjust the prefetching strategy based on the memory access pattern, the state of the memory system, and the current system load. In this way, memory access can be managed more intelligently, thereby improving the overall performance of the GPGPU system.

[0040] Figure 4 is a schematic diagram of the structure of an enhanced GPGPU thread warp cache prefetching device provided by an embodiment of the present disclosure, such as Figure 4 As shown, the enhanced GPGPU warp cache prefetching device may include: Initialization module 401, used for performing core initialization, initializing a stride prefetcher for each processor core; A configuration module 402, configured to configure an independent training prefetcher for each warp; A recording module 403, configured to record the stride information of each warp using a warp stride table, wherein the warp stride table stores the stride size, access frequency and other related parameters to provide guidance for the prefetch operation; A sending module 404 is used to send the stride information to a global stride table when it is detected that multiple thread warps have the same stride for the same program counter; The sending module 404 is further configured to allow untrained warps to issue prefetch requests using entries in the global stride table.

[0041] In some embodiments, a least recently used replacement policy is applied in each warp stride table and the global stride table.

[0042] In some embodiments, the initialized stride prefetcher is trained based on the program counter to recognize and predict memory access patterns.

[0043] In some embodiments, each prefetcher configuration includes necessary initial parameters of stride size, prefetch distance, and prefetch degree.

[0044] In some embodiments, during the training process, an independent training prefetcher configured for each warp monitors the memory access pattern of each warp in real time, and detects a stride pattern involving analysis of memory access addresses and access frequencies to identify regularity of the access pattern.

[0045] In some embodiments, during the monitoring process, the data in each warp stride table is continuously updated to reflect the latest memory access pattern, so that the prefetcher dynamically adjusts the prefetch strategy according to the latest memory access pattern.

[0046] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity of description, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present disclosure is not limited by the order of the actions described, because according to the present disclosure, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily required by the present disclosure.

[0047] Combination Figure 5As shown, the embodiment of the present disclosure provides an enhanced GPGPU warp cache prefetching system 500, including a processor 504 and a memory 501. Optionally, the system may also include a communication interface 502 and a bus 503. The processor 504, the communication interface 502, and the memory 501 may communicate with each other through the bus 503. The communication interface 502 may be used for information transmission. The processor 504 may call the logic instructions in the memory 501 to execute the enhanced GPGPU warp cache prefetching method in the above embodiment.

[0048] In addition, the logic instructions in the memory 501 described above can be implemented in the form of software functional units and can be stored in a computer-readable storage medium when sold or used as an independent product.

[0049] The memory 501 is a computer-readable storage medium that can be used to store software programs and computer executable programs, such as program instructions / modules corresponding to the method in the embodiment of the present disclosure. The processor 504 executes the functional application and data processing by running the program instructions / modules stored in the memory 501, that is, implementing the enhanced GPGPU warp cache prefetching method in the above embodiment.

[0050] The memory 501 may include a program storage area and a data storage area, wherein the program storage area may store an operating system and an application required for at least one function; the data storage area may store data created according to the use of the terminal device, etc. In addition, the memory 501 may include a high-speed random access memory and may also include a non-volatile memory.

[0051] An embodiment of the present disclosure provides a computer-readable storage medium storing computer-executable instructions, wherein the computer-executable instructions are configured to execute an enhanced GPGPU warp cache prefetching method.

[0052] The computer-readable storage medium mentioned above may be a transient computer-readable storage medium or a non-transitory computer-readable storage medium.

[0053] The technical solution of the embodiment of the present disclosure can be embodied in the form of a software product, which is stored in a storage medium and includes one or more instructions for enabling a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method of the embodiment of the present disclosure. The aforementioned storage medium may be a non-transient storage medium, including: a USB flash drive, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a disk or an optical disk, and other media that can store program codes, or a transient storage medium.

[0054] The above description and the accompanying drawings fully illustrate the embodiments of the present disclosure so that those skilled in the art can practice them. Other embodiments may include structural, logical, electrical, process and other changes. The embodiments represent possible changes only. Unless explicitly required, individual components and functions are optional, and the order of operations may vary. Parts and features of some embodiments may be included in or replace parts and features of other embodiments. As used in the description of the embodiments, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms as well. Similarly, the term "and / or" as used in this application refers to any and all possible combinations of one or more associated listings. In addition, when used in this application, the term "comprise" and its variants "comprises" and / or including (comprising) refer to the existence of stated features, wholes, steps, operations, elements, and / or components, but do not exclude the existence or addition of one or more other features, wholes, steps, operations, elements, components and / or these groups. In the absence of further restrictions, the elements defined by the sentence "comprising a ..." do not exclude the existence of other identical elements in the process, method or device comprising the elements. In this article, each embodiment may focus on the differences from other embodiments, and the same or similar parts between the embodiments may refer to each other. For the methods, products, etc. disclosed in the embodiments, if they correspond to the method part disclosed in the embodiments, then the relevant parts can refer to the description of the method part.

[0055] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software may depend on the specific application and design constraints of the technical solution. Technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the embodiments of the present disclosure. Technicians can clearly understand that for the convenience and simplicity of description, the specific working process of the systems, devices and units described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0056] In the embodiments disclosed herein, the disclosed methods and products (including but not limited to devices, equipment, etc.) can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of units can be only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between each other shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms. The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the units may be selected according to actual needs to implement this embodiment. In addition, each functional unit in the embodiment of the present disclosure may be integrated in a processing unit, or each unit may exist physically alone, or two or more units may be integrated in one unit.

[0057] The flowchart and block diagram in the accompanying drawings show the possible architecture, functions and operations of the system, method and computer program product according to the embodiment of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of the code, and a part of the module, program segment or code contains one or more executable instructions for realizing the specified logical function. In some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, which can depend on the functions involved. In the description corresponding to the flowchart and block diagram in the accompanying drawings, the operations or steps corresponding to different boxes can also occur in a different order from the order disclosed in the description, and sometimes there is no specific order between different operations or steps. For example, two consecutive operations or steps can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, which can depend on the functions involved. Each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented by a dedicated hardware-based system that performs the specified functions or actions, or may be implemented by a combination of dedicated hardware and computer instructions.

Claims

1. An enhanced GPGPU warp cache prefetching method, It is characterized in that The method comprises: Perform core initialization and initialize a stride prefetcher for each processor core; Configure independent training prefetchers for each warp; Using each warp stride table to record the stride information of each warp, wherein the warp stride table stores the stride size, access frequency and other related parameters to provide guidance for prefetch operations; When it is detected that multiple thread warps have the same stride for the same program counter, the stride information is sent to the global stride table; Allow untrained warps to issue prefetch requests using entries in the global stride table.

2. The method according to claim 1, It is characterized in that The method further comprises: A least recently used replacement policy is applied in each warp stride table and the global stride table.

3. The method according to claim 1, It is characterized in that The initialized stride prefetcher is trained based on the program counter to recognize and predict memory access patterns.

4. The method according to claim 1, It is characterized in that Each prefetcher configuration includes necessary initial parameters of stride size, prefetch distance, and prefetch degree.

5. The method according to claim 1, It is characterized in that An independent training prefetcher configured for each warp monitors the memory access pattern of each warp in real time during the training process, and detects the stride pattern involving analysis of memory access addresses and access frequencies to identify the regularity of the access pattern.

6. The method according to claim 5, It is characterized in that During the monitoring process, the data in each warp stride table is continuously updated to reflect the latest memory access pattern, so that the prefetcher can dynamically adjust the prefetch strategy according to the latest memory access pattern.

7. An enhanced GPGPU warp cache prefetching device, It is characterized in that The device comprises: An initialization module, used for core initialization, initializing a stride prefetcher for each processor core; A configuration module for configuring independent training prefetchers for each warp; A recording module, configured to record the stride information of each warp using a warp stride table, wherein the warp stride table stores the stride size, access frequency and other related parameters to provide guidance for the prefetch operation; A sending module, configured to send stride information to a global stride table when it is detected that multiple thread warps have the same stride for the same program counter; The sending module is further configured to allow untrained warps to issue pre-fetch requests using entries in the global stride table.

8. A storage medium storing program instructions, It is characterized in that When the program instructions are executed, the enhanced GPGPU warp cache prefetching method according to any one of claims 1 to 6 is executed.

Citation Information

Cited By

  • Prefetch optimization method and system of GPGPU and storage medium

    CN119415445A

  • Memory access request processing method and device for universal graphics processor

    CN121901119A

  • Memory access request processing method and apparatus for general purpose graphics processor

    CN121901119B