Prefetch optimization method and system of GPGPU and storage medium

Abstract

The invention relates to a prefetch optimization method and optimization system of a GPGPU and a storage medium, and belongs to the field of GPGPUs, and the prefetch optimization method comprises the following steps: S1, calculating a stepping distance between thread bundles based on addresses of two continuous thread bundles in a thread block, and calculating prefetch addresses of the remaining thread bundles in the thread block; calculating the stepping distance between the thread blocks based on the addresses of two continuous thread blocks in the same task, and calculating the prefetch addresses of the remaining thread blocks in the task; s2, endowing the thread bundles with stage attributes according to the calculation states of the stepping distance between the thread bundles and the stepping distance between the thread blocks; s3, capturing a thread bundle prefetching address based on the stage attribute of the thread bundle, and sending a prefetching request; s4, setting the state of the thread bundle based on the transmitting state of the prefetching request; and S5, generating the priority of the thread bundle based on the state of the thread bundle. Through the arranged step calculation unit, a prefetch queue or a wakeup history queue can be quickly calculated, and the performance and efficiency of the GPGPU are improved.

Description

Technical Field

[0001] The present invention belongs to the field of medical data processing, and in particular relates to a GPGPU pre-fetching optimization method, an optimization system and a storage medium. Background Art

[0002] The application of existing data prefetching technology in GPGPU has significant limitations: traditional data prefetching methods are often based on simple prediction models and fail to fully consider the dynamic characteristics of thread execution and the spatial locality of data access in GPGPU. This results in a low cache hit rate during execution, and memory access latency becomes a key factor restricting performance improvement. Especially when dealing with large-scale parallel tasks, resource competition and cache misses between thread blocks are particularly prominent. If the memory access latency is too high, the computing unit will be idle while waiting for data to be transferred from the memory. At the same time, when the data loading stage cannot provide data in time due to high latency, the subsequent computing and storage stages will be affected, the smoothness of the entire pipeline will be destroyed, and the number of computing tasks completed per unit time will be reduced. Summary of the invention

[0003] In view of the problems in the prior art, the present invention provides a GPGPU prefetching optimization method, optimization system and storage medium, which solves the problem that the data prefetching method in the prior art is often based on a simple prediction model, resulting in a large memory access delay and thus reducing the overall computing speed of the GPGPU.

[0004] The technical solution adopted by the present invention is as follows: In a first aspect, the present application provides a GPGPU prefetch optimization method, the method comprising the following steps: Step S1, calculating the step distance between the thread warps based on the addresses of two consecutive thread warps in the thread block, and calculating the pre-fetch addresses of the remaining thread warps in the thread block based on the step distance between the thread warps; Calculating a step distance between thread blocks based on addresses of two consecutive thread blocks in the same task, and calculating pre-fetch addresses of remaining thread blocks in the task based on the step distance between the thread blocks; storing the prefetch addresses of the remaining thread warps and the prefetch addresses of the remaining thread blocks in a step list; Step S2: assigning a phase attribute to the thread warp according to the calculation status of the step distance between thread warps and the step distance between thread blocks; Step S3: based on the phase attribute of the thread warp, the thread warp prefetch address in the step list is captured, a corresponding prefetch request is generated, and the prefetch request is sent to the data cache; Step S4, setting the state of the warp based on the emission state of the prefetch request; Step S5: Generate the priority of the warp based on the state of the warp.

[0005] Preferably, in step S1, the step distance between tasks is calculated based on the addresses of two consecutive tasks, and the pre-fetch address of the subsequent task in the cyclic task is calculated based on the step distance between tasks.

[0006] Preferably, in step S2, the pre-fetch stage attributes include an invalid stage, an initial stage, a semi-activated stage, and an activated stage, and the pre-fetch stage attributes are updated in real time according to the calculation state; The invalid stage means that the thread warp is in an idle state and has not taken effect; the initial stage means that the thread warp has taken effect, but the usable stepping distance has not been calculated; the semi-activated stage means that only the stepping distance between thread warps has been calculated in the task, but the stepping distance between thread blocks has not been calculated; the activated stage means that the stepping distance between thread warps and the stepping distance between thread blocks have been calculated in the task.

[0007] Preferably, step S3 comprises the following steps: Step S3-1, sorting calls according to pre-fetching stage attributes; Step S3-2: based on the call order of step S3-1, the thread warp prefetch address queue is captured in sequence, and a corresponding prefetch request is generated and sent to the data cache.

[0008] Preferably, in step S5, the thread warp state includes an idle state, a ready state, a completed state, and an ended state, and the thread warp state in the step queue unit is recorded in real time and the priority of the thread warp is adjusted in real time according to the thread warp state; The thread warp state of the thread warp that has not issued a prefetch request is in the idle state; the thread warp state when preparing to issue a request is in the ready state; the thread warp state that has issued a prefetch request is in the completed state; the thread warp state that has executed a memory access operation according to the prefetch request is in the ended state.

[0009] Preferably, when the computing task is completed, the thread warp prefetch addresses in the step list captured in step S3 are stored in the cache list.

[0010] In a second aspect, the present application provides a GPGPU optimization system, including: A step calculation unit, used for calculating and storing the step distance and prefetch address queue required for step prefetching in each mode; A step queue unit, used for storing the calculated pre-fetch address queue; A prefetch stage register, used to assign stage attributes to the warp according to the calculation status of the inter-warp step distance and the inter-thread block step distance: inactive stage, initial stage, semi-active stage and active stage; A warp state register for generating warp priorities based on the warp states: idle, ready, completed, and terminated; A request transmitting unit, for fetching a prefetch address queue of an entry in the step queue unit, and transmitting a corresponding prefetch request to the data cache; The pre-scheduling unit is used to read the status and stage of the entries in the step queue unit, and output the scheduling priority in real time to the scheduling module.

[0011] Preferably, a polling scheduling strategy is retained in the scheduling module.

[0012] Preferably, the optimization system also includes a historical cache unit, which is used to cache the pre-fetch information of the executed address sequence and memory access mode, and to call the cached pre-fetch information when the same task is subsequently executed. The cache queue applies a first-in-first-out replacement strategy.

[0013] In a third aspect, the present application provides a storage medium storing program instructions, which, when run, execute the GPGPU prefetch optimization method as described in the first aspect.

[0014] It can be seen from the above technical solutions that the present invention has the following advantages: by setting up a step calculation unit, it is possible to use the step prefetch technology to predict future memory access requirements, and combine with the historical wake-up mechanism to quickly wake up the executed prefetch address queue, thereby improving the performance and efficiency of GPGPU. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the technical solution of the present invention, the accompanying drawings required for use in the description will be briefly introduced below. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0016] Figure 1 It is a flowchart of the method in the specific implementation example of the present invention.

[0017] Figure 2 It is an overview of the architecture of the method and optimization system in the specific implementation example of the present invention.

[0018] Figure 3 Schematic diagram of stepping distance in a specific implementation example of the present invention.

[0019] Figure 4 It is a schematic diagram of pre-fetch address calculation in a specific implementation example of the present invention. DETAILED DESCRIPTION

[0020] In the detailed description below, various embodiments of the present disclosure will be described more fully. The present disclosure may have various embodiments, and adjustments and changes may be made therein. However, it should be understood that there is no intention to limit the various embodiments of the present disclosure to the specific embodiments disclosed herein, but rather the present disclosure should be understood to cover all adjustments, equivalents and / or alternatives that fall within the spirit and scope of the various embodiments of the present disclosure.

[0021] In view of the problems in the prior art, the present invention provides a GPGPU loading and storage unit, a GPGPU prefetching optimization method and a storage medium, which solves the problem that the data prefetching method in the prior art is often based on a simple prediction model, resulting in a large memory access delay and thus reducing the overall computing speed of the GPGPU.

[0022] like Figure 1 and Figure 3 As shown, this embodiment provides a GPGPU prefetching optimization method, the method comprising the following steps: Step S1, calculating the step distance between the thread warps based on the addresses of two consecutive thread warps in the thread block, and calculating the pre-fetch addresses of the remaining thread warps in the thread block based on the step distance between the thread warps; Calculating a step distance between thread blocks based on addresses of two consecutive thread blocks in the same task, and calculating pre-fetch addresses of remaining thread blocks in the task based on the step distance between the thread blocks; storing the prefetch addresses of the remaining thread warps and the prefetch addresses of the remaining thread blocks in a step list; The step distance between thread warps is calculated based on the address difference between two adjacent thread warps in the same thread block, such as Figure 3 As shown in ①; the step distance between thread blocks is calculated based on the address difference between adjacent thread blocks in the same task, such as Figure 3 As shown in ②; the step distance between tasks is calculated based on the address difference between adjacent tasks when executing the loop, such as Figure 3 As shown in ③, since not every task is a cyclic task and presents regularity, the step distance between tasks does not always exist. The calculated step distance will be added to the address being executed to obtain the pre-fetch address, which is sent to the request issuance unit, which will issue the request to the data cache to realize data pre-fetching; Step S2: assigning a phase attribute to the thread warp according to the calculation status of the step distance between thread warps and the step distance between thread blocks; Step S3: based on the phase attribute of the thread warp, the thread warp prefetch address in the step list is captured, a corresponding prefetch request is generated, and the prefetch request is sent to the data cache; Step S4, setting the state of the warp based on the emission state of the prefetch request; Step S5: Generate the priority of the warp based on the state of the warp.

[0023] In this embodiment, in step S1 , the step distance between tasks is calculated based on the addresses of two consecutive tasks, and the pre-fetch address of the subsequent task in the cyclic task is calculated based on the step distance between tasks.

[0024] In this embodiment, in step S2, the pre-fetch stage attributes include an invalid stage, an initial stage, a semi-activated stage, and an activated stage, and the pre-fetch stage attributes are updated in real time according to the calculation state; The invalid stage means that the entry is idle and has not yet taken effect; the initial stage means that the entry has taken effect, but the usable step distance has not yet been calculated; the semi-activated stage means that the current entry has only calculated the step distance between thread warps, but has not yet calculated the step distance between thread blocks; the activated stage means that the current entry has calculated the step distance between thread blocks. It is worth noting that the step distance between tasks does not always exist, so the judgment in the prefetch stage does not consider the step distance between tasks.

[0025] In this embodiment, step S3 includes the following steps: Step S3-1, sorting calls according to pre-fetching stage attributes; Step S3-2: based on the call order of step S3-1, the thread warp prefetch address queue is captured in sequence, and a corresponding prefetch request is generated and sent to the data cache.

[0026] In this embodiment, in step S5, the thread warp state includes an idle state, a ready state, a completed state, and an ended state, and the thread warp state register records the thread warp state in the step queue unit in real time; The priority of the thread warp is adjusted in real time according to the thread warp status.

[0027] In this embodiment, after the computing task is completed, the thread warp prefetch addresses in the step list captured in step S3 are stored in the cache list.

[0028] like Figure 2 As shown, this embodiment also provides a GPGPU optimization system, including: a step calculation unit, a step queue unit, a request emission unit, a history cache unit and a pre-scheduling unit; The step calculation unit calculates and stores the step distance required for step prefetching in each mode, which determines the address difference during prefetching. The calculated address difference is added to the current address to obtain the prefetch address queue, which is sent to the step queue unit.

[0029] The step queue unit stores the calculated prefetch address queue for the request issue unit to fetch. At the same time, the thread warp status register and prefetch stage register maintained in this unit can record the prefetch status of the thread warp and the prefetch stage of the current entry; The prefetch stage register is used to assign stage attributes to the thread warp according to the calculation status of the step distance between thread warps and the step distance between thread blocks: invalid stage, initial stage, semi-activated stage and activated stage; The warp state register is used to generate the priority of the warp based on the state of the warp: idle state, ready state, completed state, and finished state.

[0030] The request issuing unit will fetch entries from the step queue unit and issue corresponding pre-fetch requests to the data cache, wherein the pre-fetch stage register of the step queue unit will be read for determining the priority of entry fetching.

[0031] The historical cache unit caches pre-fetch information such as executed address sequences and memory access patterns. When executing the same task, the cached pre-fetch information can be quickly called to quickly activate a new pre-fetch queue. In this embodiment, the historical cache unit applies a first-in-first-out replacement strategy.

[0032] The pre-scheduling unit reads the status and stage in the step queue unit, and outputs the scheduling priority in real time based on the read information, and sends it to the scheduling module to achieve pre-fetch-based priority collaborative scheduling.

[0033] In this embodiment, the scheduling module retains the polling scheduling strategy. When prefetching is not enabled or disabled, the scheduling module schedules the thread warps using the basic polling scheduling strategy.

[0034] After reading the thread warp status register in the step queue unit, the pre-scheduling unit will adjust the priority of the thread warp in real time according to the status of each thread warp: since all requests have been issued and prefetching can be performed, the completed state has the highest priority; when the thread warp is ready to issue a request, it is in the ready state, with a higher priority; if no request has been issued yet, it is in the idle state, with a lower priority; the state where the memory access operation has been executed and no prefetching is done is the ended state, with the lowest priority.

[0035] Among them, when the thread warp is in the idle state, prefetching is not activated and the thread warp will not issue a prefetch request; when the thread warp is in the ready state, the thread warp has prepared the required information and will issue a prefetch request to activate the prefetch operation; when the thread warp successfully issues all prefetch requests, it is marked as completed; when the memory access instruction arrives in the optimization system, the memory access operation starts, prefetching is no longer performed, and the thread warp is marked as ended.

[0036] The present invention is further explained as follows: Figure 4 As shown, in this task, each thread block contains 4 thread warps, and the addresses are shown in the figure.

[0037] Before the prefetch starts, the stages in the prefetch stage register are in an idle state because the step distance has not yet been calculated; When the task starts to execute, the step calculation unit still does not have the conditions to calculate the step distance, so the pre-fetch stage is transformed into the initial stage, waiting for the calculation result of the calculation unit; When executing to thread warp 1, the step calculation unit can calculate the step distance between thread warps and send the result to the step queue unit. The prefetch stage becomes semi-activated and can start prefetching the address of the thread warp: the calculation unit will use the current address as the reference, add it to the step distance, get the prefetch address queue of the future thread warp, and write it into the step queue unit. Figure 3 In the example shown, the step distance between warps = 0x1020-0x1000 = 0x0020.

[0038] When all the thread warps in thread block 0 have been executed and thread warp 0 in thread block 1 begins to execute, the step calculation unit can calculate the step distance between thread blocks and write the result to the step queue unit. After obtaining the step distance between thread blocks, the step queue unit will transform the prefetch stage into the activation stage, even if the step distance between tasks has not been obtained at this time. At the same time, the step calculation unit can calculate the prefetch address queue of future thread blocks and complete the prediction of the address of future thread blocks and their internal thread warps. Figure 3 In the example shown, the step distance between thread blocks = 0x1080-0x1000 = 0x0080.

[0039] When task 0 is completed and task 1 is started, the step calculation unit obtains the address of thread warp 0 in thread block 0 in task 1, calculates the step distance between tasks, writes it to the step queue unit, and calculates all addresses of future tasks. Figure 3 In the example shown, the step distance between tasks = 0x3000 - 0x1000 = 0x2000.

[0040] After entering the semi-activation phase from the prefetch phase, the request issuing unit can fetch entries in the step queue unit and generate prefetch requests. When all requests are issued, the warp status register in the step queue unit sets the warp status to complete. Until the memory access instruction enters the optimization system, no more prefetching is done and the warp status is marked as finished.

[0041] According to the thread warp status, the pre-scheduling unit will generate the scheduling priority order of the thread warp in real time and send the priority to the scheduling module so that the scheduling module can coordinate the thread warps and give priority to the thread warps that have completed pre-fetching to improve memory access efficiency and further optimize the computing efficiency of GPGPU.

[0042] When a task ends and switches to other tasks, the prefetch address queue of the executed task will be sent to the historical cache unit until the next time the task is executed again. The historical cache unit will resend the queue to the step queue unit to reactivate the queue and prefetch directly based on the queue, saving recalculation time.

[0043] This embodiment also provides a storage medium storing program instructions, which, when running, execute the GPGPU prefetch optimization method as described above.

[0044] It is understandable that the systems, devices, modules or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer, and the specific form of the computer can be a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email transceiver device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0045] In a typical configuration, a computer includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0046] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0047] Computer-readable media include permanent and non-permanent, removable and non-removable media that can be used to store information by any method or technology. Information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, disk storage, quantum memory, graphene-based storage media or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include temporary computer-readable media (transitory media), such as modulated data signals and carrier waves.

[0048] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0049] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0050] The above is a description of a specific embodiment of the specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0051] The terms used in one or more embodiments of this specification are only for the purpose of describing specific embodiments, and are not intended to limit one or more embodiments of this specification. The singular forms of "a", "said" and "the" used in one or more embodiments of this specification and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more associated listed items.

[0052] It should be understood that although the terms first, second, third, etc. may be used to describe various information in one or more embodiments of this specification, these information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of one or more embodiments of this specification, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0053] The above description is merely a preferred embodiment of one or more embodiments of the present specification and is not intended to limit one or more embodiments of the present specification. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of one or more embodiments of the present specification shall be included in the scope of protection of one or more embodiments of the present specification.

Claims

1. A GPGPU prefetch optimization method, characterized in that: The method comprises the following steps: Step S1, calculating the step distance between the thread warps based on the addresses of two consecutive thread warps in the thread block, and calculating the pre-fetch addresses of the remaining thread warps in the thread block based on the step distance between the thread warps; Calculating a step distance between thread blocks based on addresses of two consecutive thread blocks in the same task, and calculating prefetch addresses of remaining thread blocks in the task based on the step distance between the thread blocks; storing the prefetch addresses of the remaining thread warps and the prefetch addresses of the remaining thread blocks in a step list; Step S2: assigning a phase attribute to the thread warp according to the calculation status of the step distance between thread warps and the step distance between thread blocks; Step S3: based on the phase attribute of the thread warp, the thread warp prefetch address in the step list is captured, a corresponding prefetch request is generated, and the prefetch request is sent to the data cache; Step S4, setting the state of the warp based on the emission state of the prefetch request; Step S5: Generate the priority of the warp based on the state of the warp.

2. The GPGPU prefetching optimization method according to claim 1, characterized in that: In step S1 , a step distance between tasks is calculated based on the addresses of two consecutive tasks, and a pre-fetch address of a subsequent task in the cyclic task is calculated based on the step distance between tasks.

3. The GPGPU prefetching optimization method according to claim 1, characterized in that: In step S2, the pre-fetch stage attributes include an invalid stage, an initial stage, a semi-activated stage, and an activated stage, and the pre-fetch stage attributes are updated in real time according to the calculation state; The invalid stage means that the thread warp is in an idle state and has not taken effect; the initial stage means that the thread warp has taken effect, but the usable stepping distance has not been calculated; the semi-activated stage means that only the stepping distance between thread warps has been calculated in the task, but the stepping distance between thread blocks has not been calculated; the activated stage means that the stepping distance between thread warps and the stepping distance between thread blocks have been calculated in the task.

4. The GPGPU prefetching optimization method according to claim 3, characterized in that: Step S3 includes the following steps: Step S3-1, sorting calls according to pre-fetching stage attributes; Step S3-2: based on the call order of step S3-1, the thread warp prefetch address queue is captured in sequence, and a corresponding prefetch request is generated and sent to the data cache.

5. The GPGPU prefetching optimization method according to claim 1, characterized in that: In step S5, the thread warp states include idle state, ready state, completed state, and ended state. The thread warp states in the step queue unit are recorded in real time and the thread warp priorities are adjusted in real time according to the thread warp states. The thread warp state of the thread warp that has not issued a prefetch request is in the idle state; the thread warp state when preparing to issue a request is in the ready state; the thread warp state that has issued a prefetch request is in the completed state; the thread warp state that has executed a memory access operation according to the prefetch request is in the ended state.

6. The GPGPU prefetching optimization method according to claim 1, characterized in that: When the computing task is completed, the thread warp prefetch addresses in the step list captured in step S3 are stored in the cache list.

7. A GPGPU optimization system, characterized in that: include: A step calculation unit, used for calculating and storing the step distance and prefetch address queue required for step prefetching in each mode; A step queue unit, used for storing the calculated pre-fetch address queue; A prefetch stage register, used to assign stage attributes to the warp according to the calculation status of the inter-warp step distance and the inter-thread block step distance: inactive stage, initial stage, semi-active stage and active stage; A warp state register for generating warp priorities based on the warp states: idle, ready, completed, and terminated; a request transmitting unit, for fetching a prefetch address queue of an entry in the step queue unit, and transmitting a corresponding prefetch request to the data cache; The pre-scheduling unit is used to read the status and stage of the entries in the step queue unit, and output the scheduling priority in real time to the scheduling module.

8. The GPGPU optimization system according to claim 7, characterized in that: The polling scheduling strategy is retained in the scheduling module.

9. The GPGPU optimization system according to claim 7, characterized in that: The optimization system also includes a history cache unit, which is used to cache the pre-fetch information of the executed address sequence and memory access mode, and to call the cached pre-fetch information when the same task is executed subsequently. The cache queue applies a first-in-first-out replacement strategy.

10. A storage medium, characterized in that: Program instructions are stored, and when the program instructions are run, the GPGPU prefetch optimization method according to any one of claims 1 to 6 is executed.

Images