Thread bundle execution method and related GPU
By optimizing the use of the note storage, checking status indicators, and performing thread bundle operations in stages, the problem of note storage space limitations was solved, and the execution efficiency of the GPU was improved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- ALIBABA (CHINA) CO LTD
- Filing Date
- 2022-06-06
- Publication Date
- 2026-07-31
AI Technical Summary
When the GPU executes kernel code, the limited note memory space of the thread bundle becomes a bottleneck for scheduling the thread bundle, affecting execution efficiency.
By checking the status indicators of the note storage, the use of the note storage can be optimized, including the execution methods of loading and processing thread bundles, performing data operations in stages, and proactively notifying of status changes, thereby improving the efficiency of memory usage.
Without increasing the size of the note storage, the number and efficiency of GPU processing threads were increased, and memory space utilization was optimized.
Smart Images
Figure CN117237178B_ABST
Abstract
Description
Technical Field
[0001] This application relates to a thread bundle, and more particularly to a thread bundle execution method and a related GPU. Background Technology
[0002] When a GPU executes kernel code, it does so on in units of warps on streaming processors (SPs). During this process, a notepad memory is used to temporarily store the data required for computation. Since this notepad memory is shared by multiple streaming processors, its limited space becomes a bottleneck in the number of warps that can be scheduled to a streaming processor, and this is a problem that urgently needs to be solved in this field. Summary of the Invention
[0003] One of the purposes of this application is to disclose a thread bundle execution method and a related GPU to solve the above-mentioned problems.
[0004] One embodiment of this application discloses a thread bundle execution method for multiple stream processors in a stream processing unit of a GPU. The multiple stream processors share a note memory, wherein the space in the note memory is in one of the following states: blank, in use of data, not in use of data, and loading data. The method includes: when a preset thread bundle loading time point has arrived, checking a first indicator to obtain the size of the area in the note memory in the state of blank, so as to determine whether to execute the loading thread bundle, wherein the first indicator is used to indicate the start position of the area in the note memory in the state of in use of data and the end position of the area in the state of blank; and when a preset operation time point has arrived, checking a second indicator and a third indicator to obtain the size of the area in the note memory in the state of not in use of data, so as to determine whether to execute the operation thread bundle, wherein the second indicator is used to indicate the end position of the area in the note memory in the state of not in use of data and the start position of the area in the state of loading data, and the third indicator is used to indicate the end position of the area in the note memory in the state of in use of data and the start position of the area in the state of not in use of data.
[0005] One embodiment of this application discloses a GPU, the GPU comprising: a stream processing unit including: a stream processor for performing the above-described method; and the note storage.
[0006] One embodiment of this application discloses a thread bundle execution method for multiple stream processors in a stream processing unit of a GPU. The multiple stream processors share a note memory, wherein the space in the note memory is in one of the following states: blank, in use of data, not in use of data, and loading data. The method includes: when receiving a data arrival notification from a loading / storage engine, checking a second indicator and a third indicator to obtain the size of the region in the note memory in the state of not in use of data, to determine whether to execute a computation thread bundle, wherein the second indicator is used to indicate the end position of the region in the note memory in the state of not in use of data and the start position of the region in the state of loading data, and the third indicator is used to indicate the end position of the region in the note memory in the state of use of data and the start position of the region in the state of not in use of data; and when it is determined that the computation thread bundle should be executed, issuing a consumption command to the loading / storage engine.
[0007] One embodiment of this application discloses a GPU, the GPU comprising: a stream processing unit including: a stream processor for executing the above-described method; the note storage; and the load / store engine.
[0008] The thread-beam execution method and related GPU disclosed in this application can optimize the space utilization of note memory, thereby increasing GPU performance. Attached Figure Description
[0009] Figure 1 This is a schematic diagram of the first embodiment of the GPU of this application.
[0010] Figure 2 This is a schematic diagram of an embodiment of the note storage device of this application.
[0011] Figure 3 This is a schematic diagram of the first part of the first embodiment of the thread bundle execution method of this application.
[0012] Figure 4 For illustrative purposes Figure 3 A schematic diagram of an embodiment.
[0013] Figure 5 This is a schematic diagram of the second part of the first embodiment of the thread bundle execution method of this application.
[0014] Figure 6 For illustrative purposes Figure 5 A schematic diagram of an embodiment.
[0015] Figure 7 This is a schematic diagram of the third part of the first embodiment of the thread bundle execution method of this application.
[0016] Figure 8 For illustrative purposes Figure 7 A schematic diagram of an embodiment.
[0017] Figure 9 This is a schematic diagram of the fourth part of the first embodiment of the thread bundle execution method of this application.
[0018] Figure 10 For illustrative purposes Figure 9 A schematic diagram of an embodiment.
[0019] Figure 11 This is a schematic diagram of a second embodiment of the GPU of this application.
[0020] Figure 12 This is a schematic diagram of the first part of the second embodiment of the thread bundle execution method of this application.
[0021] Figure 13 This is a schematic diagram of the second part of the second embodiment of the thread bundle execution method of this application.
[0022] Figure 14 This is a schematic diagram of the third part of the second embodiment of the thread bundle execution method of this application.
[0023] Figure 15 This is a schematic diagram of the fourth part of the second embodiment of the thread bundle execution method of this application.
[0024] Figure 16 This is a schematic diagram of the fifth part of the second embodiment of the thread bundle execution method of this application.
[0025] Figure 17 for Figure 11 A schematic diagram of an embodiment of the load / store engine. Detailed Implementation
[0026] The following disclosure provides various implementations or examples that can be used to achieve different features of this disclosure. Specific examples of components and configurations described below are for simplification purposes. It is understood that these descriptions are illustrative only and are not intended to limit the scope of this disclosure. For example, in the following description, forming a first feature on or over a second feature may include, in some embodiments, the first and second features being in direct contact with each other; and may also include, in some embodiments, additional components being formed between the first and second features, such that the first and second features may not be in direct contact. Furthermore, component symbols and / or reference numerals may be reused in multiple embodiments of this disclosure. Such reuse is for the purpose of brevity and clarity and does not in itself represent a relationship between the different embodiments and / or configurations discussed.
[0027] Furthermore, the use of spatially relative terms, such as "below," "below," "lower than," "above," "above," and similar terms, may be for the convenience of describing the relationship between one component or feature depicted in the figure and one or more other components or features. These spatially relative terms, in addition to the orientation shown in the figure, also encompass various different orientations of the device during use or operation. The device may be placed in other orientations (e.g., rotated 90 degrees or in other orientations), and these spatially relative descriptive terms should be interpreted accordingly.
[0028] While the numerical ranges and parameters used to define the broader scope of this application are approximate values, the relevant values in the specific embodiments have been presented as precisely as possible. However, any value inevitably contains standard deviations due to individual test methods. Here, "identical" generally means that the actual value is within plus or minus 10%, 5%, 1%, or 0.5% of a particular value or range. Alternatively, the term "identical" means that the actual value falls within the acceptable standard error of the mean, as determined by those skilled in the art to which this application pertains. It is understood that, except for experimental examples, or unless explicitly stated otherwise, all ranges, quantities, values, and percentages used herein (e.g., used to describe material usage, duration, temperature, operating conditions, quantity ratios, and others similar) are modified with "identical". Therefore, unless otherwise stated, the numerical parameters disclosed in this specification and the accompanying claims are approximate values and are subject to change as needed. At a minimum, these numerical parameters should be understood as the indicated significant digits and values obtained by applying general rounding. In this context, a range of values is expressed as a distance from one endpoint to the other or between the two endpoints; unless otherwise stated, all ranges of values herein include the endpoints.
[0029] Figure 1 This is a schematic diagram of a first embodiment of the GPU of this application. GPU 100 includes multiple streaming multiprocessors (SMs) SM0, SM1, ..., each of which includes multiple streaming processors (SPs) SP0, SP1, ... and a note memory 102, also referred to as shared memory. GPU 100 receives and executes thread blocks TB, each TB comprising multiple thread bundles WP. GPU 100 schedules the thread block TB to one of the multiple streaming multiprocessors SM0, SM1, ... For example, ... Figure 1As shown, thread block TB is scheduled to stream processing unit SM0, and multiple thread bundles of thread block TB are scheduled to multiple stream processors SP0, SP1, ... of stream processing unit SM0.
[0030] In this embodiment, the types of thread bundles include computation thread bundles and loading thread bundles. When a computation thread bundle is executed, the GPU 100 performs a "computation" operation; when a loading thread bundle is executed, the GPU 100 performs a "loading / storage" operation. For each stream processing unit, the multiple stream processors SP0, SP1, ... often need to retrieve data from global memory and temporarily load it into the note memory 102 for computation when executing thread bundles. Since the note memory 102 is shared by multiple stream processors SP0, SP1, ... and its space is limited, it becomes a bottleneck when the GPU 100 executes thread bundles. The thread bundle execution method proposed in this application can improve the overall efficiency of the GPU 100 in executing thread bundles by optimizing the utilization efficiency of the note memory 102, as detailed below.
[0031] Figure 2 This is a schematic diagram of an embodiment of the note storage device 102 of this application. For ease of subsequent explanation, the note storage device 102... Figure 2 The storage space allocation is illustrated using bars. In this embodiment, the notepad memory 102 is used from left to right. When the rightmost end is reached, it returns to the leftmost end to continue use. The space in the notepad memory 102 has four different states: blank, data in use, data not in use, and data being loaded. "Blank" indicates that the space is empty and can be loaded with data; "data in use" indicates that the space already contains data and is being used for calculations; "data not in use" indicates that the space already contains data but has not yet been used for calculations; "data being loaded" indicates that the space is expected to contain data, but the data has not yet arrived from global memory.
[0032] The first indicator I1 is used to indicate the end position of the region with a blank status and the beginning position of the region with a data usage status. For example, the first indicator I1 is specifically the end address of the region with a blank status. Since the beginning address of the region with a data usage status is adjacent to the end address of the region with a blank status, the beginning address of the region with a data usage status can also be determined from the first indicator I1. It is conceivable that the first indicator I1 can also specifically be the beginning address of the region with a data usage status.
[0033] The second indicator I2 is used to indicate the end position of a region whose status is "not in use" and the beginning position of a region whose status is "loading data". For example, the second indicator I2 can specifically be the end address of a region whose status is "not in use" or the beginning address of a region whose status is "loading data".
[0034] The third indicator I3 is used to indicate the end position of a region in use and the beginning position of a region not in use. For example, the third indicator I3 can be the end address of a region in use or the beginning address of a region not in use.
[0035] For ease of explanation, the first embodiment of the thread bundle execution method of this application is divided into four flowcharts and... Figure 3 , Figure 5 , Figure 7 and Figure 9 Explain each separately. Figure 4 , Figure 6 , Figure 8 and Figure 10 This is used to supplement the explanation. Figure 3 , Figure 5 , Figure 7 and Figure 9 In the first embodiment of the thread bundle execution method of this application, the first index I1 and the second index I2 are stored in specific preset areas in the note memory 102. The third index I3 may be stored inside or outside the note memory 102.
[0036] Specifically Figure 3 The flowchart is related to Figure 1 The GPU 100's stream processors execute a process of loading thread bundles. In step 302, the stream processor determines whether a preset loading time point has arrived. For example, the preset loading time point can be a time point that occurs based on a fixed time period. In some embodiments, determining whether the preset loading time point has arrived can also be performed by the scheduler in the GPU 100's multi-stream processors.
[0037] If the preset loading time has not yet arrived, the process will remain in step 302 until the preset loading time arrives, at which point it will proceed to step 304. In step 304, the stream processor checks the first indicator I1 to obtain the end position of the blank area in the note memory 102. In this embodiment, the stream processor itself uses indicators other than the first indicator I1, the second indicator I2, and the third indicator I3 to record and dynamically update the starting position of the blank area. Therefore, the starting position of the blank area is known to the stream processor. Thus, by checking the first indicator I1 and then using the known starting position of the blank area, the stream processor can obtain the overall size of the blank area.
[0038] In step 306, the stream processor determines whether the size of the region with a blank state is greater than a preset loading threshold. For example... Figure 4 As shown, the dashed box represents the preset loading threshold, such as... Figure 4 As illustrated, if the size of the blank region exceeds a preset loading threshold, step 308 can proceed and the loading thread bundle is executed to load data, the stream processor updates the starting position of the blank region. If the situation is not as described... Figure 4 As shown, if the size of the blank area is not greater than the preset loading threshold, then return to step 302.
[0039] In some embodiments, if step 306 is entered multiple times and step 308 cannot be entered, it means that the data in the note storage 102 cannot be processed in time, and the bottleneck is the operation thread bundle. Therefore, the preset loading time point can be postponed, that is, the attempt to execute the loading thread bundle can be temporarily postponed to reduce the waste of resources caused by repeated attempts.
[0040] In this embodiment, when the loading thread bundle is executed in step 308, the complete data is not always loaded at once, but rather in multiple loads, with only a portion of the data loaded each time. Therefore, the preset loading threshold represents the size of the data to be loaded next time. When loading in multiple loads, the data is not necessarily divided into equal parts of a fixed size, so the preset loading threshold can be variable.
[0041] Figure 5 The flowchart is related to Figure 1 The process of executing computation thread bundles by each stream processor of GPU 100. In step 502, the stream processor determines whether a preset computation time point has arrived. For example, the preset computation time point can be a time point that occurs based on a fixed time period. In some embodiments, the determination of whether the preset computation time point has arrived can also be performed by the scheduler in the multi-stream processor of GPU 100.
[0042] If the preset calculation time point has not yet arrived, the process will remain in step 502 until the preset calculation time point arrives, at which point it will proceed to step 504. In step 504, the stream processor will check the second indicator I2 and the third indicator I3 to obtain the size of the area in the note memory 102 where the data is not in use. In step 506, the stream processor will determine whether the size of the area where the data is not in use is greater than a preset calculation threshold. Figure 6 As shown, the dashed box represents the preset calculation threshold, such as... Figure 6The illustration shows that if the size of the area where data is not in use is greater than a preset calculation threshold, then step 508 can be entered and the calculation thread bundle and stream processor can be executed. The third indicator I3 is then modified to indicate that the state of a portion of the note storage 102 changes from "data not in use" to "data in use". If the situation is not as described... Figure 6 As shown, if the size of the region where the data is not in use is not greater than the preset calculation threshold, then return to step 502.
[0043] In some embodiments, if step 506 is entered multiple times and step 508 cannot be entered, it means that the data is not loaded into the note storage 102 fast enough and the bottleneck is the loading thread bundle. Therefore, the preset calculation time point can be postponed, that is, the attempt to execute the calculation thread bundle can be temporarily postponed to reduce the waste of resources caused by repeated attempts.
[0044] In this embodiment, when executing the calculation thread in step 508, it is not always necessary to wait for all the complete data to arrive at the note storage 102 before starting the calculation thread. Instead, it is divided into multiple calculations, each using only a portion of the data. Therefore, the preset calculation threshold represents the size of the data to be used for the next calculation. When performing multiple calculations, the data is not necessarily divided into equal portions of a fixed size, so the preset calculation threshold can be variable.
[0045] Figure 7 The flowchart is related to Figure 1 The GPU 100's stream processors execute a loading thread bundle process. In step 702, the stream processor determines if any new data has been loaded into the note-taking memory 102. For example, after executing the loading thread bundle in step 308, data is read from global memory and loaded into the note-taking memory 102. Figure 7 The flowchart will then proceed from step 702 to step 704. If no new data is loaded into the note storage 102, the process will continue at step 702 until new data is loaded into the note storage 102.
[0046] In step 704, the stream processor modifies the second index I2 according to the size of the newly loaded data into the note storage 102 to indicate that the state of a portion of the note storage is changed from "loading data" to "data not in use". For example... Figure 8 As shown, the dashed boxes represent data newly loaded into the note storage 102.
[0047] Figure 9 The flowchart is related to Figure 1The process of executing computational thread bundles by each stream processor of GPU 100. In step 902, the stream processor determines if there is newly added data in the note memory 102 that is no longer needed. For example, when executing a computational thread bundle in step 508, data in the note memory 102 is used. Even if the computational thread bundle has not yet finished executing, some data may have been determined to no longer be used; this data is the newly added data that is no longer needed mentioned in step 902. Whenever newly added data that is no longer needed is found, step 904 is entered to modify the first indicator I1 to indicate that the state of a certain area of the note memory 102 changes from "data in use" to "blank," indicating that the space occupied by this data in the note memory 102 can be released. Figure 10 As shown, the dashed boxes represent newly added data that is no longer needed.
[0048] The above Figure 3 , Figure 5 , Figure 7 and Figure 9 The provided embodiments enable the GPU to further refine the loading and computation threads when executing thread bundles, making the note memory more granular in use. This allows the present application to increase the number of thread bundles that the GPU can process per unit time without increasing the size of the note memory.
[0049] Figure 11 This is a schematic diagram of a second embodiment of the GPU of this application. The difference between GPU 1100 and GPU 100 is that each of the stream processing units SM0, SM1, ... of GPU 1100 includes a load / store engine 1104 coupled to the stream processors SP0, SP1, ... and the note memory 102. The load / store engine 1104 can be used to determine the address of the data required by each thread bundle in the execution thread block TB in global memory based on the primitives in the thread block TB, thereby replacing the loading thread bundle executed on the stream processor. In some embodiments, the load / store engine 1104 can also be obtained by modifying the direct memory access engine (DMA engine).
[0050] For ease of explanation, the second embodiment of the thread bundle execution method of this application is divided into five flowcharts and... Figure 12 , Figure 13 , Figure 14 , Figure 15 and Figure 16The following explanations will be provided separately. In the second embodiment of the thread bundle execution method of this application, the definitions of the state of the note storage 102 and the definitions of the first index I1, the second index I2, and the third index I3 are the same as those in the first embodiment described above. However, in this embodiment, the first index I1 and the second index I2 are stored in the loading / storage engine 1104. The third index I3 may be stored inside or outside the loading / storage engine 1104.
[0051] Specifically Figure 12 The flowchart is related to Figure 11 The GPU 1100's load / store engine 1104 executes the loading process. In step 1202, the load / store engine 1104 determines whether a flush command has been received. This flush command is issued by the stream processor based on the status of the execution thread bundle, indicating that some space in the note memory 102 can be freed. The flush command may contain address information of the freed area in the note memory 102, such as the starting position and length of data that is no longer needed. Details are described in... Figure 15 .
[0052] If it is determined that a clear command has not yet been received, the process will remain in step 1202 until a clear command is received, at which point it will proceed to step 1204. In step 1204, the loading / storage engine 1104 modifies the first index I1 based on the information in the clear command, indicating that the state of a portion of the note storage 102 has changed from "in use" to "blank." Simultaneously, the first index I1 also represents the end position of the blank state in the note storage 102. In this embodiment, the starting position of the blank state area is recorded by the loading / storage engine 1104 itself and dynamically updated, thus it is known to the loading / storage engine 1104. Therefore, by checking the first index I1 and then using the known starting position of the blank state area, the loading / storage engine 1104 can obtain the overall size of the blank state area.
[0053] In step 1206, the load / store engine 1104 determines whether the size of the region with a blank status is greater than a preset load threshold. For example... Figure 4 As shown, the dashed box represents the preset loading threshold, such as... Figure 4 As illustrated, if the size of the blank area exceeds a preset loading threshold, step 1208 can proceed, and the loading / storage engine 1104 will execute a loading procedure to load data. The loading / storage engine 1104 will then update the starting position of the blank area. If the situation is not as described... Figure 4 As shown, if the size of the blank area is not greater than the preset loading threshold, then return to step 1202.
[0054] In this embodiment, when space becomes available in the note storage 102, it will proactively notify the load / storage engine 1104 directly via a clear command. Therefore, compared to... Figure 3 The first indicator I1 needs to be continuously checked at the preset loading time point. Figure 12 The proposed implementation is more efficient.
[0055] In this embodiment, when the loading procedure is executed in step 1208, the complete data is not always loaded at once, but rather in multiple loads, with only a portion of the data loaded each time. Therefore, the preset loading threshold represents the size of the data to be loaded next time. When loading in multiple loads, the data is not necessarily divided into equal parts of a fixed size, so the preset loading threshold can be variable.
[0056] Figure 13 The flowchart is related to Figure 11 The process of executing computational threads by each stream processor of the GPU 1100. In step 1302, the stream processor determines whether a data arrival notification has been received. This data arrival notification is issued by the load / store engine 1104 based on the execution status of the loader, indicating that data has arrived at the note storage 102. Details are described in... Figure 15 .
[0057] If no data arrival notification has been received, the process will remain in step 1302 until a data arrival notification is received, at which point it will proceed to step 1304. In step 1304, the stream processor checks the second indicator I2 and the third indicator I3 to obtain the size of the area in the note storage 102 where the data is not in use. In step 1306, the stream processor determines whether the size of the area where the data is not in use is greater than a preset calculation threshold. Figure 6 As shown, the dashed box represents the preset calculation threshold, such as... Figure 6 The illustration shows that when the size of the area where data is not in use exceeds a preset calculation threshold, it indicates that a calculation thread bundle is about to be executed. At this point, step 1308 is initiated, whereby the stream processor sends a consumption command to the load / storage engine 1104 as a notification. Then, step 1310 is initiated, causing the stream processor to execute the calculation thread bundle. If the third indicator I3 is stored outside the load / storage engine 1104, the stream processor will modify the third indicator I3 to indicate that the state of a portion of the note memory 102 has changed from data not in use to data in use. If the situation is not as described... Figure 6 As shown, if the size of the region where the data is not in use is not greater than the preset calculation threshold, then return to step 1302.
[0058] In this embodiment, when new data arrives at the note storage 102, the loading / storage engine 1104 will proactively notify the stream processor directly via a data arrival notification. Therefore, compared to... Figure 5The second indicator I2 and the third indicator I3 need to be checked continuously at the preset calculation time points. Figure 13 The proposed implementation is more efficient.
[0059] In this embodiment, when executing the calculation thread bundle in step 1310, it is not always necessary to wait for all the complete data to arrive at the note storage 102 before starting the calculation thread bundle. Instead, it is divided into multiple calculations, each using only a portion of the data for calculation. Therefore, the preset calculation threshold represents the size of the data to be used for the next calculation. When performing multiple calculations, the data is not necessarily divided into equal portions of a fixed size, so the preset calculation threshold can be variable.
[0060] Figure 14 The flowchart is related to Figure 11 The GPU 1100's load / store engine 1104 executes the load thread bundle process. In step 1402, the load / store engine 1104 determines whether any new data has been loaded into the note storage 102. For example, after executing the load program in step 1208, data is read from global memory and loaded into the note storage 102. Figure 14 The flowchart will then proceed from step 1402 to step 1404. If no new data is loaded into the note storage 102, the process will continue in step 1402 until new data is loaded into the note storage 102.
[0061] In step 1404, the load / store engine 1104 modifies the second index I2 based on the size of the newly loaded data into the note storage 102 to indicate that the state of a portion of the note storage is changed from "loading data" to "data not in use". For example... Figure 8 As shown, the dashed boxes represent newly loaded data into the note storage 102. Next, the load / storage engine 1104 issues a data arrival notification in step 1406.
[0062] Figure 15 The flowchart is related to Figure 11The process of executing computational thread bundles by each stream processor of the GPU 1100. In step 1502, the stream processor determines if there is newly added data in the note memory 102 that is no longer needed. For example, when executing the computational thread bundle in step 1310, data in the note memory 102 is used. Even if the computational thread bundle has not yet finished executing, some data may have been determined to no longer be used; this data is the newly added data that is no longer needed mentioned in step 1502. Whenever newly added data that is no longer needed is found, step 1504 is entered to issue a clear command to notify the load / storage engine 1104 to modify the first indicator I1 to indicate that the state of a certain area of the note memory 102 has changed from "data in use" to "blank," indicating that the space occupied by this data in the note memory 102 can be released. Figure 10 As shown, the dashed boxes represent newly added data that is no longer needed.
[0063] Figure 16 The flowchart is related to Figure 11 The GPU 1100's load / store engine 1104 executes the process of loading the computation thread bundle. In step 1602, the load / store engine 1104 determines whether a consumption command has been received. The consumption command is issued by the stream processor based on the status of the computation thread bundle, indicating that a portion of the data in the note memory 102 is about to be used to execute the computation thread bundle. The consumption command may include the starting position and length of the data in the note memory 102 that is expected to be used by the computation thread bundle.
[0064] If it is determined that no consumption command has been received, the process will remain in step 1602 until a consumption command is received, at which point it will proceed to step 1604. In step 1604, the load / storage engine 1104 will modify the third indicator I3 according to the consumption command to change the status of a portion of the note storage 102 from non-data use to data use.
[0065] Figure 17 for Figure 11 A schematic diagram of an embodiment of the load / store engine 1104 is provided. The load / store engine 1104 includes a command parser 1702, an initial information accessor 1704, a loader 1706, a register 1708, and a notification unit 1710. The register 1708 is used to store a first index I1, a second index I2, and / or a third index I3. The command parser 1702 is used to receive clear commands, consumption commands, and initial information. The clear commands and consumption commands have been described previously. The initial information includes, for example, thread block primitives. After analyzing the thread block primitives, the command parser 1702 can determine the addresses of the data to be loaded in global memory corresponding to each thread bundle in the execution thread block TB, as well as the loading order, and stores this information in the command parser 1702.
[0066] Under the control of command parser 1702 and initial information accessor 1704, loader 1706 executes the loading program and correspondingly modifies the first index I1, second index I2, and / or third index I3 recorded in register 1708. The first index I1 and second index I2 can be read from register 1708. Loader 1706 also causes notifier 1710 to issue a data arrival notification based on the status of the loading program execution.
[0067] The above Figure 12 , Figure 13 , Figure 14 , Figure 15 and Figure 16 The provided embodiments can further improve the efficiency of the GPU when executing thread bundles by actively notifying state changes, thereby further increasing the number of thread bundles that the GPU can process per unit time.
[0068] The foregoing description briefly outlines the features of certain embodiments of this application, enabling those skilled in the art to more fully understand the various forms of this disclosure. Those skilled in the art will readily recognize that this disclosure serves as a basis for designing or modifying other processes and structures to achieve the same objectives and / or advantages as the embodiments described herein. Those skilled in the art should understand that these equivalent embodiments remain within the spirit and scope of this disclosure, and various changes, substitutions, and modifications can be made without departing from the spirit and scope of this disclosure.
Claims
1. A method of thread warp execution for a plurality of streaming processors of a streaming processing unit of a GPU, the method comprising: The multiple stream processors share a note memory, wherein the space in the note memory is in one of the following states: blank, in use of data, not in use of data, or loading data. The method includes: When the preset thread load loading time has arrived, a first indicator is checked to obtain the size of the blank area in the note storage, in order to determine whether to execute the thread load loading. The first indicator is used to indicate the start position of the area in the note storage where data is in use and the end position of the blank area. When the preset calculation time point has arrived, check the second and third indicators to obtain the size of the area in the note storage where the data is not in use, so as to determine whether to execute the calculation thread bundle. The second indicator is used to indicate the end position of the area in the note storage where the data is not in use and the start position of the area where the data is loaded. The third indicator is used to indicate the end position of the area in the note storage where the data is in use and the start position of the area where the data is not in use.
2. The method as described in claim 1, characterized in that, The steps for determining whether to execute the loaded thread bundle include: When the size of the blank area in the note storage is greater than a preset loading threshold, the loading thread bundle is executed.
3. The method of claim 1, wherein, The steps for determining whether to execute the computation thread bundle include: When the size of the area in the note storage that is in a state of "data not in use" is greater than a preset calculation threshold, the calculation thread bundle is executed, and the third indicator is modified to indicate that the state of a portion of the note storage area changes from "data not in use" to "data in use".
4. The method of claim 1, wherein, Also includes: When new data is loaded into the note storage, the second index is modified according to the size of the newly loaded data to indicate that the state of a portion of the note storage changes from "data being loaded" to "data not in use".
5. The method of claim 1, wherein, Also includes: When the computation thread bundle has only partially completed execution and it is determined that there is no longer any data in the note storage, the first indicator is modified to indicate that the state of a portion of the note storage is changed from "data in use" to "blank".
6. The method of claim 1, wherein, The first indicator and the second indicator are stored in the note storage.
7. The method of claim 1, wherein, The third indicator is stored in the note storage or outside the note storage.
8. A GPU comprising: The GPU includes: Stream processing unit, including: A stream processor for performing the method as described in any one of claims 1 to 7; and The note storage device.
9. A method of thread warp execution for a plurality of stream processors of a streaming multiprocessor of a GPU, the method comprising: The multiple stream processors share a note memory, wherein the space in the note memory is in one of the following states: blank, in use of data, not in use of data, or loading data. The method includes: When a data arrival notification is received from the load / storage engine, a second and a third indicator are checked to determine the size of the region in the note storage where the data is not in use, in order to decide whether to execute the computation thread bundle. The second indicator indicates the end position of the region in the note storage where the data is not in use and the start position of the region where the data is loaded. The third indicator indicates the end position of the region in the note storage where the data is in use and the start position of the region where the data is not in use. When it is determined that the operation thread bundle is to be executed, a consumption command is issued to the loading / storage engine.
10. The method of claim 9, wherein, The steps for determining whether to execute the computation thread bundle include: When the size of the area in the note storage where the data is not in use is greater than a preset calculation threshold, the calculation thread bundle is executed.
11. The method as described in claim 9, characterized in that, The consumption command includes the starting position and length of the data in the note storage that is expected to be used by the computation thread bundle, so as to control the loading / storage engine to modify the third indicator to indicate that the state of a part of the note storage is changed from data not in use to data in use.
12. The method of claim 9, wherein, Also includes: When the computation thread has only partially completed its execution and it is determined that there is no longer any data in the note storage that needs to be used, a clear command is issued to the loading / storage engine.
13. The method of claim 12, wherein, The clear command includes the starting position and length of the data that is no longer needed, so as to control the loading / storage engine to modify the first indicator to indicate that the state of a portion of the note storage area changes from "data in use" to "blank".
14. The method of claim 13, wherein, The load / store engine determines whether to execute the load program based on the clear command.
15. The method of claim 14, wherein, When the size of the blank area in the note storage is greater than a preset loading threshold, the loading / storage engine executes the loading program.
16. The method of claim 9, wherein, When new data is loaded into the note storage, the loading / storage engine modifies the second indicator based on the size of the newly loaded data to indicate that the state of a portion of the note storage changes from "loading data" to "data not in use," and issues a data arrival notification.
17. The method of claim 13, wherein, The first metric is stored in the loading / storage engine.
18. The method of claim 9, wherein, The second metric is stored in the loading / storage engine.
19. The method of claim 9, wherein, The third indicator is stored in the loading / storage engine or outside of the loading / storage engine.
20. A GPU, characterized in that, The GPU includes: Stream processing unit, including: A stream processor for performing the method as described in any one of claims 9 to 19; The note storage device; and The loading / storage engine.