Method, computing device, medium, and program product for performing input / output and computation operations in parallel

Abstract

The present application relates to a method, a computing device, a medium and a program product for performing input / output and computing operations of data in parallel. The method comprises: dividing at least one barrier configured by a graphics processor into a plurality of sub-barriers, such that a sub-barrier pass instruction and a sub-barrier wait instruction corresponding to each sub-barrier are associated with a same sub-barrier identifier; dividing data to be loaded into a plurality of sub-data, such that each sub-data is associated with a sub-barrier identifier; inserting a corresponding sub-barrier wait instruction before a computing instruction of each sub-data; loading the plurality of sub-data, and issuing a corresponding sub-barrier pass instruction at the same time when each sub-data is loaded; and in response to determining that the sub-barrier wait instructions associated with the same sub-barrier identifier wait for the corresponding sub-barrier pass instructions, causing a computing unit to perform a computing operation of the corresponding sub-data, such that the computing operation for the sub-data is performed in parallel with the loading operation for at least one sub-data.

Description

TECHNICAL FIELD

[0001] Embodiments of the present application generally relate to the field of artificial intelligence, and more particularly to a method, a computing device, a computer readable storage medium and a computer program product for parallel execution of input / output and computation operations of data. BACKGROUND

[0002] Conventional methods for parallel execution of input / output (IO) and computation operations, for example, use a ping-pong pipeline approach. For example, there are 100 data, which are divided into two groups of ping data and pong data, each group having 50 data. Taking loading data and computing data as an example, first, an instruction for loading the ping data (i.e., a "load ping" instruction) is executed, and at the same time, an instruction for loading the pong data (i.e., a "load pong" instruction) is executed. As shown in FIG. 6, the loading of the pong data (50 data) indicated by marker 604 and the computing of the ping data (50 data) indicated by marker 606 can be executed in parallel. However, the loading of the ping data (50 data) indicated by marker 602 and the computing of the pong data (50 data) indicated by marker 608 are not executed in parallel. Therefore, the loading operation of the 50 data (ping data) at the head of the pipeline and the computing operation of the 50 data (pong data) at the tail of the pipeline cannot be parallel. Due to the limited number of barriers of the image processor, this results in a large pipeline granularity of IO and computation operations, a long loading time at the head of the pipeline and a long computing time at the tail of the pipeline, and the parallel time cannot be maximized. Figure 6

[0003] In summary, the conventional method for parallel execution of input / output and computation operations of data has the following disadvantages: the pipeline granularity of IO and computation is large, the loading time at the head of the pipeline and the computing time at the tail of the pipeline are long, and the parallel time cannot be maximized. SUMMARY

[0004] The present application provides a method, a computing device, a computer readable storage medium and a computer program product for parallel execution of input / output and computation operations of data, which can significantly reduce the loading time at the head of the pipeline and the computing time at the tail of the pipeline, and is beneficial to maximize the parallel time.

[0005] ​According to a first aspect of the present application, there is provided a method for performing input / output and computation operations of data in parallel, the method comprising: dividing at least one barrier configured for a graphics processing unit into a plurality of sub-barriers, such that a sub-barrier pass instruction and a sub-barrier wait instruction corresponding to each of the sub-barriers are associated with a same sub-barrier identification; dividing data to be loaded into a plurality of sub-data, such that each of the sub-data is associated with a sub-barrier identification; inserting the corresponding sub-barrier wait instruction before a computation instruction of each of the sub-data; loading the plurality of sub-data, and issuing the corresponding sub-barrier pass instruction at a same time when each of the sub-data is loaded; and in response to determining that the sub-barrier wait instructions associated with the same sub-barrier identification wait for the corresponding sub-barrier pass instructions, causing a computation unit to perform a computation operation of the corresponding sub-data, such that the computation operation of the sub-data is performed in parallel with a loading operation of at least one of the sub-data.

[0006] According to a second aspect of the present application, there is also provided a computing device. The computing device comprises: at least one processor; and a memory communicatively connected with the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the computing device to perform the method of the first aspect of the present application.

[0007] According to a third aspect of the present application, there is also provided a computer-readable storage medium. The computer-readable storage medium stores a computer program, and the computer program is executed by a machine to perform the method of the first aspect of the present application.

[0008] According to a fourth aspect of the present application, there is also provided a computer program product comprising a computer program, and the computer program is executed by a machine to perform the method of the first aspect of the present application.

[0009] In some embodiments, dividing the data to be loaded into the plurality of sub-data, such that each of the sub-data is associated with a sub-barrier identification comprises: configuring an association between the sub-data and the sub-barrier identification based on a number of the sub-data and a number of the sub-barriers, such that each of the sub-data is associated with a sub-barrier identification.

[0010] In some embodiments, configuring the association between the sub-data and the sub-barrier identification based on the number of the sub-data and the number of the sub-barriers, such that each of the sub-data is associated with a sub-barrier identification comprises: in response to determining that the number of the sub-data is greater than the number of the sub-barriers, configuring the association between the sub-data and the sub-barrier identification based on a round-robin mechanism, such that each of the sub-data is associated with a sub-barrier identification.

[0011] In some embodiments, the multiple sub-data are loaded, and the corresponding sub-barrier pass instruction is issued at the same time when each sub-data is loaded completely, including: loading the current sub-data associated with the current sub-barrier identifier from the high-bandwidth memory to the thread register, and issuing the sub-barrier pass instruction corresponding to the current sub-barrier identifier at the same time when the current sub-data is loaded completely; and loading the next sub-data associated with the next sub-barrier identifier from the high-bandwidth memory to the thread register, and issuing the sub-barrier pass instruction corresponding to the next sub-barrier identifier at the same time when the next sub-data is loaded completely.

[0012] In some embodiments, the number of sub-barriers divided by each barrier is associated with a thread group allocated by each computing unit in a computing core performing sub-data calculation, and the computing core includes a plurality of computing units.

[0013] In some embodiments, the computing core is a vector core, and dividing each barrier configured by the graphics processor into a plurality of sub-barriers includes: dividing each barrier in the plurality of barriers configured by the graphics processor into a plurality of sub-barriers via a thread group computing unit included in a computing unit of the vector core, each vector core including a plurality of computing units, and each computing unit in the plurality of computing units being associated with a plurality of thread groups.

[0014] In some embodiments, the multiple sub-data are loaded, and the corresponding sub-barrier pass instruction is issued at the same time when each sub-data is loaded completely, including: loading each sub-data in the multiple sub-data from the high-bandwidth memory to the thread register in parallel or sequentially via the computing unit; and issuing the corresponding sub-barrier pass instruction at the same time when each sub-data is loaded completely based on the sub-barrier identifier associated with each sub-data.

[0015] The present application divides a barrier in a graphics processor into multiple more fine-grained sub-barriers, each of which has a unique sub-barrier identifier. The data block to be loaded is divided into multiple sub-data, and each sub-data is associated with a sub-barrier identifier. In this way, by splitting the barrier and the data to be loaded, the present application can reduce the granularity of IO and computing data, and make smaller granularity, more sub-data associated with the corresponding sub-barrier. In addition, by inserting the corresponding sub-barrier waiting instruction before the computing instruction of each sub-data, issuing the corresponding sub-barrier pass instruction as soon as each sub-data block is loaded, and making the computing unit execute the computing operation of the corresponding sub-data in response to determining that the sub-barrier waiting instructions associated with the same sub-barrier identifier are waiting for the corresponding sub-barrier pass instruction, the present application not only can realize the parallel loading of multiple sub-data and the parallel of the loading of sub-data and the computing operation, but also can shorten the time of the head loading data and the tail computing operation, and further optimize the overall time of the operator to be smaller. Therefore, the present application can significantly reduce the loading time of the head of the pipeline and the computing time of the tail of the pipeline, which is beneficial to maximize the parallel time.

[0016] It should be understood that the matters described in this section are not intended to identify key or essential features of embodiments of the present application, nor is it intended to limit the scope of the present application. Other features of the present application will become apparent from the following description. BRIEF DESCRIPTION OF DRAWINGS

[0017] The above and other features, advantages and aspects of embodiments of the present application will become more apparent by describing in detail some embodiments thereof with reference to the attached drawings in which:

[0018] Figure 1 A schematic diagram of a computing device implementing a method for parallel execution of input / output and computing operations of data according to embodiments of the present application is schematically shown.

[0019] Figure 2 A flowchart of a method for parallel execution of input / output and computing operations of data according to some embodiments of the present application is shown.

[0020] Figure 3 A schematic diagram of a method for parallel execution of input / output and computing operations of data according to some embodiments of the present application is shown.

[0021] Figure 4 A schematic diagram of an architecture of a graphics processor according to some embodiments of the present application is shown.

[0022] Figure 5A flowchart is shown below illustrating a method for issuing a corresponding sub-barrier passage instruction at the same time as each piece of sub-data is loaded, according to some embodiments of the present invention.

[0023] Figure 6 This diagram illustrates the effect of traditional methods for parallel execution of data input / output and computational operations.

[0024] Figure 7 A schematic diagram illustrating the effect of a method for performing data input / output and computation operations in parallel according to some embodiments of the present invention is shown.

[0025] In the various figures, the same or corresponding reference numerals indicate the same or corresponding parts. Detailed Implementation

[0026] Preferred embodiments of the invention will now be described in more detail with reference to the accompanying drawings. While preferred embodiments of the invention are shown in the drawings, it should be understood that the invention can be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that the invention will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art.

[0027] The term "comprising" and its variations as used herein signify open inclusion, i.e., "including but not limited to". Unless otherwise stated, the term "or" means "and / or". The term "based on" means "at least partially based on". The terms "one example embodiment" and "one embodiment" mean "at least one example embodiment". The term "another embodiment" means "at least one additional embodiment". The terms "first", "second", etc., may refer to different or the same objects.

[0028] As described above, the shortcomings of traditional methods for parallel execution of data input / output and computation operations are that the granularity of I / O and computation pipelines is too large, the loading time at the head of the pipeline and the computation time at the tail of the pipeline are too long, and the parallel time cannot be maximized.

[0029] To at least partially address one or more of the aforementioned problems and other potential issues, exemplary embodiments of the present invention provide at least one method for parallel execution of data input / output and computational operations. In this method, at least one of a plurality of barriers in a graphics processor is divided into a plurality of finer-grained sub-barriers, each sub-barrier having a unique sub-barrier identifier. The data to be loaded is divided into multiple sub-data portions, and each sub-data portion is associated with a sub-barrier identifier. Thus, by splitting barriers and the data to be loaded, the present invention can reduce the granularity of I / O and computational data, and associate smaller-grained, more portions of sub-data with corresponding sub-barriers. Furthermore, by inserting a corresponding sub-barrier wait instruction before the computation instruction for each portion of sub-data; issuing a corresponding sub-barrier pass instruction immediately upon completion of loading each portion of sub-data; and responding to the determination of sub-barrier wait instructions associated with the same sub-barrier identifier and receiving the corresponding sub-barrier pass instruction, the computational unit executes the computational operation for the corresponding sub-data. The present invention not only enables parallel loading of multiple portions of sub-data and parallel loading and computational operations of sub-data, but also shortens the time for loading head data and performing tail computational operations, thereby optimizing and reducing the overall operator runtime. Therefore, the present invention can significantly reduce the loading time at the head of the pipeline and the computation time at the tail of the pipeline, which is beneficial to maximizing parallel time.

[0030] The following combination Figure 1 A computing device 100 according to an embodiment of the present invention is described, which implements a method for parallel execution of data input / output and computation operations. For example... Figure 1 As shown, the computing device 100 may have one or more processing units and a memory communicatively connected to the one or more processing units; wherein the memory stores instructions executable by the at least one processor. The processing units include dedicated processing units such as Graphics Processing Units (GPUs), Field Programmable Gate Arrays (FPGAs), Application Specific Integrated Circuits (ASICs), and General-purpose computing on graphics processing units (GPGPUs), as well as general-purpose processing units such as CPUs. The computing device 100 also includes at least: a barrier partitioning module 102, a sub-data and sub-barrier identifier association module 104, a sub-barrier waiting instruction insertion module 106, a sub-barrier passage instruction issuance module 108, and a computing unit execution sub-data computation operation module 110.

[0031] Regarding the barrier partitioning module 102, it is used to divide at least one barrier configured by the graphics processor into multiple sub-barriers, such that the sub-barrier passage instruction and sub-barrier waiting instruction corresponding to each sub-barrier are associated with the same sub-barrier identifier.

[0032] Regarding the sub-data and sub-barrier identifier association module 104, it is used to divide the data to be loaded into multiple sub-data, so that each sub-data is associated with a sub-barrier identifier.

[0033] Regarding the sub-barrier wait instruction insertion module 106, it is used to insert the corresponding sub-barrier wait instruction before the calculation instruction of each piece of data.

[0034] Regarding the sub-barrier passage instruction issuing module 108, it is used to load multiple sub-data sets and issue the corresponding sub-barrier passage instruction as each sub-data set is loaded.

[0035] Regarding the calculation unit performing sub-data calculation operation module 110, it is used to respond to the determination of a sub-barrier waiting instruction associated with the same sub-barrier identifier and the corresponding sub-barrier passage instruction, so that the calculation unit performs the corresponding sub-data calculation operation, so that the calculation operation for the sub-data is parallel to the loading operation for at least one copy of the sub-data.

[0036] The following will combine Figure 2 , Figure 3 and Figure 4 This invention describes a method 200 for parallel execution of data input / output and computational operations, according to embodiments of the present invention. It should be understood that method 200 can, for example, be implemented in... Figure 1 The described computing device 100 performs the operation. Method 200 may also include additional actions not shown and / or the actions shown may be omitted; the scope of the invention is not limited in this respect.

[0037] At step 202, the computing device 100 divides at least one barrier configured by the graphics processor into a plurality of sub-barriers, such that the sub-barrier pass instruction and sub-barrier wait instruction corresponding to each sub-barrier are associated with the same sub-barrier identifier.

[0038] Regarding the computing device 100, it includes, for example, an artificial intelligence chip for parallel execution of data input / output and computational operations. It should be understood that the computing device 100 may also include other components besides the artificial intelligence chip. The artificial intelligence chip is, for example, but not limited to, […]. Figure 4 The graphics processing unit (or GPU) shown. A GPU 400, for example, includes one or more computing cores (such as... Figure 4 The example in the text includes a computation core 404, a high-bandwidth memory 402 (i.e., HBM), and one or more barriers 418. Figure 4Only one barrier is shown in the example. The computational core 404 is, for example, a vector core, and can include multiple computational units (such as...). Figure 4 The example shows the first computing unit 406 and the second computing unit 408.

[0039] Regarding barrier 418, also known as a fence, it is, for example, a thread synchronization mechanism in a graphics processing unit (GPU) used to coordinate the execution order of multiple threads within the same thread group. The GPU 400 may include one or more barriers 418, for example, two barriers, three barriers, etc. It should be understood that this invention divides barrier 418 into multiple sub-barriers. For example... Figure 4 The first sub-barrier 420 and the second sub-barrier 422 are shown. It should be noted that when the GPU 400 includes multiple barriers, each barrier can be divided into multiple sub-barriers (such as 4 or 8), or some barriers can be divided into multiple sub-barriers. This invention does not limit the division of barriers. The specific division method depends on the resource limitations of the graphics processor, the granularity of the data to be loaded, and the mechanism for cyclically allocating sub-barriers.

[0040] A sub-barrier can be an independent synchronization channel within a barrier resource. A sub-barrier can be assigned a sub-barrier identifier (such as a slot ID), which can be a unique identifier for the sub-barrier. The sub-barrier identifier can be in the form of a number, string, etc., for example, a sub-barrier identifier can be slot 0, slot 1, slot 2, slot 3, etc. Each sub-barrier includes, for example, a counter, comparison logic, and a thread scheduler, where the counter is initialized to 0. When the thread associated with the corresponding sub-barrier identifier (such as slot ID) executes the corresponding sub-barrier pass instruction, it notifies the sub-barrier, causing the sub-barrier's counter to atomically increment by a predetermined value. Once the counter reaches a target value (such as a predetermined value), the hardware (e.g., a computing unit) releases the thread associated with the barrier wait instruction associated with the sub-barrier identifier (such as slot ID), allowing that thread to continue executing instructions following the corresponding sub-barrier wait instruction (e.g., executing the computation instruction for the corresponding sub-data). It should be understood that before the counter reaches the target value, the thread associated with the sub-barrier wait instruction is suspended, and the computation operation of the corresponding sub-data following the corresponding sub-barrier wait instruction cannot be executed.

[0041] It should be understood that the number of sub-barriers divided into each barrier is related to the thread group allocated to each computing unit in the computing core that performs the sub-data computation. In some embodiments, each of the multiple barriers configured by the graphics processor (e.g., barrier 418) can be divided into multiple sub-barriers via the thread group computing unit 412 included in the computing unit (e.g., the first computing unit 406) of the computing core 404. Figure 4Only the first sub-barrier 420 and the second sub-barrier 422 are shown schematically.

[0042] Regarding the thread group computing unit 412, it is configured, for example, at least to divide each of the multiple barriers configured by the graphics processor into multiple sub-barriers.

[0043] In some embodiments, each sub-barrier may correspond to a sub-barrier pass instruction and a sub-barrier wait instruction. The sub-barrier pass instruction and sub-barrier wait instruction corresponding to a sub-barrier are associated with the sub-barrier identifier corresponding to that sub-barrier.

[0044] At step 204, the computing device 100 divides the data to be loaded into multiple sub-data, such that each sub-data is associated with a sub-barrier identifier.

[0045] The data to be loaded is, for example, data loaded into the thread registers of the computation unit of the vector kernel.

[0046] In some embodiments, the data to be loaded can be divided into multiple sub-data based on the hardware resources of the graphics processor. These hardware resources may include the size of the warp, the bandwidth for memory transfer, and the number of sub-barriers. The data to be loaded can be divided sequentially; for example, if the total amount of data to be loaded is 100, it can be divided sequentially into four sub-data such as sub-data 1 (data 1-25), sub-data 2 (data 26-50), sub-data 3 (data 51-75), and sub-data 4 (data 76-100), or other division methods may be used.

[0047] Regarding the number of sub-data sets, it is, for example, greater than or equal to the number of sub-barriers.

[0048] Methods for associating each piece of data with a sub-barrier identifier include, for example, configuring the association between sub-data and sub-barrier identifiers based on the number of sub-data pieces and the number of sub-barriers, so that each piece of sub-data is associated with each sub-barrier identifier. The association can be a mapping rule that assigns each piece of sub-data (e.g., from 0 to N-1, where the number of sub-data pieces is N, and N is a positive integer) a sub-barrier identifier (e.g., from 0 to M-1, where the number of sub-barriers is M, and M is a positive integer). The association can take the form of an association table, an association model, etc.

[0049] When the number of sub-data sets is the same as the number of sub-barriers, a one-to-one mapping relationship between sub-data sets and sub-barrier identifiers can be established. Table 1 shows the association relationship between sub-data sets and sub-barrier identifiers in some embodiments. As shown in Table 1, when the number of sub-data sets and the number of sub-barriers are both 4, the sub-barrier identifier associated with sub-data set 0 is slot0, the sub-barrier identifier associated with sub-data set 1 is slot1, the sub-barrier identifier associated with sub-data set 2 is slot2, and the sub-barrier identifier associated with sub-data set 3 is slot3.

[0050] Table 1

[0051]

[0052] When the number of sub-data sets exceeds the number of sub-barriers, a polling mechanism can be used to configure the association between sub-data sets and sub-barrier identifiers, so that each sub-data set is associated with a single sub-barrier identifier. When a sub-barrier associated with the same sub-barrier identifier receives a pass instruction (e.g., pass slot 0) while waiting for a command (e.g., wait slot 0), its task in the current synchronization event is complete, and its state can be cleared and used for the next sub-data set requiring synchronization. This achieves efficient recycling of limited synchronization resources.

[0053] For example, when the number of sub-data units N is 8 and the number of sub-barriers M is 4, the sub-barrier identifier associated with the i-th sub-data unit (i is any integer from 0 to N-1) can be determined by i mod M. For example, for the 5th data unit, 5 mod 4 = 1, so the sub-barrier identifier associated with the 5th data unit is slot 1. Table 2 shows the association between sub-data units and sub-barrier identifiers in some other embodiments. As shown in Table 2, the sub-barrier identifiers associated with sub-data units 0 to 3 are slot 0, slot 1, slot 2, and slot 3, respectively. For sub-data unit 4, since the sub-data unit 0 associated with slot 0 has been calculated and its barrier state has been reset, the sub-barrier can be assigned to sub-data unit 4. Similarly, for sub-data unit 5, since the sub-data unit 1 associated with slot 1 has been calculated and its barrier state has been reset, the sub-barrier can be assigned to sub-data unit 5. This process continues until the sub-barrier identifiers associated with all sub-data units are determined.

[0054] Table 2

[0055]

[0056] At step 206, a corresponding sub-barrier waiting instruction is inserted before the calculation instruction for each piece of data.

[0057] The sub-barrier wait instruction is a blocking instruction used to ensure that computational operations on sub-data are executed only after the corresponding sub-data has been loaded. It can be understood that each piece of sub-data can correspond to one sub-barrier wait instruction and one sub-barrier pass instruction.

[0058] like Figure 3 As shown, the calculation instructions for the first subset of data (e.g.) Figure 3 Before (as indicated by marker 308), a sub-barrier wait instruction (i.e., wait slot 0 instruction, such as...) can be inserted for the first piece of sub-data. Figure 3 (As indicated by mark 306). The calculation instructions for the second subset of data (e.g.) Figure 3 Before (as indicated by marker 314), a second sub-barrier wait instruction (i.e., wait slot 1 instruction, such as...) can be inserted. Figure 3 (As indicated by mark 312). In the calculation instructions for the third subset of data (e.g.) Figure 3 Before (as indicated by marker 320), a third sub-barrier wait instruction (wait slot 2 instruction, such as...) can be inserted. Figure 3 (As indicated by mark 318). Similarly, in the calculation instructions for the fourth subset of data (e.g.) Figure 3 Before (as indicated by marker 324), a fourth sub-barrier wait instruction (wait slot 3 instruction, such as...) can be inserted. Figure 3 (As indicated by mark 322).

[0059] like Figure 3 As shown, if the number of sub-data sets is greater than the number of sub-barrier data sets, then for the fifth group of sub-data sets, sub-barrier slot 0 can be assigned to the fifth group of sub-data sets. The calculation instructions for the fifth group of sub-data sets (e.g., ...) Figure 3 Before (as indicated by marker 332), a fifth sub-barrier waiting instruction (e.g., ...) can be inserted. Figure 3 (As indicated by mark 330).

[0060] In step 208, multiple sub-data sets are loaded, and a corresponding sub-barrier passage instruction is issued simultaneously with the loading of each sub-data set.

[0061] Each piece of sub-data can be loaded from high-bandwidth memory 402 into thread register (TLR) 410 via the sub-data loading operation. The sub-barrier pass instruction is a non-blocking instruction, which can be issued immediately when the sub-data loading is complete. It should be understood that multiple load instructions can be issued in parallel or sequentially, but since the sub-barrier pass instruction is a non-blocking instruction, the loading operations of each piece of sub-data will not wait for each other.

[0062] The high-bandwidth memory 402 is used to store data, descriptors, or instructions to be loaded. In some embodiments, each computing unit loads the sub-data required for the computing task and the instructions for performing the computing task from the high-bandwidth memory 402 before performing a computing task (e.g., data addition). The path to obtain the aforementioned sub-data and / or instructions includes, for example, via the computing core bus interface (or "CBI"). Figure 3 (Not shown in the image) goes to the second-level cache (L2 cache, Figure 4 (not shown in the image), and then via a memory link (link). Figure 4 (Not shown in the image) retrieves relevant sub-data and instructions from high-bandwidth memory 402. The retrieved sub-data or instructions can be stored in thread register 410.

[0063] like Figure 4 As shown, execute the instruction to load the first piece of sub-data (such as...) Figure 3 (As indicated by marker 302); and simultaneously after loading the first piece of sub-data, issue the sub-barrier passage instruction corresponding to the first piece of sub-data, namely, the pass slot 0 instruction ( Figure 3 (Not shown in the image). Execute the instruction to load the second sub-data (e.g., ...). Figure 3 (As indicated by marker 304); and simultaneously after loading the second sub-data, issue the sub-barrier passage instruction corresponding to the second sub-data, namely, the pass slot 1 instruction ( Figure 3 (Not shown in the image). Execute the load instruction for loading the third sub-data (e.g., ...). Figure 3 (As indicated by marker 310); and simultaneously after loading the third sub-data, issue the sub-barrier passage instruction corresponding to the third sub-data, namely, the pass slot 2 instruction ( Figure 3 (Not shown in the image). Similarly, execute the instruction to load the fourth sub-data (e.g., ...). Figure 3 (As indicated by marker 316); and simultaneously, upon completion of loading the fourth sub-data, issue the sub-barrier passage instruction corresponding to the fourth sub-data, namely, the pass slot 3 instruction ( Figure 3 (Not shown in the image). Additionally, for the fifth sub-data, execute the instruction to load the fifth sub-data (e.g., ...). Figure 3 (As indicated by marker 326); and simultaneously after loading the fifth sub-data, issue the sub-barrier passage instruction corresponding to the first sub-data, namely, the passslot 0 instruction ( Figure 3 (Not shown in the image). For the sixth subdata, execute the instruction to load the sixth subdata (e.g., ...). Figure 3 (As indicated by marker 328); and simultaneously after loading the sixth sub-data, issue the sub-barrier passage instruction corresponding to the first sub-data, namely, the pass slot 1 instruction ( Figure 3 (Not shown in the image).

[0064] The method for loading multiple sub-data sets and issuing a corresponding sub-barrier passage command upon completion of loading each sub-data set includes, for example, loading the current sub-data set associated with the current sub-barrier identifier from a high-bandwidth memory into a thread register, and issuing a sub-barrier passage command corresponding to the current sub-barrier identifier upon completion of loading the current sub-data set; and loading the next sub-data set associated with the next sub-barrier identifier from a high-bandwidth memory into a thread register, and issuing a sub-barrier passage command corresponding to the next sub-barrier identifier upon completion of loading the next sub-data set. The method 500 for issuing the corresponding sub-barrier passage command upon completion of loading each sub-data set will be discussed in conjunction with... Figure 3 A detailed explanation will not be repeated here.

[0065] In step 210, in response to determining that a sub-barrier waiting instruction associated with the same sub-barrier identifier has been received, the corresponding sub-barrier passage instruction is received, causing the computing unit to perform the corresponding sub-data calculation operation, so that the calculation operation for the sub-data is parallel to the loading operation for at least one copy of the sub-data.

[0066] For example, in response to the sub-barrier wait instruction (e.g., wait slot 0 instruction, such as...) for the first piece of sub-data. Figure 5 As indicated by marker 306, once the sub-barrier passage instruction for the first sub-data (i.e., the pass slot 0 instruction; it should be understood that the wait slot 0 instruction and the pass slot 0 instruction for the first sub-data are associated with the same sub-barrier identifier "slot 0") is received, the computation unit (such as the first computation unit 406) can perform the computation operation (e.g., summation operation) for the first sub-data. In this way, the loading operation for the second sub-data and the computation operation for the first sub-data are executed in parallel.

[0067] Similarly, in response to the sub-barrier wait instruction for the second sub-data (i.e., the wait slot 1 instruction, such as...) Figure 3When the instruction to pass slot 1 for the second sub-data (indicated by marker 312) is received, the computation unit (such as the first computation unit 406) can execute the computation operation for the second sub-data. Thus, the loading operation for the third sub-data and the computation operation for the second sub-data are executed in parallel. Similarly, the loading operation for the fourth sub-data and the computation operation for the third sub-data are executed in parallel. It can be understood that, in the case of four sub-data, the loading operation for the first sub-data (header data) and the computation operation for the fourth data (tail data) are not parallel. Because the pass and wait instructions based on more sub-barriers enable the parallel execution of more loading and computation operations, the pipeline granularity is finer, significantly reducing the time of non-parallel operations at the beginning and end, thereby significantly reducing the time required to complete the entire data processing task, and thus maximizing parallel time. Furthermore, by precisely matching the loading of sub-data with the same sub-barrier identifier and releasing the corresponding sub-barrier's wait instruction after loading, it is ensured that the computation operation for each sub-data always occurs after the corresponding sub-data has been loaded, thus avoiding data races and conflicts.

[0068] The following will combine Figure 3 and Figure 4 This invention describes a method 500 for issuing a corresponding sub-barrier passage command simultaneously with the completion of loading each piece of sub-data. It should be understood that method 500 can, for example, be used in... Figure 5 The described computing device 100 performs the operation. Method 500 may also include additional actions not shown and / or the actions shown may be omitted; the scope of the invention is not limited in this respect. It should be understood that... Figure 1 The diagram also illustrates the data flow between relevant components of a computing device (e.g., GPU 400) during the parallel execution of data input / output and computational operations.

[0069] like Figure 4 As shown, in step 502, the computing device 100 loads the current sub-data associated with the current sub-barrier identifier from the high-bandwidth memory into the thread register, and issues the sub-barrier passage instruction corresponding to the current sub-barrier identifier at the same time as the current sub-data is loaded.

[0070] The high-bandwidth memory 402 can store multiple sub-data, which together form a complete set of data to be loaded and computed.

[0071] The thread register 410 is configured to store data required for computation, such as multiple sub-data sets. For example, the sub-data loaded by the first computation unit 406 from the high-bandwidth memory 402 can be cached in the thread register 410. After the first computation unit 406 completes the computation operation on the loaded sub-data, it can cache the computation result in the thread register 410 and send the computation result to the high-bandwidth memory 402 for storage via the thread register 410.

[0072] Taking the loading and calculation of data performed by the first computing unit 406 as an example, the first computing unit 406 can load sub-data (such as...) corresponding to the current sub-barrier identifier from the high-bandwidth memory 402. Figure 5 (As indicated by marker 424). For example, when the current sub-barrier identifier is slot 0, the first piece of sub-data (or simply the first piece of sub-data) can be loaded from the high-bandwidth memory 402. It is understood that the loading of multiple pieces of sub-data can be performed sequentially or in parallel. After the current piece of sub-data has been loaded, the first computing unit 406 can simultaneously issue a sub-barrier passage instruction corresponding to the current sub-barrier identifier, such as issuing a pass slot 0 instruction for the first piece of sub-data.

[0073] In step 504, the computing device 100 loads the next piece of data associated with the next sub-barrier identifier from the high-bandwidth memory into the thread register, and issues the sub-barrier passage instruction corresponding to the next sub-barrier identifier at the same time as the next piece of data is loaded.

[0074] Taking the first computing unit 406 sequentially loading multiple sub-data sets as an example, after loading the first sub-data set, the first computing unit 406 can load the next sub-data set associated with the next sub-barrier identifier, based on the next sub-barrier identifier. For example, if the next sub-barrier identifier is slot 1, the second sub-data set can be loaded from the high-bandwidth memory 402. Simultaneously with the completion of loading the next sub-data set, a sub-barrier passage instruction corresponding to the next sub-barrier identifier is issued. For example, a pass slot 1 instruction for the second sub-data set is issued.

[0075] By repeatedly executing the above steps, multiple sub-data can be continuously loaded and access commands can be issued, ensuring the continuous operation of the pipeline.

[0076] In some embodiments, the first computing unit 406 may also load multiple sub-data in parallel. For example, the first computing unit 406 may simultaneously load multiple sub-data, such as a first sub-data, a second sub-data, a third sub-data, etc., from the high-bandwidth memory 402.

[0077] The following combination Figure 4This will illustrate the effects of some embodiments of the present invention on the methods for parallel execution of data input / output and computational operations. For example... Figure 7 As shown, a barrier can be divided into multiple sub-barriers, which is equivalent to creating multiple miniature synchronization channels. This allows data to be split into more parts with smaller granularity (e.g., splitting from the original 2 parts into 4 or 8 parts), thereby enabling I / O and computation to run in parallel for a longer period of time, reducing the non-parallel time at the beginning and end, and maximizing the parallel time.

[0078] Take, for example, a barrier divided into four sub-barriers. This allows for a reduction in the granularity of I / O and computational data, enabling parallel I / O operations and computations on smaller, more numerous sub-barriers. For instance, loading the second sub-data (such as...) Figure 7 (as indicated by marker 704) and the calculation of the first subset of data (such as...) Figure 7 (As indicated by marker 710) is executed in parallel; loading a third subset of data (such as...) Figure 7 (as indicated by marker 706) and the calculation of the second subset data (such as...) Figure 7 (As indicated by marker 712) is executed in parallel; loading the fourth subset of data (such as...) Figure 7 (as indicated by marker 708) and the calculation of the third component data (such as...) Figure 7 (As indicated by marker 714) is executed in parallel. Loading the first sub-data (such as...) Figure 7 (as indicated by mark 702) and the calculation of the fourth set of data (such as...) Figure 7 (As indicated by marker 716) is not executed in parallel. Figure 7 Figure 7 As shown, although the loading operations for the 25 data points at the beginning of the pipeline and the computation operations for the 25 data points at the end cannot be parallelized, the data volume is relatively small. Therefore, the time for loading data at the beginning and computation operations at the end can be shortened, thereby optimizing and reducing the overall operator runtime. Thus, this invention can significantly reduce the loading time at the beginning of the pipeline and the computation time at the end, which is beneficial for maximizing parallel processing time.

[0079] The various processes and procedures described above, such as methods 200 and 500, can be executed at a computing device. This computing device may include, for example, at least one processor (at least one graphics processor and at least one central processing unit); and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor. In some embodiments, methods 200 and 500 may be implemented as a computer software program or program product tangibly contained in a machine-readable medium. In some embodiments, part or all of the computer program may be loaded and / or installed on the computing device via read-only memory (ROM) and / or a communication unit. When the computer program is loaded into random-access memory (RAM) and executed by the GPU and CPU, one or more actions of methods 200 and 500 described above can be performed.

[0080] This invention can be a method, apparatus, system, and / or computer program product. A computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for performing various aspects of the invention. The computer-readable storage medium may be a tangible device capable of holding and storing instructions used by an instruction execution device. The computer-readable storage medium may be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination thereof.

[0081] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to various computing / processing devices, or downloaded via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network, to an external computer or external storage device. Various aspects of the invention are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0082] These computer-readable program instructions can be provided to the central processing unit of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the central processing unit of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of a flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner. Thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of a flowchart and / or block diagram.

[0083] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction, which contains one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those shown in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0084] It should be understood that the various forms of processes shown above can be used to reorder, add, or delete steps. For example, the steps loaded in this application can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this application can be achieved, and this is not limited herein.

[0085] The specific embodiments described above do not constitute a limitation on the scope of protection of this application. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors.

Claims

1. A method for parallel execution of data input / output and computation operations, characterized in that, The method includes: Divide at least one barrier configured by the graphics processor into multiple sub-barriers, such that the sub-barrier passage instruction and the sub-barrier wait instruction corresponding to each sub-barrier are associated with the same sub-barrier identifier; The data to be loaded is divided into multiple sub-data, so that each sub-data is associated with a sub-barrier identifier; Insert a corresponding sub-barrier waiting instruction before the calculation instruction for each piece of data; Load multiple sub-data sets, and issue corresponding sub-barrier passage instructions upon completion of loading each sub-data set; and In response to determining that a sub-barrier waiting instruction with the same sub-barrier identifier is associated with a corresponding sub-barrier passage instruction, the computing unit performs the corresponding sub-data computation operation so that the computation operation for the sub-data is parallel to the loading operation for at least one copy of the sub-data.

2. The method according to claim 1, characterized in that, The data to be loaded is divided into multiple sub-data sets, each of which is associated with a sub-barrier identifier, including: Based on the number of sub-data sets and the number of sub-barriers, configure the association between sub-data sets and sub-barrier identifiers so that each set of sub-data sets is associated with a sub-barrier identifier.

3. The method according to claim 2, characterized in that, Based on the number of sub-data sets and the number of sub-barriers, configure the association between sub-data sets and sub-barrier identifiers so that each set of sub-data sets is associated with one sub-barrier identifier, including: In response to the determination that the number of sub-data sets is greater than the number of sub-barriers, the association between sub-data sets and sub-barrier identifiers is configured based on a polling mechanism so that each sub-data set is associated with a sub-barrier identifier.

4. The method according to claim 1, characterized in that, Loading multiple sub-data sets, and issuing corresponding sub-barrier passage instructions upon completion of loading each sub-data set, includes: Load the current sub-data associated with the current sub-barrier identifier from the high-bandwidth memory into the thread register, and issue the sub-barrier passage command corresponding to the current sub-barrier identifier simultaneously with the completion of loading the current sub-data; and Load the next piece of data associated with the next sub-barrier identifier from the high-bandwidth memory into the thread register, and issue the sub-barrier passage instruction corresponding to the next sub-barrier identifier at the same time as the next piece of data is loaded.

5. The method according to claim 1, characterized in that, The number of sub-barriers in each barrier is associated with the thread group allocated to each computing unit in the computing core that performs the sub-data computation, wherein the computing core comprises multiple computing units.

6. The method according to claim 5, characterized in that, The computational core is a vector core, which divides at least one barrier configured by the graphics processor into multiple sub-barriers, including: The computational units of the vector core include thread groups, which divide each of the multiple barriers configured by the graphics processor into multiple sub-barriers. Each vector core includes multiple computational units, and each of the multiple computational units is associated with multiple thread groups.

7. The method according to claim 6, characterized in that, Loading multiple sub-data sets, and issuing corresponding sub-barrier passage instructions upon completion of loading each sub-data set, includes: Each sub-data set is loaded from high-bandwidth memory in parallel or sequentially into the thread registers of the computing unit; and Based on the sub-barrier identifier associated with each piece of data, the corresponding sub-barrier passage instruction is issued as soon as each piece of data is loaded.

8. A computing device, characterized in that, include: At least one processor; as well as A memory that is communicatively connected to the at least one processor; in The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-7.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a machine, performs the method according to any one of claims 1-7.

10. A computer program product, characterized in that, Includes a computer program, which, when executed by a machine, performs the method according to any one of claims 1-7.

Images