A multi-threaded parallel computing efficiency optimization system

Abstract

The present application relates to the field of GPU parallel processing technology, and in particular to a multi-threaded parallel computing efficiency optimization system. The system includes: a GPU, a processor, and a memory storing a computer program. When the computer program is executed by the processor, the following steps are implemented: at a preset time point, multiple instruction streams corresponding to a parallel computing module are traversed, the first I instructions in the corresponding instruction stream are obtained, and a plurality of instructions are obtained; according to the instructions that meet the preset conditions, a plurality of requests to be arbitrated are generated; all requests to be arbitrated are traversed, and according to the access objects corresponding to the requests to be arbitrated, the requests to be arbitrated are added to the request queue of the corresponding access object to obtain a plurality of request queues of the corresponding access objects; each request queue is arbitrated respectively, and each arbitration result is sent to its corresponding access object for processing respectively, so that the instructions of different access objects can be executed in parallel, thereby improving the parallel computing efficiency of multi-threads.

Description

Technical Field

[0001] The present invention relates to the field of GPU parallel processing technology, and in particular to a multi-threaded parallel computing efficiency optimization system. Background Art

[0002] Since GPU has more Arithmetic And Logic Unit (ALU) for data processing, it is suitable for parallel processing of dense data and is good at large-scale concurrent computing.

[0003] At present, in the prior art, the instruction stream corresponding to a program is usually handed over to a parallel computing module containing multiple threads for processing. Multiple threads in a parallel computing module execute the same instructions to achieve parallel computing. Each thread has its corresponding computing unit, and the computing unit can be regarded as composed of several ALUs.

[0004] However, the instructions corresponding to the parallel computing module include instructions for accessing data from the memory, and the data access operation takes a long time and does not actually need to occupy the computing unit in the parallel computing module. In the case of data access instructions, subsequent computing instructions need to wait for the data access instructions to be executed before they can continue to be executed, resulting in idle time for the computing unit when it is in use, which in turn leads to low efficiency of parallel computing. Therefore, how to improve the efficiency of multi-threaded parallel computing has become an urgent problem to be solved. Summary of the invention

[0005] In view of the above technical problems, the technical solution adopted by the present invention is:

[0006] A multi-threaded parallel computing efficiency optimization system, the system comprising: a GPU, a processor and a memory storing a computer program, wherein the GPU comprises a parallel computing module set A={a1, a2, ..., a m , …, a M} and external memory D, a m is the mth parallel computing module in A, where m is an integer in the range [1, M], and M is the number of parallel computing modules contained in A. m Including control unit b m , computing unit c m and internal memory m , c m It includes computing subunits corresponding to K types of computing {g m 1, g m 2,…,g m k , …, g m K}, g mk The calculation unit c m The computing subunit corresponding to the k-th computing type, when the computer program is executed by the processor, implements the following steps:

[0007] S101, at a preset time point, b m Get a m The corresponding N(m) instruction streams {f m 1, f m 2, …, f m n(m) , …, f m N(m)}, where f m n(m) It is the n(m)th instruction stream corresponding to the mth parallel computing module, where n(m) is an integer in the range of [1, N(m)], and the instruction stream includes several instructions.

[0008] S102, traverse each instruction stream, obtain the first I instructions in the corresponding instruction stream, and obtain I×N(m) instructions.

[0009] S103, generating a plurality of requests to be arbitrated according to the instructions that meet the preset conditions among the I×N(m) instructions, wherein one instruction that meets the preset conditions corresponds to one request to be arbitrated.

[0010] S104, traverse all requests to be arbitrated, and add the requests to be arbitrated to the request queue of the corresponding access object according to the access object corresponding to the request to be arbitrated, so as to obtain K+2 request queues, wherein the access object includes the external memory D, a m Corresponding internal memory m and a m The corresponding K computing sub-units.

[0011] S105, arbitrating each request queue separately to obtain several arbitration results.

[0012] S106, sending each arbitration result to its corresponding access object for processing.

[0013] S107, deleting the instruction corresponding to each arbitration result from the instruction stream to which it belongs, updating the preset time point, and returning to step S101.

[0014] Compared with the prior art, the present invention has obvious beneficial effects. By means of the above technical solution, a multi-threaded parallel computing efficiency optimization system provided by the present invention can achieve considerable technical advancement and practicality, and has wide industrial utilization value, and has at least the following beneficial effects:

[0015] The present invention provides a multi-threaded parallel computing efficiency optimization system, the system comprising: a GPU, a processor and a memory storing a computer program, wherein the GPU comprises a parallel computing module set A={a1, a2, ..., a m , …, a M} and external memory D, a m is the mth parallel computing module in A, where m is an integer in the range [1, M], and M is the number of parallel computing modules contained in A. m Including control unit b m , computing unit c m and internal memory m , c m It includes computing subunits corresponding to K types of computing {g m 1, g m 2,…,g m k , …, g m K}, g m k The calculation unit c m The computing subunit corresponding to the kth computing type in the computer program is executed by the processor to implement the following steps: S101, at a preset time point, m Get a m The corresponding N(m) instruction streams {f m 1, f m 2, …, f m n(m) , …, f m N(m)}, where f m n(m) is the n(m)th instruction stream corresponding to the mth parallel computing module, n(m) is an integer in the range of [1, N(m)], and the instruction stream includes several instructions, S102, traverse each instruction stream, obtain the first I instructions in the corresponding instruction stream, and obtain I×N(m) instructions, S103, generate several pending arbitration requests according to the instructions that meet the preset conditions in the I×N(m) instructions, and one instruction that meets the preset conditions corresponds to one pending arbitration request, S104, traverse all pending arbitration requests, and according to the access object corresponding to the pending arbitration request, add the pending arbitration request to the request queue corresponding to the access object, and obtain K+2 request queues, wherein the access object includes the external memory D, a m Corresponding internal memory m and a mThe corresponding K computing sub-units, S105, arbitrate each request queue separately to obtain several arbitration results, S106, send each arbitration result to its corresponding access object for processing, S107, delete the instructions corresponding to each arbitration result from the instruction stream to which it belongs, update the preset time point, and return to execute step S101.

[0016] It can be seen that a parallel computing module can correspond to multiple instruction streams, and the computing unit is divided into multiple computing sub-units according to the computing type, the internal memory, external memory and each computing sub-unit are regarded as different access objects, and according to the access object corresponding to the acquired instruction, the request generated according to the instruction is added to the request queue of the corresponding access object, and is sent to the corresponding access object for execution after arbitration. This not only allows the computing instructions and memory access instructions to be executed in parallel, but also allows computing instructions of different computing types to be executed in parallel, which greatly reduces the idle time of the computing unit and thereby improves the parallel computing efficiency of multi-threaded. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0018] Figure 1 A flowchart of a computer program executed by a processor in a multi-threaded parallel computing efficiency optimization system provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0019] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0020] This embodiment provides a multi-threaded parallel computing efficiency optimization system, the system comprising: a GPU, a processor and a memory storing a computer program, wherein the GPU comprises a parallel computing module set A={a1, a2, ..., a m , …, a M} and external memory D, a m is the mth parallel computing module in A, where m is an integer in the range [1, M], and M is the number of parallel computing modules contained in A. m Including control unit bm , computing unit c m and internal memory m , c m It includes computing subunits corresponding to K types of computing {g m 1, g m 2,…,g m k , …, g m K}, g m k The calculation unit c m The calculation subunit corresponding to the kth calculation type in Figure 1 , is a flowchart of a computer program executed by a processor in a multi-threaded parallel computing efficiency optimization system provided by an embodiment of the present invention. When the computer program is executed by the processor, the following steps are implemented:

[0021] S101, at a preset time point, b m Get a m The corresponding N(m) instruction streams {f m 1, f m 2, …, f m n(m) , …, f m N(m)}, where f m n(m) is the n(m)th instruction stream corresponding to the mth parallel computing module, where n(m) is an integer in the range of [1, N(m)], and the instruction stream includes a plurality of instructions;

[0022] S102, traverse each instruction stream, obtain the first I instructions in the corresponding instruction stream, and obtain I×N(m) instructions;

[0023] S103, generating a plurality of requests to be arbitrated according to the instructions that meet the preset conditions among the I×N(m) instructions, where one instruction that meets the preset conditions corresponds to one request to be arbitrated;

[0024] S104, traverse all requests to be arbitrated, and add the requests to be arbitrated to the request queue of the corresponding access object according to the access object corresponding to the request to be arbitrated, so as to obtain K+2 request queues, wherein the access object includes the external memory D, a m Corresponding internal memory m and a m The corresponding K computing subunits;

[0025] S105, arbitrating each request queue separately to obtain several arbitration results;

[0026] S106, sending each arbitration result to its corresponding access object for processing;

[0027] S107, deleting the instruction corresponding to each arbitration result from the instruction stream to which it belongs, updating the preset time point, and returning to step S101.

[0028] The external memory D may refer to a memory such as SDRAM, DDR, etc., and the internal memory e m It can refer to the private local memory in the GPU. One parallel computing module corresponds to a private local memory. One computing unit also corresponds to a register. The access rates of the external memory, internal memory and register increase successively, and the storage capacity decreases successively.

[0029] A computing unit can be considered as consisting of multiple computing logic units (ALUs). Accordingly, a computing subunit of a computing unit includes part of the ALU of the computing unit. An instruction stream corresponds to a program.

[0030] In a specific embodiment, I takes a value of 2.

[0031] It should be noted that the implementer can adjust the value of I according to the actual situation, and only needs to ensure that I is an integer greater than 1. No matter what value of I the implementer adopts, it is within the protection scope of the present invention.

[0032] In a specific implementation, the following steps are further included after step S102:

[0033] S1031, for any instruction stream, if there are J instructions that do not meet the preset conditions among the I instructions corresponding to the instruction stream, continue to obtain J instructions from the instruction stream to replace the J instructions that do not meet the preset conditions until the I instructions corresponding to the instruction stream all meet the preset conditions or all instructions in the instruction stream are obtained.

[0034] Among them, if there are J instructions that do not meet the preset conditions among the I instructions corresponding to the instruction stream, then after eliminating these J instructions, some request queues may not contain requests, thereby reducing parallel efficiency. Therefore, this embodiment provides a method for continuing to obtain instructions.

[0035] In one embodiment, the execution may be triggered to continue obtaining instructions from the instruction stream according to the access object corresponding to the obtained instruction. For example, if there is any access object without a corresponding instruction, the instruction is triggered to continue obtaining instructions from the instruction stream.

[0036] It should be noted that the I instructions corresponding to the above instruction stream all meet the preset conditions or all instructions in the instruction stream are obtained are only two examples of the conditions for stopping iteration. The implementer can adjust the conditions for stopping iteration according to actual conditions, such as setting a threshold for the number of instructions obtained for each instruction stream.

[0037] In a specific implementation manner, the instruction corresponds to an instruction type, and the instruction type includes a dependency type and a non-dependency type;

[0038] The preset condition is that the corresponding instruction belongs to the non-dependency type.

[0039] Among them, the dependency type may mean that the execution of the instruction needs to be based on the execution result of the associated instruction with an associated relationship and the execution result of the associated instruction has not yet been obtained, and the non-dependency type may mean that the instruction has no associated instruction or the execution result of the associated instruction has been obtained.

[0040] Specifically, a page tag may be added to the register to record the state of the data corresponding to the instruction, and the instruction type to which the instruction belongs may be determined according to the state of the data corresponding to the associated instruction of the instruction.

[0041] In a specific implementation, the calculation type includes at least a multiplication type and an addition type.

[0042] The calculation type may include but is not limited to multiplication calculation, addition calculation, multiplication and addition combination calculation and the like at different bit widths, and the calculation type may be determined by the implementer according to actual needs.

[0043] In a specific implementation manner, when arbitrating each request queue separately, a polling arbitration method is adopted.

[0044] The arbitration method may also adopt other arbitration methods such as fixed priority arbitration.

[0045] In a specific embodiment, g m k Contains p m k Threads.

[0046] In this embodiment, the number of threads corresponding to different computing sub-units can be set to the same value, for example, set to 32 threads, and the implementer can allocate a different number of threads to each computing sub-unit according to actual conditions.

[0047] In a specific implementation, updating the preset time point includes:

[0048] The preset time point is added to the preset period, and the preset time point is updated with the addition result.

[0049] The preset period may be a clock period.

[0050] In one embodiment, after obtaining the arbitration results corresponding to each request queue at the current preset time point, the implementer may temporarily not execute the arbitration results, but determine the arbitration results corresponding to each request queue at the next preset time point based on the arbitration results corresponding to each request queue at the current preset time point and each instruction stream. After obtaining the arbitration results corresponding to each request queue at the next preset time point, the implementer executes the arbitration results corresponding to each request queue at the previous preset time point.

[0051] Specifically, when there are multiple access objects, since each access object must have corresponding input data when executing the arbitration result, the amount of input data is uncertain. Therefore, compared with the conventional method, the method of this embodiment will require more registers to store the input data corresponding to the arbitration results of each request queue at the preset time point. However, the number of registers is usually fixed. This embodiment can pre-determine the amount of input data at each preset time point and the next preset time point through two consecutive arbitrations. Therefore, when the amount of input data required at the current preset time point is small, the input data of the next preset time point can be pre-loaded, thereby improving the parallel computing efficiency when the number of registers is fixed.

[0052] In this embodiment, a parallel computing module is enabled to correspond to multiple instruction streams, and the computing unit is divided into multiple computing sub-units according to the computing type. The internal memory, external memory and each computing sub-unit are regarded as different access objects. According to the access object corresponding to the acquired instruction, the request generated according to the instruction is added to the request queue of the corresponding access object, and is sent to the corresponding access object for execution after arbitration. Not only can the computing instructions and memory access instructions be executed in parallel, but also computing instructions of different computing types can be executed in parallel, which greatly reduces the idle time of the computing unit and thus improves the parallel computing efficiency of multi-threaded.

[0053] Although some specific embodiments of the present invention have been described in detail by way of example, it will be appreciated by those skilled in the art that the above examples are for illustration only and are not intended to limit the scope of the present invention. It will also be appreciated by those skilled in the art that various modifications may be made to the embodiments without departing from the scope and spirit of the present invention. The scope of the present invention is defined by the appended claims.

Claims

1. A multi-threaded parallel computing efficiency optimization system, characterized in that: The system comprises: a GPU, a processor and a memory storing a computer program, wherein the GPU comprises a parallel computing module set A={a1, a2, ..., a m , …, a M } and external memory D, a m is the mth parallel computing module in A, where m is an integer in the range [1, M], and M is the number of parallel computing modules contained in A. m Including control unit b m , computing unit c m and internal memory m , c m It includes computing subunits corresponding to K types of computing {g m 1, g m 2,…,g m k , …, g m K }, g m k The calculation unit c m The computing subunit corresponding to the k-th computing type, when the computer program is executed by the processor, implements the following steps: S101, at a preset time point, m Get a m The corresponding N(m) instruction streams {f m 1, f m 2, …, f m n(m) , …, f m N(m) }, where f m n(m) is the n(m)th instruction stream corresponding to the mth parallel computing module, where n(m) is an integer in the range of [1, N(m)], and the instruction stream includes a plurality of instructions; S102, traverse each instruction stream, obtain the first I instructions in the corresponding instruction stream, and obtain I×N(m) instructions; S103, generating a plurality of requests to be arbitrated according to the instructions that meet the preset conditions among the I×N(m) instructions, where one instruction that meets the preset conditions corresponds to one request to be arbitrated; S104, traverse all requests to be arbitrated, and add the requests to be arbitrated to the request queue of the corresponding access object according to the access object corresponding to the request to be arbitrated, so as to obtain K+2 request queues, wherein the access object includes the external memory D, a m Corresponding internal memory m and a m The corresponding K computing subunits; S105, arbitrating each request queue separately to obtain several arbitration results; S106, sending each arbitration result to its corresponding access object for processing; S107, deleting the instructions corresponding to each arbitration result from the instruction stream to which it belongs, updating the preset time point, and returning to execute step S101, wherein updating the preset time point includes: Adding the preset time point to the preset period, and updating the preset time point with the addition result; After step S105, the following steps are also included: After obtaining the arbitration results corresponding to each request queue at the current preset time point, the arbitration results corresponding to each request queue at the current preset time point are temporarily not executed. According to the arbitration results corresponding to each request queue at the current preset time point and each instruction stream, the arbitration results corresponding to each request queue at the next preset time point are determined. After obtaining the arbitration results corresponding to each request queue at the next preset time point, the arbitration results corresponding to each request queue at the current preset time point are executed.

2. The multi-threaded parallel computing efficiency optimization system according to claim 1, characterized in that: The value of I is 2.

3. The multi-threaded parallel computing efficiency optimization system according to claim 1, characterized in that: After step S102, the following steps are also included: S1031, for any instruction stream, if there are J instructions that do not meet the preset conditions among the I instructions corresponding to the instruction stream, continue to obtain J instructions from the instruction stream to replace the J instructions that do not meet the preset conditions until the I instructions corresponding to the instruction stream all meet the preset conditions or all instructions in the instruction stream are obtained.

4. The multi-threaded parallel computing efficiency optimization system according to claim 1, characterized in that: The instruction corresponds to an instruction type, and the instruction type includes a dependency type and a non-dependency type; The preset condition is that the corresponding instruction belongs to the non-dependency type.

5. The multi-threaded parallel computing efficiency optimization system according to claim 1, characterized in that: The calculation type includes at least a multiplication type and an addition type.

6. The multi-threaded parallel computing efficiency optimization system according to claim 1, characterized in that: When arbitrating each request queue separately, a polling arbitration method is adopted.

7. The multi-threaded parallel computing efficiency optimization system according to claim 1, characterized in that: g m k Contains p m k Threads.

Images