A method and computing device for obtaining reduction results

Abstract

This application discloses a method and computing device for obtaining reduction results. In this method, for any given thread block, each thread within the thread block is run, and the elements stored in multiple registers corresponding to each thread are reduced to obtain the reduction result corresponding to that thread. Based on the reduction result corresponding to each thread, the reduction result in the shared memory corresponding to the thread block is updated; the elements belong to the input tensor. The target thread in each thread block is run, and based on the reduction result in the shared memory corresponding to each thread block, the first reduction result in the global memory is updated. The target thread in the target thread block is run, and the first reduction result in the global memory is determined as the reduction result corresponding to the input tensor; the target thread block belongs to multiple thread blocks. Through the above register-shared memory-global memory reduction method, the problem of low efficiency in obtaining reduction results in related technologies is solved.

Description

Technical Field

[0001] This application relates to the field of computing device technology, and in particular to a method for obtaining specification results and a computing device. Background Technology

[0002] Reduction operations combine multiple numbers into a single result, such as finding the maximum value or calculating the sum.

[0003] In related technologies, typically, a thread within a thread block directly writes the reduction result it calculates to global memory. Then, communication within the same thread block and across thread blocks is performed using global memory to determine the reduction result corresponding to the input tensor based on the reduction results calculated by multiple threads.

[0004] However, in the methods of related technologies, each thread in each thread block needs to access global memory. Since the time spent by a thread accessing global memory is relatively long, the methods of related technologies suffer from low efficiency. Summary of the Invention

[0005] This application provides a method and computing device for obtaining reduction results, so as to at least solve the problem of low efficiency in obtaining reduction results in related technologies.

[0006] This application provides a method for obtaining specification results, including:

[0007] For any given thread block, each thread within the thread block is run, and the elements stored in the multiple registers corresponding to the thread are reduced to obtain the reduction result corresponding to the thread. Based on the reduction result corresponding to the thread, the reduction result in the shared memory corresponding to the thread block is updated; the elements belong to the input tensor.

[0008] Run the target thread in each thread block, and update the first reduction result in global memory according to the reduction result in the shared memory corresponding to each thread block;

[0009] Run the target thread in the target thread block and determine the first reduction result in global memory as the reduction result corresponding to the input tensor; the target thread block belongs to multiple thread blocks.

[0010] This application provides a computing device, including:

[0011] The graphics processor is used to run each thread in any given thread block, perform reduction processing on the elements stored in multiple registers corresponding to the thread, obtain the reduction result corresponding to the thread, and update the reduction result in the shared memory corresponding to the thread block based on the reduction result corresponding to the thread; the elements belong to the input tensor.

[0012] The graphics processor is also used to run the target threads in each thread block and update the first reduction result in global memory based on the reduction result in the shared memory corresponding to each thread block.

[0013] The graphics processor is also used to run the target thread in the target thread block, and to determine the first reduction result in global memory as the reduction result corresponding to the input tensor; the target thread block belongs to multiple thread blocks.

[0014] This application provides a computing device, including:

[0015] The processor, and the memory that is in communication with the processor;

[0016] Memory is used to store instructions that the computer executes;

[0017] The processor is used to execute computer execution instructions stored in memory to implement the steps of the method for obtaining the reduction result as described above.

[0018] This application also provides a computer-readable storage medium storing a computer program, wherein when the computer program is executed by a processor, it implements the steps of the aforementioned method for obtaining the reduction result.

[0019] This application also provides a computer program product, including a computer program, which, when executed by a processor, implements the steps of the method for obtaining the reduction result as described above.

[0020] This application, for any given thread block, runs each thread within the thread block, performs reduction processing on the elements (belonging to the input tensors) stored in multiple registers corresponding to the thread, obtains the reduction result corresponding to the thread, and updates the reduction result in the shared memory corresponding to the thread block based on the reduction result corresponding to the thread. The computing device runs the target thread in each thread block, updates the first reduction result in global memory based on the reduction result in the shared memory corresponding to each thread block. The computing device runs the target thread in the target thread block, and determines the first reduction result in global memory as the reduction result corresponding to the input tensor; the target thread block belongs to multiple thread blocks. Because this application's method uses a register-shared memory-global memory reduction approach, it does not need to access global memory during reduction within a thread block, reducing the number of global memory accesses. Since the rate of thread access to global memory is lower than the rate of thread access to shared memory, it improves the efficiency of obtaining the reduction result corresponding to the input tensor. Attached Figure Description

[0021] To more clearly illustrate the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 This is a schematic diagram of the structure of a computing device provided in an embodiment of this application;

[0023] Figure 2 A flowchart illustrating a method for obtaining specification results provided in this application, according to embodiment one;

[0024] Figure 3a A flowchart illustrating a second embodiment of a method for obtaining specification results provided in this application;

[0025] Figure 3b This is a schematic diagram illustrating a scenario for creating a thread block, provided in an embodiment of this application.

[0026] Figure 4 A flowchart illustrating a method for obtaining specification results according to an embodiment of this application;

[0027] Figure 5 A schematic diagram of the computing device provided in this application. Detailed Implementation

[0028] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, other embodiments obtained by those of ordinary skill in the art without creative effort are all within the protection scope of this application.

[0029] It should be noted that, in the description of this application, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. The terms "first," "second," etc., in this application are used to distinguish similar objects and are not used to describe a specific order or sequence.

[0030] Reduction operations combine multiple numbers into a single result, such as finding the maximum value or calculating the sum.

[0031] In related technologies, typically, a thread within a thread block directly writes the reduction result it calculates to global memory. Then, communication within the same thread block and across thread blocks is performed using global memory to determine the reduction result corresponding to the input tensor based on the reduction results calculated by multiple threads.

[0032] However, in the methods of related technologies, each thread in each thread block needs to access global memory. Since the time spent by a thread accessing global memory is relatively long, the methods of related technologies suffer from low efficiency.

[0033] Therefore, in response to the aforementioned technical problems in related technologies, it was discovered during the research process that using a register-shared memory-global memory reduction method can reduce the number of global memory accesses, thereby improving the efficiency of obtaining the reduction result corresponding to the input tensor. Therefore, this application proposes a method and computing device for obtaining the reduction result. Specifically, for any given thread block, the computing device runs each thread within the thread block, performs reduction processing on the elements (belonging to the input tensor) stored in multiple registers corresponding to the thread, obtains the reduction result corresponding to the thread, and updates the reduction result in the shared memory corresponding to the thread block based on the reduction result corresponding to the thread. The computing device runs the target thread in each thread block, updates the first reduction result in the global memory based on the reduction result in the shared memory corresponding to each thread block. The computing device runs the target thread in the target thread block (belonging to multiple thread blocks), and determines the first reduction result in the global memory as the reduction result corresponding to the input tensor.

[0034] To enable those skilled in the art to better understand the present application, the present application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0035] The specific application environment architecture or hardware architecture upon which the execution of the method in this application depends is described herein. (References) Figure 1 , Figure 1 This is a schematic diagram of the structure of a computing device provided in an embodiment of this application.

[0036] like Figure 1 As shown, computing device 10 includes a central processing unit 101 and a graphics processing unit 102. The graphics processing unit 102 includes global memory 1021, total shared memory 1022, and multiple registers. For example, Figure 1 Four registers are shown: register 1, register 2, register 3, and register 4. It should be noted that the total shared memory of 1022 can include the shared memory corresponding to each thread block. It should also be noted that the time a thread spends accessing global memory is greater than the time a thread spends accessing shared memory.

[0037] The central processing unit 101 is electrically connected to the graphics processing unit 102.

[0038] exist Figure 1 In this context, the central processing unit 101 can determine the reduction mode based on the input tensor.

[0039] The graphics processor 102 can run each thread in any thread block, perform reduction processing on the elements stored in multiple registers corresponding to the thread, obtain the reduction result corresponding to the thread, and update the reduction result in the shared memory corresponding to the thread block according to the reduction result corresponding to the thread; the elements belong to the input tensor.

[0040] The graphics processor 102 can run the target thread in each thread block and update the first reduction result in global memory according to the reduction result in the shared memory corresponding to each thread block.

[0041] The graphics processor 102 can run the target thread in the target thread block and determine the first reduction result in global memory as the reduction result corresponding to the input tensor; the target thread block belongs to multiple thread blocks.

[0042] It should be noted that, Figure 1 This is a schematic diagram of a computing device provided in an embodiment of this application. This embodiment of the application does not... Figure 1 The actual form of the various components included is not limited, nor is the actual form of the components included defined. Figure 1 The interaction methods between components are limited, and can be set according to actual needs in the application of the solution.

[0043] The technical solution of this application will now be described in detail through specific embodiments. It should be noted that the following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments.

[0044] Figure 2 This is a flowchart illustrating a method for obtaining specification results according to an embodiment of this application. See also... Figure 2 The method specifically includes the following steps:

[0045] S201: For any thread block, run each thread in the thread block, perform reduction processing on the elements stored in the multiple registers corresponding to the thread, obtain the reduction result corresponding to the thread, and update the reduction result in the shared memory corresponding to the thread block according to the reduction result corresponding to the thread.

[0046] In this embodiment, the graphics processor in the computing device may include at least one thread block. Each thread block may include multiple threads. It should be noted that, in one implementation, when the reduction mode is a level 3 reduction mode, the graphics processor in the computing device may include multiple thread blocks; when the reduction mode is a register mode, the graphics processor in the computing device may include one thread block; when the reduction mode is a shared memory mode, the graphics processor in the computing device may include one thread block.

[0047] Each thread can correspond to multiple registers. Each register can store elements. It should be noted that elements are input tensors. In one implementation, a register can store one element.

[0048] The graphics processor in a computing device, for any given thread block, runs each thread within that thread block, performs reduction processing on the elements stored in multiple registers corresponding to that thread, and obtains the reduction result corresponding to that thread.

[0049] The graphics processing unit (GPU) in a computing device can run each thread within the thread block and update the reduction results in the shared memory corresponding to the thread block based on the reduction results for each thread. In one implementation, the GPU can run each thread within the thread block and call BlockReduce (a code module) to update the reduction results in the shared memory corresponding to the thread block based on the reduction results for each thread. It should be noted that BlockReduce can be used to perform reduction processing on the reduction results corresponding to multiple threads within a thread block.

[0050] In one implementation:

[0051] The central processing unit in a computing device can determine the number of elements in an input tensor.

[0052] The central processing unit (CPU) in a computing device can determine the reduction mode corresponding to the input tensor based on the number of elements. The reduction modes include register mode, shared memory mode, or three-level reduction mode.

[0053] The graphics processor in a computing device can create at least one thread block according to a specification pattern.

[0054] It should be noted that, in one implementation, when the reduction mode is level 3, the number of thread blocks can be greater than 1; in another implementation, when the reduction mode is register mode, the number of thread blocks can be 1; and in yet another implementation, when the reduction mode is shared memory mode, the number of thread blocks can be 1.

[0055] It should also be noted that, in one implementation, the graphics processor in the computing device can determine the kernel corresponding to the reduction mode based on the reduction mode, and start the kernel corresponding to the reduction mode to create at least one thread block. In another implementation, the graphics processor in the computing device can determine the kernel corresponding to the level 3 reduction mode based on the level 3 reduction mode, and start the kernel corresponding to the level 3 reduction mode to create multiple thread blocks. In another implementation, the graphics processor in the computing device can determine the kernel corresponding to the register mode based on the register mode reduction mode, and start the kernel corresponding to the register mode to create one thread block. In yet another implementation, the graphics processor in the computing device can determine the kernel corresponding to the shared memory mode based on the shared memory mode reduction mode, and start the kernel corresponding to the shared memory mode to create one thread block.

[0056] The graphics processor in the computing device can, in the case of the three-level reduction mode, run each thread in the thread block for any thread block, perform reduction processing on the elements stored in multiple registers corresponding to the thread, obtain the reduction result corresponding to the thread, and update the reduction result in the shared memory corresponding to the thread block according to the reduction result corresponding to the thread.

[0057] S202: Run the target thread in each thread block, and update the first reduction result in global memory according to the reduction result in the shared memory corresponding to each thread block.

[0058] In this embodiment, the graphics processor in the computing device can run the target thread in each thread block and update the first reduction result in the global memory according to the reduction result in the shared memory corresponding to each thread block. In one implementation, for any thread block, the graphics processor in the computing device can run the target thread in that thread block and update the first reduction result in the global memory using an atomic write method according to the reduction result in the shared memory corresponding to that thread block.

[0059] By using atomic writes, when multiple threads update the first reduction result in global memory simultaneously, it can be guaranteed that one thread will execute first, and then another thread will execute, in order to avoid data corruption.

[0060] It should be noted that, for any given thread block, the target thread in that thread block updates the first reduction result in the global memory based on the reduction result in the shared memory corresponding to the thread block after each thread in that thread block updates the reduction result in the shared memory corresponding to the thread block.

[0061] It should also be noted that, for any given thread block, the target thread in that thread block updates the first reduction result in the shared memory corresponding to the thread block after each thread in that thread block updates the reduction result in the shared memory corresponding to the thread block based on the reduction result of each batch of the thread.

[0062] In one implementation, the target thread in each thread block is the first thread in that thread block.

[0063] S203: Run the target thread in the target thread block and determine the first reduction result in global memory as the reduction result corresponding to the input tensor.

[0064] In this embodiment, the multiple thread blocks may include a target thread block. In one implementation, the target thread block may be the first thread block among the multiple thread blocks.

[0065] It should be noted that the target thread in the target thread block is the target thread in each thread block. After updating the first reduction result in the global memory based on the reduction result in the shared memory corresponding to the thread block, the first reduction result in the global memory is determined as the reduction result corresponding to the input tensor.

[0066] In one implementation, the target thread block can be the thread block with the smallest thread block number. In another implementation, the target thread block can be the thread block with thread block number 0.

[0067] The graphics processor in the computing device can run the target thread in the target thread block and determine the first reduction result in global memory as the reduction result corresponding to the input tensor.

[0068] The beneficial effects of this embodiment are as follows: In this embodiment, for any given thread block, the computing device runs each thread within the thread block, performs reduction processing on the elements stored in multiple registers corresponding to the thread, obtains the reduction result corresponding to the thread, and updates the reduction result in the shared memory corresponding to the thread block based on the reduction result corresponding to the thread; the elements belong to the input tensor. The computing device can run the target thread in each thread block, update the first reduction result in the global memory based on the reduction result in the shared memory corresponding to each thread block; run the target thread in the target thread block, and determine the first reduction result in the global memory as the reduction result corresponding to the input tensor; the target thread block belongs to multiple thread blocks. Through the above method, registers are first used for intra-thread reduction, then shared memory is used for intra-thread block reduction, and then global memory is used for inter-thread block reduction, reducing the number of global memory accesses and avoiding direct global synchronization of reduction results. Since the speed of thread access to shared memory is greater than the speed of thread access to global memory, the above method improves the efficiency of obtaining reduction results.

[0069] Figure 3a This is a flowchart illustrating a second embodiment of a method for obtaining specification results provided in this application. See also... Figure 3a The method specifically includes the following steps:

[0070] S301: Determine the number of elements in the input tensor.

[0071] In this embodiment, the input tensor may include multiple elements.

[0072] The central processing unit in a computing device can determine the number of elements in an input tensor.

[0073] S302: Determine the reduction mode corresponding to the input tensor based on the number of elements.

[0074] In this embodiment, the central processing unit in the computing device can determine the reduction mode corresponding to the input tensor based on the number of elements.

[0075] The following section explains the process by which the central processing unit in a computing device determines the reduction mode corresponding to the input tensor based on the number of elements.

[0076] In one implementation:

[0077] The central processing unit in a computing device can obtain the range of element counts corresponding to each reduction mode (also known as a reduction path). For example, the range of element counts corresponding to the register mode is (0, 512]; the range of element counts corresponding to the shared memory mode is (512, 8064]; and the range of element counts corresponding to the three-level reduction mode is (8064, +∞).

[0078] The central processing unit (CPU) in a computing device can determine the reduction mode of the input tensor based on the number of elements (data_size) and the range of element counts corresponding to each reduction mode. For example, the CPU in the computing device can determine that the input tensor corresponds to a level 3 reduction mode based on the number of elements being 9000.

[0079] Additionally, it should be noted that in one implementation:

[0080] The central processing unit in a computing device can also obtain the model of the graphics processor when the protocol mode is shared memory mode or three-level protocol mode.

[0081] The central processing unit (CPU) in a computing device can determine the size of shared memory based on the graphics processing unit (GPU) model. In one implementation, shared memory is part of the total shared memory. It should be noted that the shared memory corresponding to each thread block is accessible only to the threads within that thread block; threads in other thread blocks cannot access this shared memory.

[0082] The central processing unit in a computing device can send the size of shared memory to the graphics processing unit.

[0083] The graphics processor in a computing device can determine the shared memory corresponding to each thread block based on the size of the shared memory. For example, the size of the shared memory could be 164KB.

[0084] S303: Create at least one thread block according to the specification pattern.

[0085] In this embodiment, the computing device can create at least one thread block according to the specification pattern.

[0086] For example, Figure 3b This is a schematic diagram illustrating a scenario for creating a thread block, as provided in an embodiment of this application. Figure 3b As shown, in register mode, the computing device can create a thread block, which can be a 32x64 thread block; in shared memory mode, the computing device can create a thread block, which can be a 128x256 thread block; and in shared memory mode, the computing device can create 4xSM (number of streaming multiprocessors) thread blocks, which include 1024 threads.

[0087] In one implementation:

[0088] The central processing unit in the computing device can determine the first quantity according to the protocol mode.

[0089] The first quantity is the number of thread blocks.

[0090] The following describes the process by which the central processing unit in a computing device determines the first quantity according to the protocol mode.

[0091] In one implementation, the central processing unit (CPU) in the computing device, when the reduction mode is level 3, determines the first quantity by multiplying the number of streaming multiprocessors in the computing device by a preset multiple. In one implementation, the preset multiple can be 4. That is, the first quantity can be 4 x SM (the number of streaming multiprocessors).

[0092] In one implementation, the central processing unit in the computing device can determine the sequence length value (num_tokens) corresponding to the input tensor as the first quantity when the reduction mode is three-level reduction mode.

[0093] In one implementation, the central processing unit (CPU) in the computing device can determine a preset first quantity as the first quantity when the reduction mode is not a level 3 reduction mode. In another implementation, the preset first quantity can be 1. In yet another implementation, the CPU in the computing device can determine the sequence length value (num_tokens) corresponding to the input tensor as the initial first quantity when the reduction mode is register mode. The CPU in the computing device can further reduce the initial first quantity based on the register mode reduction mode to obtain a reduced initial first quantity, and then determine the reduced initial first quantity as the first quantity.

[0094] The central processing unit in the computing device can determine the second quantity according to the specification pattern. The second quantity is the number of threads included in the thread block.

[0095] The following describes the process by which the central processing unit in a computing device determines the second quantity according to the protocol mode.

[0096] In one implementation:

[0097] The central processing unit in a computing device can obtain the feature dimension value (hidden_size) corresponding to the input tensor.

[0098] The central processing unit in the computing device can determine whether the feature dimension value is greater than a preset threshold. In one implementation, the preset threshold can be 1024.

[0099] The central processing unit in the computing device can determine the preset threshold as the second quantity if the feature dimension value is greater than the preset threshold.

[0100] or,

[0101] The central processing unit in the computing device can determine the preset threshold as the second quantity if the feature dimension value is less than or equal to the preset threshold.

[0102] In one implementation:

[0103] The central processing unit in the computing device can determine the preset threshold as the second quantity when the specification mode is three-level specification mode.

[0104] The central processing unit (CPU) in the computing device can obtain the feature dimension values ​​corresponding to the input tensor even when the reduction mode is not a level 3 reduction mode. The CPU in the computing device can determine whether the feature dimension value is greater than a preset threshold. If the feature dimension value is greater than the preset threshold, the CPU in the computing device can determine the preset threshold as a second quantity. Alternatively, if the feature dimension value is less than or equal to the preset threshold, the CPU in the computing device can determine the preset threshold as a second quantity.

[0105] Furthermore, the central processing unit in the computing device, after determining the first quantity and the second quantity, can send a processing message to the graphics processing unit; wherein the processing message may include the first quantity and the second quantity. In one implementation, the processing message may also include a specification mode.

[0106] The graphics processor in the computing device can, after obtaining a first quantity and a second quantity, create at least one thread block based on the first quantity and the second quantity. The number of threads included in each thread block is consistent with the second quantity.

[0107] S304: When the reduction mode is three-level reduction mode, for any thread block, run each thread in the thread block, perform reduction processing on the elements stored in the multiple registers corresponding to the thread, obtain the reduction result corresponding to the thread, and update the reduction result in the shared memory corresponding to the thread block according to the reduction result corresponding to the thread.

[0108] In this embodiment, the graphics processor in the computing device can obtain the reduction mode corresponding to the input tensor sent by the central processing unit in the computing device.

[0109] The graphics processor in the computing device can, in the case of the three-level reduction mode, run each thread in the thread block for any thread block, perform reduction processing on the elements stored in multiple registers corresponding to the thread, obtain the reduction result corresponding to the thread, and update the reduction result in the shared memory corresponding to the thread block according to the reduction result corresponding to the thread.

[0110] Additionally, it should be noted that in one implementation:

[0111] The graphics processing unit (GPU) in a computing device can, in register mode, run each thread within a thread block to perform reduction processing on the elements stored in multiple registers corresponding to each thread, obtain the reduction result for that thread, and update the second reduction result stored in global memory based on the thread's reduction result. The GPU in the computing device can also run the target thread within the thread block to determine the second reduction result in global memory as the reduction result corresponding to the input tensor.

[0112] Understandably, when the reduction mode is register mode, each thread in the thread block is a thread created by the kernel corresponding to register mode.

[0113] Additionally, it should be noted that in one implementation:

[0114] The graphics processing unit (GPU) in a computing device can, in shared memory mode, run each thread within a thread block to perform reduction processing on the elements stored in multiple registers corresponding to each thread, obtain the reduction result for that thread, and update the reduction result in the shared memory corresponding to the thread block based on the reduction result for each thread. The GPU can also run the target thread within the thread block, determine the reduction result in the shared memory corresponding to the thread block as the reduction result for the input tensor, and store the reduction result for the input tensor in global memory.

[0115] Understandably, when the specification mode is shared memory mode, each thread in the thread block is a thread created by the kernel corresponding to the shared memory mode.

[0116] S305: Run the target thread in each thread block and update the first reduction result in global memory based on the reduction result in the shared memory corresponding to each thread block.

[0117] In this embodiment, the graphics processor in the computing device can run the target thread in each thread block and update the first reduction result in the global memory according to the reduction result in the shared memory corresponding to each thread block.

[0118] The specific implementation process is the same as that of S202, and will not be described in detail here.

[0119] S306: Run the target thread in the target thread block and determine the first reduction result in global memory as the reduction result corresponding to the input tensor.

[0120] In this embodiment, the target thread block belongs to multiple thread blocks.

[0121] The graphics processor in the computing device can run the target thread in the target thread block and determine the first reduction result in global memory as the reduction result corresponding to the input tensor.

[0122] The specific implementation process is the same as that of S203, and will not be described in detail here.

[0123] The beneficial effects of this embodiment are as follows: In this embodiment, the computing device can determine the reduction mode (including three-level reduction mode, register mode, or shared memory mode) of the input tensor based on the number of elements in the input tensor. The computing device can create at least one thread block according to the reduction mode. When the reduction mode is three-level reduction mode, the computing device can run each thread in any thread block, perform reduction processing on the elements stored in multiple registers corresponding to the thread, obtain the reduction result corresponding to the thread, and update the reduction result in the shared memory corresponding to the thread block according to the reduction result corresponding to the thread. The computing device can run the target thread in each thread block, update the first reduction result in the global memory according to the reduction result in the shared memory corresponding to each thread block; run the target thread in the target thread block, and determine the first reduction result in the global memory as the reduction result corresponding to the input tensor; the target thread block belongs to multiple thread blocks. The above approach achieves two main benefits. First, it allows for accurate determination of the reduction pattern based on the number of elements in the input tensor. This reduction pattern then enables the determination of the number of thread blocks to be created and the number of threads included in each thread block, thus improving resource utilization. Second, based on the three-level reduction pattern, a register-shared memory-global memory approach is used for step-by-step reduction, reducing the number of global memory accesses and avoiding direct global synchronization of reduction results, thereby improving the efficiency of obtaining reduction results.

[0124] The following will use method embodiment three to explain the process in S201 of the embodiment: "For any thread block, run each thread in the thread block, perform reduction processing on the elements stored in the multiple registers corresponding to the thread, obtain the reduction result corresponding to the thread, and update the reduction result in the shared memory corresponding to the thread block according to the reduction result corresponding to the thread."

[0125] Figure 4 This is a flowchart illustrating a method for obtaining specification results according to an embodiment of this application. See also... Figure 4 The method specifically includes the following steps:

[0126] S401: For the i-th thread in any thread block, run the i-th thread and, based on the vectorization width, obtain multiple j-th batches of elements corresponding to the i-th thread from the input tensor stored in global memory.

[0127] In this embodiment, the graphics processor in the computing device runs the i-th thread in any thread block, and retrieves multiple j-th batches of elements corresponding to the i-th thread from the input tensor stored in global memory, based on the vectorization width (VEC_SIZ). For example, the vectorization width (VEC_SIZ) can be 1. Another example is that the vectorization width (VEC_SIZ) can be 4. Yet another example is that the vectorization width (VEC_SIZ) can be 8.

[0128] It should be noted that i takes values ​​of 1, 2, ..., up to N; N is the number of threads included in the thread block;

[0129] j takes values ​​of 1, 2, ..., up to M; M is the total number of elements in each batch corresponding to each thread.

[0130] Below, we will first explain the process by which the graphics processor in a computing device obtains the vectorized width.

[0131] In one implementation:

[0132] The central processing unit in a computing device can obtain the preset vectorization width corresponding to each reduction mode. For example, the preset vectorization width (VEC_SIZ) corresponding to register mode can be 8, the preset vectorization width (VEC_SIZ) corresponding to shared memory mode can be 4, and the preset vectorization width (VEC_SIZ) corresponding to level 3 reduction mode can be 1.

[0133] The central processing unit in the computing device can determine the vectorization width based on the reduction mode corresponding to the input tensor and the preset vectorization width corresponding to each reduction mode after determining the reduction mode corresponding to the input tensor.

[0134] The central processing unit in a computing device can send vectorized widths to the graphics processing unit in the computing device.

[0135] In one implementation:

[0136] The central processing unit in a computing device can obtain the number of elements corresponding to the input tensor.

[0137] The central processing unit in a computing device can obtain the storage address corresponding to the input tensor.

[0138] The central processing unit in a computing device can obtain multiple candidate vectorization widths. For example, the multiple candidate vectorization widths can include 1, 4, and 8.

[0139] For any candidate vectorization width, the central processing unit in the computing device can determine whether the number of elements and the storage address are integer multiples of the candidate vectorization width.

[0140] The central processing unit in a computing device can determine the vectorization width as the selected vectorization width when the number of elements and the storage address are integer multiples of the selected vectorization width.

[0141] The central processing unit in a computing device can send vectorized widths to the graphics processing unit in the computing device.

[0142] The following describes the process by which the i-th thread in the graphics processor of the computing device obtains multiple j-th batches of elements corresponding to the i-th thread from the input tensor stored in global memory, based on the vectorization width.

[0143] In one implementation:

[0144] The graphics processor in the computing device, running the i-th thread, can determine the storage addresses of multiple j-th batches of elements based on the thread number of the i-th thread, the thread block number of the thread block to which the i-th thread belongs, the storage address of the input tensor, and the vectorization width.

[0145] The graphics processor in the computing device, running the i-th thread, can retrieve multiple elements of the j-th batch corresponding to the i-th thread from the input tensor stored in global memory, based on the storage addresses of multiple elements of the j-th batch.

[0146] S402: Run the i-th thread and store the multiple j-th batches of elements corresponding to the i-th thread into multiple registers corresponding to the i-th thread.

[0147] In this embodiment, the graphics processor in the computing device, when running the i-th thread, can store the multiple j-th batches of elements corresponding to the i-th thread into multiple registers corresponding to the i-th thread after obtaining the multiple j-th batches of elements corresponding to the i-th thread.

[0148] S403: Run the i-th thread, perform reduction processing on the j-th batch of elements stored in the multiple registers corresponding to the i-th thread, and obtain the reduction result of the j-th batch corresponding to the i-th thread.

[0149] In this embodiment, the graphics processor in the computing device can run the i-th thread to perform reduction processing on the j-th batch of elements stored in multiple registers corresponding to the i-th thread, and obtain the reduction result of the j-th batch corresponding to the i-th thread.

[0150] In one implementation, the graphics processor in the computing device can run the i-th thread, compare the elements of the j-th batch stored in multiple registers corresponding to the i-th thread, determine the maximum element, and set the maximum element as the reduction result of the j-th batch corresponding to the i-th thread.

[0151] In one implementation, the graphics processor in the computing device can run the i-th thread and perform parallel reduction processing on the j-th batch of elements stored in multiple registers corresponding to the i-th thread based on the loop unrolling function, to obtain the reduction result of the j-th batch corresponding to the i-th thread.

[0152] Additionally, it should be noted that the graphics processor in a computing device runs its threads in parallel. For the i-th thread, the graphics processor in the computing device can run the i-th thread and perform reduction processing on the j-th batch of elements stored in the multiple registers corresponding to the i-th thread to obtain the reduction result of the j-th batch corresponding to the i-th thread.

[0153] S404: Run the i-th thread and update the reduction result in the shared memory corresponding to the thread block based on the reduction result of the j-th batch corresponding to the i-th thread.

[0154] In this embodiment, the graphics processor in the computing device can run the i-th thread and update the reduction result in the shared memory corresponding to the thread block according to the reduction result of the j-th batch corresponding to the i-th thread.

[0155] It should be noted that, in one implementation, the graphics processor in the computing device can run the i-th thread and call BlockReduce so that BlockReduce updates the reduction result in the shared memory corresponding to the thread block according to the reduction result of the j-th batch corresponding to the i-th thread.

[0156] In one implementation:

[0157] The graphics processor in the computing device can run the i-th thread, perform reduction processing on the reduction results of the j-th batch corresponding to the i-th thread and the reduction results in shared memory, obtain a new reduction result, and determine the new reduction result as the reduction result in shared memory corresponding to the thread block.

[0158] For example, when the reduction process is to find the maximum value, the graphics processor in the computing device can run the i-th thread, determine the larger of the reduction result of the j-th batch corresponding to the i-th thread and the reduction result in the shared memory as the new reduction result, and determine the new reduction result as the reduction result in the shared memory corresponding to the thread block.

[0159] The beneficial effects of this embodiment are as follows: In this embodiment, for the i-th thread in any thread block, the computing device (graphics processor in the computing device) can run the i-th thread and, based on the vectorization width, obtain multiple batches of elements corresponding to the i-th thread from the input tensor stored in global memory. The computing device (graphics processor in the computing device) can run the i-th thread and store the multiple batches of elements corresponding to the i-th thread into multiple registers corresponding to the i-th thread. The computing device (graphics processor in the computing device) can run the i-th thread and perform reduction processing on the batches of elements stored in the multiple registers corresponding to the i-th thread to obtain the reduction result of the batches corresponding to the i-th thread. The computing device (graphics processor in the computing device) can run the i-th thread and update the reduction result in the shared memory corresponding to the thread block based on the reduction result of the batches corresponding to the i-th thread. Here, i sequentially takes values ​​of 1, 2, ..., up to N; N is the number of threads included in the thread block. j sequentially takes values ​​of 1, 2, ..., up to M; M is the total number of element batches corresponding to each thread. The above method allows for the orderly acquisition of elements based on the vector width, followed by reduction of the elements, thereby improving the efficiency and accuracy of obtaining the reduction results for the corresponding thread.

[0160] The following are embodiments of the apparatus described in this application, which can be used to execute the embodiments of the method described in this application. For details not disclosed in the apparatus embodiments of this application, please refer to the embodiments of the method described in this application.

[0161] This application provides a computing device, including a graphics processor and a central processing unit.

[0162] The graphics processor is used to run each thread in any given thread block, perform reduction processing on the elements stored in multiple registers corresponding to the thread, obtain the reduction result corresponding to the thread, and update the reduction result in the shared memory corresponding to the thread block based on the reduction result corresponding to the thread; the elements belong to the input tensor.

[0163] The graphics processor is also used to run the target threads in each thread block and update the first reduction result in global memory based on the reduction result in the shared memory corresponding to each thread block.

[0164] The graphics processor is also used to run the target thread in the target thread block and determine the first reduction result in global memory as the reduction result corresponding to the input tensor; the target thread block belongs to at least one thread block.

[0165] The computing device provided in this application embodiment can execute the technical solution shown in the above method embodiment. Its implementation principle and beneficial effects are similar, and will not be repeated here.

[0166] In one implementation,

[0167] The central processing unit is used to determine the number of elements in the input tensor.

[0168] The central processing unit is also used to determine the reduction mode corresponding to the input tensor based on the number of elements; the reduction mode includes three-level reduction mode, register mode or shared memory mode;

[0169] The graphics processor is also used to create at least one thread block according to the specification pattern;

[0170] The graphics processor is also used to, in the case of a three-level reduction mode, for any thread block, run each thread in the thread block, perform reduction processing on the elements stored in multiple registers corresponding to the thread, obtain the reduction result corresponding to the thread, and update the reduction result in the shared memory corresponding to the thread block according to the reduction result corresponding to the thread.

[0171] The computing device provided in this application embodiment can execute the technical solution shown in the above method embodiment. Its implementation principle and beneficial effects are similar, and will not be repeated here.

[0172] In one implementation, the central processing unit is specifically used for:

[0173] Get the range of the number of elements corresponding to each specification pattern;

[0174] The reduction mode corresponding to the input tensor is determined based on the number of elements and the range of element numbers corresponding to each reduction mode.

[0175] The computing device provided in this application embodiment can execute the technical solution shown in the above method embodiment. Its implementation principle and beneficial effects are similar, and will not be repeated here.

[0176] In one implementation,

[0177] The central processing unit is used to determine a first quantity based on the specification pattern; the first quantity is the number of thread blocks.

[0178] The central processing unit is used to determine a second quantity based on the specification pattern; the second quantity is the number of threads included in the thread block.

[0179] A graphics processor for creating at least one thread block based on a first quantity and a second quantity.

[0180] The computing device provided in this application embodiment can execute the technical solution shown in the above method embodiment. Its implementation principle and beneficial effects are similar, and will not be repeated here.

[0181] In one implementation, the central processing unit is used for:

[0182] In the case of a three-level reduction mode, the first quantity is determined by multiplying the number of streaming multiprocessors in the computing device by a preset multiple; or,

[0183] When the reduction mode is a level 3 reduction mode, the sequence length value corresponding to the input tensor is determined as the first quantity.

[0184] The computing device provided in this application embodiment can execute the technical solution shown in the above method embodiment. Its implementation principle and beneficial effects are similar, and will not be repeated here.

[0185] In one implementation, the central processing unit is used for:

[0186] Obtain the feature dimension values ​​corresponding to the input tensor;

[0187] Determine whether the feature dimension value is greater than a preset threshold;

[0188] If the feature dimension value is greater than a preset threshold, the preset threshold is set as the second quantity; or,

[0189] If the feature dimension value is less than or equal to a preset threshold, the preset threshold is determined as the second quantity.

[0190] The computing device provided in this application embodiment can execute the technical solution shown in the above method embodiment. Its implementation principle and beneficial effects are similar, and will not be repeated here.

[0191] In one implementation, the central processing unit is used for:

[0192] In the case of a three-level specification mode, the preset threshold is set as the second quantity; or,

[0193] When the reduction mode is not a level 3 reduction mode, obtain the feature dimension value corresponding to the input tensor.

[0194] The computing device provided in this application embodiment can execute the technical solution shown in the above method embodiment. Its implementation principle and beneficial effects are similar, and will not be repeated here.

[0195] In one implementation, the graphics processor is used for:

[0196] For the i-th thread in any thread block, run the i-th thread and, based on the vectorization width, retrieve multiple j-th batches of elements corresponding to the i-th thread from the input tensor stored in global memory;

[0197] Run the i-th thread and store the multiple j-th batches of elements corresponding to the i-th thread into multiple registers corresponding to the i-th thread;

[0198] Run the i-th thread and reduce the j-th batch of elements stored in the multiple registers corresponding to the i-th thread to obtain the reduction result of the j-th batch corresponding to the i-th thread.

[0199] Run the i-th thread, and update the reduction result in the shared memory corresponding to the thread block according to the reduction result of the j-th batch corresponding to the i-th thread;

[0200] Where i takes values ​​of 1, 2, ..., up to N; N is the number of threads included in the thread block;

[0201] j takes values ​​of 1, 2, ..., up to M; M is the total number of elements in each batch corresponding to each thread.

[0202] The computing device provided in this application embodiment can execute the technical solution shown in the above method embodiment. Its implementation principle and beneficial effects are similar, and will not be repeated here.

[0203] In one implementation, the central processing unit is also used for:

[0204] Obtain the preset vectorized width corresponding to each specification mode;

[0205] The vectorization width is determined based on the reduction mode corresponding to the input tensor and the preset vectorization width corresponding to each reduction mode.

[0206] The computing device provided in this application embodiment can execute the technical solution shown in the above method embodiment. Its implementation principle and beneficial effects are similar, and will not be repeated here.

[0207] In one implementation, the central processing unit is also used for:

[0208] Get the number of elements corresponding to the input tensor;

[0209] Obtain the storage address corresponding to the input tensor;

[0210] Get multiple candidate vectorized widths;

[0211] For any candidate vectorization width, determine the number of elements and storage address, and whether they are integer multiples of the candidate vectorization width;

[0212] If the number of elements and the storage address are integer multiples of the candidate vectorization width, the candidate vectorization width is determined as the vectorization width.

[0213] The computing device provided in this application embodiment can execute the technical solution shown in the above method embodiment. Its implementation principle and beneficial effects are similar, and will not be repeated here.

[0214] In one implementation, the graphics processor is also used for:

[0215] When the reduction mode is register mode, each thread in the running thread block performs reduction processing on the elements stored in the multiple registers corresponding to the thread, obtains the reduction result corresponding to the thread, and updates the second reduction result stored in global memory according to the reduction result corresponding to the thread.

[0216] The target thread in the running thread block determines the second reduction result in global memory as the reduction result corresponding to the input tensor.

[0217] The computing device provided in this application embodiment can execute the technical solution shown in the above method embodiment. Its implementation principle and beneficial effects are similar, and will not be repeated here.

[0218] In one implementation, the graphics processor is also used for:

[0219] When the reduction mode is shared memory mode, each thread in the running thread block performs reduction processing on the elements stored in the multiple registers corresponding to the thread, obtains the reduction result corresponding to the thread, and updates the reduction result in the shared memory corresponding to the thread block according to the reduction result corresponding to the thread.

[0220] Run the target thread in the thread block, determine the reduction result in the shared memory corresponding to the thread block as the reduction result corresponding to the input tensor, and store the reduction result corresponding to the input tensor in global memory.

[0221] The computing device provided in this application embodiment can execute the technical solution shown in the above method embodiment. Its implementation principle and beneficial effects are similar, and will not be repeated here.

[0222] In one implementation,

[0223] The central processing unit is also used to obtain the model of the graphics processor when the protocol mode is shared memory mode or three-level protocol mode;

[0224] The central processing unit is also used to determine the size of shared memory based on the model of the graphics processor;

[0225] The graphics processor is also used to determine the shared memory corresponding to each thread block based on the size of the shared memory.

[0226] The computing device provided in this application embodiment can execute the technical solution shown in the above method embodiment. Its implementation principle and beneficial effects are similar, and will not be repeated here.

[0227] In one implementation, the target thread block is the first thread block among multiple thread blocks; the target thread in each thread block is the first thread of that thread block.

[0228] The computing device provided in this application embodiment can execute the technical solution shown in the above method embodiment. Its implementation principle and beneficial effects are similar, and will not be repeated here.

[0229] Figure 5 A schematic diagram of the computing device provided in this application. Figure 5 As shown, the computing device 50 provided in this embodiment includes a processor 51 and a memory 52 that is communicatively connected to the processor 51.

[0230] Memory 52 is used to store computer-executed instructions.

[0231] The processor 51 is used to execute computer execution instructions stored in the memory 52 to implement the scheme of the aforementioned method embodiment.

[0232] In one implementation, the memory 52 can be either standalone or integrated with the processor 51. Optionally, when the memory 52 is a device independent of the processor 51, the computing device 50 may further include a bus 53 for connecting the aforementioned devices.

[0233] The specific implementation process of processor 51 can be found in the above method embodiments, and its implementation principle and technical effect are similar. It will not be repeated here.

[0234] Embodiments of this application also provide a computer-readable storage medium storing a computer program, wherein the computer program is configured to execute the steps in any of the above method embodiments when it is run.

[0235] In one exemplary embodiment, the aforementioned computer-readable storage medium may include, but is not limited to, various media capable of storing computer programs, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard disk, magnetic disk, or optical disk.

[0236] Embodiments of this application also provide a computer program product, which includes a computer program that, when executed by a processor, implements the steps in any of the above method embodiments.

[0237] Embodiments of this application also provide another computer program product, including a non-volatile computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps in any of the above method embodiments.

[0238] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0239] The above describes a method and computing device for obtaining specification results provided in this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the embodiments above are only for the purpose of helping to understand the method and core ideas of this application. It should be noted that those skilled in the art can make several improvements and modifications to this application without departing from the principles of this application, and these improvements and modifications also fall within the protection scope of the claims of this application.

Claims

1. A method for obtaining reduction results, characterized in that, include: For any given thread block, each thread within the thread block is run, and the elements stored in the multiple registers corresponding to the thread are reduced to obtain the reduction result corresponding to the thread. Based on the reduction result corresponding to the thread, the reduction result in the shared memory corresponding to the thread block is updated; the element belongs to the input tensor. Run the target thread in each thread block, and update the first reduction result in global memory according to the reduction result in the shared memory corresponding to each thread block; The target thread in each thread block is the first thread of that thread block; Run the target thread in the target thread block and determine the first reduction result in the global memory as the reduction result corresponding to the input tensor; The target thread block belongs to multiple thread blocks; The target thread block is the first thread block among the plurality of thread blocks.

2. The method according to claim 1, characterized in that, For any given thread block, the process involves running each thread within that thread block, performing reduction processing on the elements stored in multiple registers corresponding to that thread to obtain the reduction result for that thread, and updating the reduction result in the shared memory corresponding to the thread block based on the reduction result for that thread. This includes: Determine the number of elements in the input tensor; The reduction mode corresponding to the input tensor is determined based on the number of elements; the reduction mode includes a three-level reduction mode, a register mode, or a shared memory mode. Create at least one thread block according to the specification pattern; When the reduction mode is the three-level reduction mode, for any thread block, each thread in the thread block is run, and the elements stored in the multiple registers corresponding to the thread are reduced to obtain the reduction result corresponding to the thread. Based on the reduction result corresponding to the thread, the reduction result in the shared memory corresponding to the thread block is updated.

3. The method according to claim 2, characterized in that, Determining the reduction mode corresponding to the input tensor based on the number of elements includes: Get the range of the number of elements corresponding to each specification pattern; Based on the number of elements and the range of element numbers corresponding to each reduction mode, the reduction mode corresponding to the input tensor is determined.

4. The method according to claim 3, characterized in that, The step of creating at least one thread block according to the specification pattern includes: Based on the specification pattern, a first quantity is determined; the first quantity is the number of thread blocks. Based on the specification pattern, a second quantity is determined; the second quantity is the number of threads included in the thread block. The at least one thread block is created based on the first quantity and the second quantity.

5. The method according to claim 4, characterized in that, Determining the first quantity according to the specification pattern includes: When the specification mode is the three-level specification mode, the product of the number of streaming multiprocessors in the computing device and a preset multiple is determined as the first number; or, When the reduction mode is the three-level reduction mode, the sequence length value corresponding to the input tensor is determined as the first quantity.

6. The method according to claim 4, characterized in that, Determining the second quantity according to the specification pattern includes: Obtain the feature dimension value corresponding to the input tensor; Determine whether the feature dimension value is greater than a preset threshold; If the feature dimension value is greater than the preset threshold, the preset threshold is determined as the second quantity; or, If the feature dimension value is less than or equal to the preset threshold, the preset threshold is determined as the second quantity.

7. The method according to claim 6, characterized in that, The step of obtaining the feature dimension value corresponding to the input tensor includes: When the specification mode is the three-level specification mode, the preset threshold is determined to be the second quantity; or, If the reduction mode is not the third-level reduction mode, obtain the feature dimension value corresponding to the input tensor.

8. The method according to claim 1 or 2, characterized in that, For any given thread block, the process involves running each thread within that thread block, performing reduction processing on the elements stored in multiple registers corresponding to that thread to obtain the reduction result for that thread, and updating the reduction result in the shared memory corresponding to the thread block based on the reduction result for that thread. This includes: For the i-th thread in any thread block, run the i-th thread and, based on the vectorization width, obtain multiple j-th batches of elements corresponding to the i-th thread from the input tensor stored in the global memory; Run the i-th thread and store the elements of the j-th batch corresponding to the i-th thread into the registers corresponding to the i-th thread; Run the i-th thread and perform reduction processing on the j-th batch of elements stored in the multiple registers corresponding to the i-th thread to obtain the reduction result of the j-th batch corresponding to the i-th thread; Run the i-th thread, and update the reduction result in the shared memory corresponding to the thread block according to the reduction result of the j-th batch corresponding to the i-th thread; Where i takes values ​​of 1, 2, ..., up to N; N is the number of threads included in the thread block; j takes values ​​1, 2, ..., up to M; M is the total number of elements in each thread's batch.

9. The method according to claim 8, characterized in that, The method further includes: Obtain the preset vectorized width corresponding to each specification mode; The vectorization width is determined based on the reduction mode corresponding to the input tensor and the preset vectorization width corresponding to each reduction mode.

10. The method according to claim 8, characterized in that, The method further includes: Obtain the number of elements corresponding to the input tensor; Obtain the storage address corresponding to the input tensor; Get multiple candidate vectorized widths; For any candidate vectorization width, determine whether the number of elements and the storage address are integer multiples of the candidate vectorization width; If the number of elements and the storage address are integer multiples of the candidate vectorization width, the candidate vectorization width is determined as the vectorization width.

11. The method according to claim 2, characterized in that, The method further includes: When the reduction mode is the register mode, each thread in the running thread block performs reduction processing on the elements stored in the multiple registers corresponding to the thread to obtain the reduction result corresponding to the thread, and updates the second reduction result stored in the global memory according to the reduction result corresponding to the thread. Run the target thread in the thread block and determine the second reduction result in the global memory as the reduction result corresponding to the input tensor.

12. The method according to claim 2, characterized in that, The method further includes: When the reduction mode is the shared memory mode, each thread in the running thread block performs reduction processing on the elements stored in the multiple registers corresponding to the thread to obtain the reduction result corresponding to the thread, and updates the reduction result in the shared memory corresponding to the thread block according to the reduction result corresponding to the thread. Run the target thread in the thread block, determine the reduction result in the shared memory corresponding to the thread block as the reduction result corresponding to the input tensor, and store the reduction result corresponding to the input tensor in the global memory.

13. The method according to claim 2, characterized in that, The method further includes: If the protocol mode is a shared memory mode or a three-level protocol mode, obtain the model of the graphics processor; The size of the shared memory is determined based on the model of the graphics processor; Based on the size of the shared memory, determine the shared memory corresponding to each thread block.

14. A computing device, characterized in that, include: A processor, and a memory communicatively connected to the processor; The memory is used to store computer-executed instructions; The processor is configured to execute computer execution instructions stored in the memory to implement the method of any one of claims 1-13.

Images