Arbitrary-granularity thread sub-block reduction method, electronic device, and storage medium

Abstract

The application relates to the technical field of artificial intelligence, and provides an arbitrary-granularity thread subblock reduction method, an electronic device and a storage medium, which comprise the following steps: acquiring a size parameter of a target thread subblock and a target reduction operation type; in the case that the size parameter is not a power of 2, decomposing the size parameter into a first value and a second value, and determining a target reduction strategy based on the first value and the second value; and calling a register-level thread communication instruction to perform multi-step inter-thread data exchange according to the target reduction strategy, so as to obtain a reduction result corresponding to the target reduction operation type. According to the application, the size of a thread subblock that is not a power of 2 is decomposed in dimension, and a register-level thread communication instruction is dispatched based on the size, so that pure register-level reduction calculation without shared memory transfer is realized under arbitrary granularity, and the application has the advantages of greatly reducing calculation delay, eliminating memory bandwidth bottlenecks, and improving the overall parallel computing throughput of a system.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and particularly to the field of parallel computing and thread-level data communication technology for artificial intelligence chips, and especially to an arbitrary granularity thread sub-block reduction method, electronic device and storage medium. Background Technology

[0002] In parallel computing of artificial intelligence chips, reduction operations (such as summation, finding maximum and minimum values) are among the most fundamental and widely used core operations. In the architecture of modern AI chips, reduction operations are typically performed on sub-tiles of a specified size within a warp. With the development of technologies such as deep learning, scenarios such as variable-length sequence partitioning and non-uniform grouping of sparse computations frequently arise, often leading to the need for reducing sub-tiles of arbitrary granularity (including granularities other than powers of 2) in practical applications.

[0003] However, current low-level efficient register-level thread communication instructions are inherently designed for power-of-two granularity. When directly applied to thread sub-block reduction at non-power-of-two granularity, they disrupt the topological closure of data exchange, leading to data out-of-bounds pollution across thread sub-block boundaries. To address this contradiction between hardware instruction characteristics and the irregular computational requirements of higher layers, related technologies have had to adopt some compromise alternatives when handling such arbitrary granularity reductions, but this often incurs certain performance overhead and application limitations.

[0004] Specifically, in related technologies, the reduction of thread sub-blocks with a granularity smaller than that of a warp mainly involves the following implementation methods: The first method is register-level communication reduction based on dynamic masks, which isolates different thread sub-blocks by dynamically calculating masks. However, its mask calculation logic is extremely complex, and different sub-block sizes and strides will produce mask values ​​that are difficult to unify, resulting in poor code maintainability and difficulty in compile-time optimization.

[0005] The second method is the reduction method for truncating steps. Although this method is relatively simple to execute, its applicability is severely limited. It only supports thread sub-blocks with a power of 2 granularity and cannot be directly applied to computation scenarios with a non-power of 2 granularity.

[0006] The third approach is a reduction method based on shared memory. When faced with granularities other than powers of 2, related technologies typically have to revert to this approach. However, frequent access to on-chip shared memory introduces additional clock cycle latency and can easily lead to severe bandwidth contention and resource conflicts during multi-threaded concurrency, resulting in a significant drop in overall computational throughput. Furthermore, switching between multiple different solutions for different granularities increases development complexity and maintenance costs. Summary of the Invention

[0007] This invention provides a method, electronic device, and storage medium for arbitrary granularity thread sub-block reduction, which solves the defects in related technologies such as complex computational logic, limited applicability of register-level thread communication instructions, and high latency and performance degradation caused by the use of shared memory schemes when performing arbitrary granularity thread sub-block reduction.

[0008] This invention provides a method for reducing thread sub-blocks with arbitrary granularity, comprising: Obtain the size parameter of the target thread sub-block and the target reduction operation type, wherein the size parameter is used to indicate the number of threads participating in the reduction operation; When the size parameter is not a power of 2, the size parameter is decomposed into a first value and a second value, and a target reduction strategy is determined based on the first value and the second value, wherein the first value is no more than the largest power of 2 of the size parameter, and the second value is the difference between the size parameter and the first value. According to the target reduction strategy, register-level thread communication instructions are invoked to perform multi-step inter-thread data exchange, and a reduction result corresponding to the target reduction operation type is obtained. The register-level thread communication instructions are used to perform hardware-level aggregation of data in the registers of each thread in the target thread sub-block through shared memory transfer at the chip physical hardware level.

[0009] According to the present invention, an arbitrary granularity thread sub-block reduction method, when the target reduction strategy is a conditional downward reduction strategy, includes the following steps: performing multi-step inter-thread data exchange by calling register-level thread communication instructions according to the target reduction strategy to obtain a reduction result corresponding to the target reduction operation type. Within the subset of threads defined by the first value in the target thread sub-block, the first type of register-level thread communication instruction is invoked to perform a main reduction operation and obtain the main reduction result; The second type of register-level thread communication instruction is invoked to perform a conditional downward reduction operation, so as to accumulate and merge the data of the residual thread corresponding to the second value and the main reduction result to obtain the reduction result.

[0010] According to the arbitrary granularity thread sub-block reduction method provided by the present invention, the step of calling a first type of register-level thread communication instruction to perform a main reduction operation and obtain a main reduction result includes: Based on the first value, a main reduction step sequence is generated that is applied within the subset of threads; According to the step size indicated by the main reduction step size sequence, the first type of register-level thread communication instructions control each thread in the thread subset to pair up and exchange data and perform reduction calculations, so that the data exchange range of each round is closed within the boundary of the thread subset, until the first thread in the thread subset obtains the main reduction result.

[0011] According to the present invention, an arbitrary granularity thread sub-block reduction method is provided, wherein the step of calling a second type of register-level thread communication instruction to perform a conditional downward reduction operation, so as to accumulate and merge the data of the residual thread corresponding to the second value and the main reduction result, includes: Multiple rounds of iterative communication are initiated according to an increasing communication step size to transmit the data of the residual thread level by level; In each round of iterative communication, the data to be transmitted by the target communication thread is obtained by calling the second type of register-level thread communication instruction. The target communication thread is the thread whose thread index is equal to the sum of the thread index of the currently executing thread and the current communication step size. Calculate the relative offset of the currently executing thread within the target thread sub-block, and sum the relative offset with the current communication step size to obtain the addressing span; Based on the comparison result between the addressing span and the size parameter, it is determined whether to accumulate and merge the transmitted data with the current data held by the currently executing thread, so as to gradually complete the data reduction between residual threads and the accumulation and merging with the main reduction result through multiple rounds of iterative communication.

[0012] According to the arbitrary granularity thread sub-block reduction method provided by the present invention, the step of determining whether to accumulate and merge the transmitted data with the current data held by the currently executing thread based on the comparison result of the addressing span and the size parameter includes: If the addressing span is less than the size parameter, the transmitted data is accumulated and merged with the current data held by the currently executing thread; If the addressing span is greater than or equal to the size parameter, the accumulation and merging of the transmitted data is not performed, and the current data held by the currently executing thread remains unchanged.

[0013] According to the arbitrary granularity thread sub-block reduction method provided by the present invention, when the target reduction strategy is an identity fill strategy, the step of calling register-level thread communication instructions to perform multi-step inter-thread data exchange according to the target reduction strategy to obtain a reduction result corresponding to the target reduction operation type includes: The first value is determined as twice the value, and the corresponding reduction identity is determined according to the target reduction operation type. For invalid threads in the target thread sub-block whose thread index is greater than or equal to the size parameter and less than the extended value, the reduction identity is applied to fill in the data. Based on the extended value, the first type of register-level thread communication instruction is invoked to perform a complete butterfly reduction communication on the filled thread sub-block, and the data obtained by the first thread after the communication is completed is determined as the reduction result.

[0014] According to the present invention, a method for reducing thread sub-blocks with arbitrary granularity is provided, wherein determining the corresponding reduction identity element based on the target reduction operation type includes: When the target reduction operation type is addition or bitwise OR, the zero value is determined as the reduction identity. When the target reduction operation type is bitwise AND, the all-1 value is determined as the reduction identity; When the target reduction operation type is a maximum value operation, the negative infinity value is determined as the reduction identity element; When the target reduction operation type is a minimum value operation, the positive infinity value is determined as the reduction identity.

[0015] The arbitrary granularity thread sub-block reduction method provided by the present invention further includes: When the size parameter is a power of 2, a butterfly reduction step size sequence is generated based on the size parameter; Based on the butterfly reduction step sequence, the first type of register-level thread communication instruction is invoked within the target thread sub-block to execute the complete butterfly reduction communication, so as to obtain the reduction result corresponding to the target reduction operation type.

[0016] The present invention also provides an arbitrary granularity thread sub-block reduction device, comprising: The parameter acquisition unit is used to acquire the size parameter of the target thread sub-block and the target reduction operation type, wherein the size parameter is used to indicate the number of threads participating in the reduction operation; The decomposition decision unit is used to decompose the size parameter into a first value and a second value when the size parameter is not a power of 2, and to determine a target reduction strategy based on the first value and the second value, wherein the first value is no more than the largest power of 2 of the size parameter, and the second value is the difference between the size parameter and the first value. The reduction execution unit is used to call register-level thread communication instructions to perform multi-step inter-thread data exchange according to the target reduction strategy, and obtain a reduction result corresponding to the target reduction operation type. The register-level thread communication instructions are used to perform hardware-level aggregation of data in the registers of each thread in the target thread sub-block through shared memory transfer at the chip physical hardware level.

[0017] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the processor, when executing the computer program, implements the arbitrary granularity thread sub-block reduction method as described above.

[0018] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the arbitrary granularity thread subblock reduction method as described above.

[0019] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the arbitrary granularity thread sub-block reduction method as described above.

[0020] The arbitrary granularity thread sub-block reduction method, electronic device, and storage medium provided by this invention obtain the size parameter and reduction operation type of the target thread sub-block, and decomposes it into a maximum power of 2 (not exceeding the parameter) and a residual difference (i.e., a first value and a second value) when the size parameter is not a power of 2. This allows for precise matching of a suitable target reduction strategy based on the decomposition result. This mechanism enables the system to overcome the technical limitations of relying on specific sizes in related technologies when processing non-standard granularities, eliminating dependence on high-latency shared memory. Furthermore, it directly invokes efficient register-level thread communication instructions to complete multi-step inter-thread data exchange according to the determined target reduction strategy. This pure register-level data exchange method not only avoids complex dynamic mask calculations but also eliminates the additional latency and resource contention issues caused by shared memory read / write operations, ensuring efficient and continuous execution of the reduction operation. Attached Figure Description

[0021] To more clearly illustrate the technical solutions in this invention or related technologies, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 This is a schematic diagram of the structure of the general-purpose graphics processor provided by the present invention; Figure 2This is one of the flowcharts illustrating the arbitrary granularity thread sub-block reduction method provided by the present invention; Figure 3 This is a schematic diagram of the overall architecture of the arbitrary granularity thread sub-block reduction system provided by the present invention; Figure 4 This is the second flowchart of the arbitrary granularity thread sub-block reduction method provided by the present invention; Figure 5 This is a schematic diagram of the arbitrary granularity thread sub-block reduction device provided by the present invention; Figure 6 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0023] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0024] With the rapid development of artificial intelligence and deep learning technologies, AI chips play a crucial role in parallel computing. Here, AI chips can refer to general-purpose graphics processing units (GPGPUs), graphics processing units (GPUs), tensor processing units (TPUs), etc.

[0025] For example, in modern GPGPU programming models, a warp typically contains 32 threads executing in parallel. Threads within the same warp can efficiently exchange data through register-level thread communication instructions (such as shuffle instructions). Reduction operations (such as summation, finding the maximum and minimum values) are among the most fundamental and frequent operations in GPGPU parallel computing, and are widely used in core deep learning operators such as Softmax normalization, layer normalization, attention mechanism computation, and TopK selection.

[0026] In practical parallel computing applications, it is often necessary to perform reduction operations on thread tiles smaller than 32 within a warp. Especially in many deep learning algorithms, due to scenarios such as block processing of variable-length sequences in attention mechanisms, reduction of non-uniform grouping in sparse computation, and matrix block computation where the row and column dimensions cannot be divided by the warp size, there is often a need for reducing thread tiles of arbitrary granularity (especially granularity that is not a power of 2), such as reducing every 3, 5, or 10 threads.

[0027] In the existing GPGPU architecture, tile reduction operations with granularity smaller than warp are mainly implemented using the following three methods: The first method is XOR reduction based on dynamic masks. This method uses the mask parameter of register-level thread communication instructions (such as the `shfl_xor_sync` instruction) to isolate different tiles, ensuring that each operation only exchanges data between threads within the same tile, thus achieving reduction at a smaller warp granularity. Specifically, for the reduction of multiple tiles within a warp, each call to the `shfl_xor_sync` instruction needs to calculate the corresponding mask to ensure that threads crossing tile boundaries do not participate in data exchange. For example, when the tile size is 4, a mask needs to be calculated to ensure that two shuffle operations with step size 1 and step size 2 do not cross the tile boundary of 4 threads.

[0028] The second method is butterfly reduction with truncated steps. This method observes that the first few steps of a butterfly network exchange data only within a specific subset of threads, naturally avoiding crossing tile boundaries. Therefore, for a tile size of N=2... k In the case where k is a positive integer, the reduction can be completed naturally by directly executing the first k XOR communication steps (i.e., the shfl_xor_sync instruction), without the need to calculate complex masks. For example, when N=2, 1 step (step size 1) is executed; when N=4, 2 steps (step sizes 1 and 2 respectively) are executed; when N=8, 3 steps (step sizes 1, 2 and 4 respectively) are executed, and so on.

[0029] The third method is reduction based on shared memory. Since the efficient `shfl_xor_sync` instruction mentioned earlier cannot handle non-power-of-two granularities, related technologies are forced to revert to using shared memory when dealing with non-power-of-two tile sizes. That is, each thread writes data to the GPGPU's shared memory, and data aggregation within the same tile is achieved through shared memory address mapping. Then, a single thread reads and completes the reduction. This method requires shared memory allocation, synchronization barriers, and multiple memory accesses.

[0030] However, the methods mentioned above all have certain technical drawbacks when handling thread sub-block reduction of arbitrary granularity (especially non-powers of 2). Specifically, the first method, XOR reduction based on dynamic masks, requires dynamic calculation of the mask for each step to prevent crossing tile boundaries. The mask calculation is heavily dependent on the current tile size and step size, making its logic complex and prone to errors. More importantly, different combinations of tile sizes and step sizes will produce different mask values, which are difficult to express using a unified template or mathematical formula. This not only leads to extremely poor code maintainability but also poses significant challenges to compile-time performance optimization.

[0031] The second butterfly reduction method for truncating steps, while concise and efficient, has an extremely narrow scope of application, only suitable for scenarios where the tile size is a power of 2 (e.g., 2, 4, 8, 16). In actual computation, this method becomes completely ineffective once the matrix dimension (e.g., attention head dimension, sequence length block) is not divisible by a power of 2 (e.g., tile sizes of 3, 5, 6, 10, etc.), leading to serious data pollution problems.

[0032] The third shared-memory-based reduction method requires writing data from registers to shared memory, implementing thread synchronization barriers, and then reading it back from shared memory. This introduces a significant additional clock cycle latency (typically 20-40 clock cycles). In high-frequency reduction scenarios (such as multiple-row reduction in fast attention mechanisms), this latency accumulates repeatedly along critical computation paths, leading to a substantial reduction in overall computational throughput. Moreover, shared memory is a scarce resource on the GPGPU chip. When multiple warps perform shared-memory reduction simultaneously, it can trigger shared-memory bandwidth contention and resource conflicts.

[0033] Furthermore, from a macro-level perspective of system development, the relevant technologies lack a unified programming interface. While efficient shuffle schemes can be used for power-of-2 granularity reduction, inefficient shared memory schemes are forced to be used for non-power-of-2 granularity reduction. These two completely different APIs (Application Programming Interfaces) and programming models significantly increase the burden on developers and the cost of code maintenance. Mainstream GPGPU computing libraries typically reject register-level reductions for non-power-of-2 granularities directly at compile time, forcing users to use inefficient shared memory paths.

[0034] To address the aforementioned shortcomings, this invention provides an arbitrary-granularity thread sub-block reduction method. Through a combination of butterfly topological decomposition, identity filling, and residual reduction strategies, the reduction of thread sub-blocks with any positive integer granularity can be efficiently completed via register-level thread communication instructions (such as shuffle instructions) without the need for shared memory intermediaries. This significantly reduces reduction latency, eliminates shared memory bandwidth bottlenecks, and provides a unified programming interface. This invention is applicable to all parallel computing scenarios involving reduction with granularity smaller than warp, demonstrating broad technical applicability.

[0035] It should be noted that the arbitrary granularity thread sub-block reduction method provided by this invention can be executed by a computing device with parallel computing capabilities. Specifically, this execution entity is typically an artificial intelligence chip such as a GPGPU, GPU, or TPU. The method of this invention is usually executed by the logic control unit, arithmetic logic unit, or dedicated decomposition decision unit and communication executor within the aforementioned chip. The structure of the execution entity of this invention will be described below using a GPGPU as an example.

[0036] Figure 1 This is a schematic diagram of the structure of the general-purpose graphics processor provided by the present invention, as shown below. Figure 1 As shown, this general-purpose graphics processing unit (GPGPU) includes at least multiple streaming processor clusters (SPCs) and global memory. Each SPC handles one computation task, or multiple SPCs collaboratively process a computation task, with data shared between SPCs via global memory. Each SPC includes multiple compute units (CUs) and intermediate caches (such as L1 caches) for sharing data among the compute units. Each compute unit includes multiple compute cores, which are used to execute specific computation tasks. Compute cores are typically scheduled in warps.

[0037] Each computing unit also includes registers, shared caches, or shared memory for hierarchical storage of data related to computing tasks. Shared caches or shared memory within a computing unit are used to share data among the computing cores of that unit. A shared cache (also known as an on-chip cache) is a temporary memory, for example, a Gemm Main Buffer (GMB), which has a smaller capacity than an L1 cache but a faster data exchange speed. In contrast, registers have a smaller capacity than shared caches but a faster data exchange speed. Specifically, registers can be thread-local registers (TLRs).

[0038] GPGPUs may also include an L2 cache, a high-speed cache located between global memory and L1 cache, used to reduce global memory access latency. The L2 cache is typically a shared resource across the entire GPGPU chip, rather than being private to each SPC; multiple SPCs can share data through the L2 cache.

[0039] Furthermore, the computational units in SPC are divided into tensor computation units and vector computation units. Tensor computation units perform tensor computations, such as matrix multiplication and convolution. Within a tensor computation unit, the computational core, also called a tensor core, is used to perform specific computational tasks. Vector computation units primarily perform arithmetic and logical operations other than tensor computations such as matrix multiplication and convolution, including accumulation, reduction, and regular addition, subtraction, multiplication, and division. Within a vector computation unit, the computational core, also called a vector core, is used to perform specific computational tasks.

[0040] In addition to the structures described above, the general-purpose graphics processor of this invention may also include other structures, and this invention does not specifically limit them.

[0041] It is understood that the method provided by this invention can be widely and directly applied to the core operator computation of large deep learning models and graphics rendering. For example, in variable-length attention computation scenarios, since the length of the input natural language text or image sequences varies, dividing the sequence into blocks of different lengths will generate edge thread sub-blocks (tiles) of size that are not powers of 2. This invention can be directly used for efficient local aggregation of such variable-length sequence features. Furthermore, in sparse computation scenarios (such as sparse attention mechanisms or sparse matrix computation), this invention provides seamless reduction operation support for nodes with non-uniform grouping.

[0042] Meanwhile, in different specific application scenarios, the thread data participating in the underlying register-level reduction calculation is not an abstract number detached from reality, but rather an underlying multidimensional feature derived from objectively existing natural signals such as text, images, and speech. This data is stored in real physical registers. For example, in text processing scenarios, thread data typically represents word embedding vector components after text character mapping, hidden layer feature tensors output by each layer of a neural network model, or attention scores between query vectors and key vectors generated during attention mechanism calculations. In image processing scenarios, thread data typically represents feature components of image pixels, activation values ​​of high-dimensional feature maps generated during feature extraction, or local pixel receptive field features in image pooling layers. In speech processing scenarios, thread data typically represents various speech features after signal processing and feature extraction. This invention, by scheduling underlying register-level thread communication instructions, directly and efficiently aggregates this thread data representing real-world environmental features at the chip's physical hardware level, thereby achieving the technical effects of reducing computational latency and saving on-chip shared memory resources.

[0043] Based on the above embodiments, Figure 2 This is one of the flowcharts illustrating the arbitrary granularity thread sub-block reduction method provided by this invention, such as... Figure 2 As shown, the method includes: Step S10: Obtain the size parameter of the target thread sub-block and the target reduction operation type. The size parameter is used to indicate the number of threads participating in the reduction operation.

[0044] Specifically, in the field of parallel computing, reduction operations aim to aggregate multiple data elements into a single result using specific binary operators. In practical scenarios involving variable-length sequence processing, sparse computation, or matrix edge-block computation, it is often necessary to perform local reduction on a group of consecutive threads within a warp (typically less than the total number of threads in the warp, for example, less than 32). This group of consecutive threads is referred to as a target thread tile in this embodiment of the invention.

[0045] Specifically, the size parameter of the target thread sub-block (denoted as N) is the size of the tile, representing the number of valid elements or threads participating in this reduction operation. Its value range is usually any positive integer of 1 ≤ N ≤ 32. The target reduction operation type refers to the specific mathematical or logical operation performed on the data value held by the thread. Common operation types include addition (sum), finding the maximum value (max), finding the minimum value (min), bitwise AND, and bitwise OR. When executing step S10, the system (e.g., the built-in decomposition decision unit or input layer logic) captures the above reduction parameters and obtains the original data value (i.e., thread data) currently held in the register of each participating thread, preparing for subsequent calculations and communication.

[0046] Step S20: When the size parameter is not a power of 2, the size parameter is decomposed into a first value and a second value, and a target reduction strategy is determined based on the first value and the second value, wherein the first value is no more than the largest power of 2 of the size parameter, and the second value is the difference between the size parameter and the first value.

[0047] It should be noted that the fast reduction in related technologies mainly relies on instructions based on XOR butterfly topology (i.e., XOR communication instructions, such as the shfl_xor_sync instruction). This topology inherently requires the number of participating nodes to be a power of 2. When encountering arbitrary granularity scenarios where the size parameter N is not a power of 2 (e.g., N=3, 5, 6, 10, etc.), directly applying the existing topology will lead to out-of-bounds pollution. Therefore, this embodiment of the invention introduces a decomposition mechanism based on mathematical characteristics.

[0048] Specifically, when the system determines that N is not a power of 2, it will uniquely decompose any positive integer N into its first value (denoted as 2). k ) and the second value (denoted as r), i.e., N=2 k +r. Where k is mathematically expressed as... ( (This indicates rounding down), i.e., the first value 2. k The first value is the largest power of 2 not exceeding N, representing the size of the largest subset of threads to which the standard butterfly topology can be directly applied; the second value satisfies r = N - 2. k (0≤r<2) k ), which represents the number of residual threads that cannot be directly placed into a standard butterfly network after the main reduction subset is stripped from the entire tile.

[0049] Furthermore, based on the first and second values ​​obtained from the decomposition, the system automatically determines the target reduction strategy. The strategy decision logic here aims to find the optimal execution path for the currently non-uniform thread sub-blocks. For example, if r > 0 is detected, the system will abandon a single pure butterfly reduction and instead choose from at least two reduction strategies: one is a strategy that utilizes the characteristic of the first value to construct virtual boundaries for identity element filling; the other is a strategy that first reduces the subset of threads covered by the first value, and then performs a unidirectional accumulation and merging conditional downward reduction strategy on the remaining portion covered by the second value. This strategy routing design based on numerical decomposition ensures that reduction requests of various irregular granularities can be mapped to suitable topologies for execution.

[0050] Step S30: According to the target reduction strategy, register-level thread communication instructions are invoked to perform multi-step inter-thread data exchange to obtain a reduction result corresponding to the target reduction operation type. The register-level thread communication instructions are used to perform hardware-level aggregation of data in the registers of each thread in the target thread sub-block through shared memory transfer at the chip physical hardware level.

[0051] Specifically, after determining the specific target reduction strategy, the hardware will perform the actual data aggregation operation. The aforementioned register-level thread communication instructions (such as the shuffle instruction) are low-level instructions that can directly share register values ​​between different threads in the same warp, without going through on-chip shared memory or global memory.

[0052] In this step, whether targeting the subset of the main thread containing the first value or the residual portion corresponding to the second value, the hardware (such as the Shuffle executor in the execution logic) generates a series of step-size sequences according to a predetermined strategy and calls the corresponding register-level thread communication instructions (such as the XOR communication instruction shfl_xor_sync or the down communication instruction shfl_down_sync). During the multi-step iterative data exchange process, the data in the registers of each thread converges and is accumulated in combination with the obtained target reduction operation type. After the multi-step instruction execution is completed, the register of the specified thread within the target thread sub-block (usually the first thread with an offset of 0 relative to the tile's starting position) will hold the fully aggregated value, which is the final required reduction result. This result can then be directly used for downstream operator calculations or written to global memory.

[0053] The method provided in this invention obtains the size parameter and reduction operation type of the target thread sub-block, and decomposes it into a maximum power of 2 (not exceeding the parameter) and a remaining difference (i.e., a first value and a second value) when the size parameter is not a power of 2. This allows for precise matching of a suitable target reduction strategy based on the decomposition result. This mechanism enables the system to overcome the technical limitations of relying on specific sizes in related technologies when processing non-standard granularities, eliminating dependence on high-latency shared memory. Furthermore, it directly invokes efficient register-level thread communication instructions to complete multi-step inter-thread data exchange according to the determined target reduction strategy. This pure register-level data exchange method not only avoids complex dynamic mask calculations but also eliminates the additional latency and resource contention issues caused by shared memory read / write operations, ensuring efficient and continuous execution of the reduction operation.

[0054] Based on any of the above embodiments, when the target reduction strategy is a conditional downward reduction strategy, step S30 specifically includes: Step S31: Within the subset of threads defined by the first value in the target thread sub-block, a first type of register-level thread communication instruction is invoked to perform a main reduction operation to obtain the main reduction result.

[0055] Specifically, when using a conditional downward reduction strategy (also known as a hybrid shuffle strategy or residual reduction strategy), the system does not directly perform indiscriminate communication on the entire non-uniform target thread sub-block (of size N). Instead, it first decomposes the reduction task. Specifically, the system defines a preceding subset of threads within the current target thread sub-block. The number of consecutive threads in this subset is equal to the first value obtained from the aforementioned decomposition (i.e., 2). k Since the size of this subset naturally satisfies the power of 2 condition, the system specifically calls the first type of register-level thread communication instruction for this subset.

[0056] The first type of register-level thread communication instruction here typically refers to low-level instructions based on symmetric interaction logic (such as the XOR shuffle instruction `shfl_xor_sync` in modern GPGPU libraries). This instruction is used in the above two... k A fast main reduction operation is performed within a local subset consisting of several threads. Once this phase of the operation is complete, the subset of threads can aggregate and output a main reduction result based on the partial data.

[0057] Step S32: Invoke the second type of register-level thread communication instruction to perform a conditional downward reduction operation, so as to accumulate and merge the data of the residual thread corresponding to the second value and the main reduction result to obtain the reduction result.

[0058] Specifically, after completing the main reduction phase, a number of residual threads (limited by the second value, r) remain in the target thread sub-block that have not yet participated in the reduction. Since the number of these residual threads is irregular, continuing to use the `shfl_xor_sync` instruction might cause data to cross the boundary of the target thread sub-block. Therefore, the system seamlessly switches to calling a second-type register-level thread communication instruction to perform the conditional down-reduction operation. Here, the second-type register-level thread communication instruction typically refers to instructions with specific unidirectional transmission characteristics, such as the down-shuffle instruction `shfl_down_sync`.

[0059] Through this supplementary operation, the system can safely extract the r data points scattered in the residual threads and perform a secondary accumulation and merging with the just obtained main reduction result. The final merged value is the reduction result corresponding to the entire target thread sub-block of size N.

[0060] The method provided in this invention designs an architecture combining decomposition and hybrid communication instructions for non-power-of-2 granularity scenarios. By splitting the reduction task into a main reduction for regular subsets and a conditional downward residual reduction for irregular tails, it fully leverages the topological advantages of different underlying instructions. This ensures that most data can be aggregated at the fastest speed while ensuring that irregular residuals can be safely integrated. Thus, without introducing shared memory, it maximizes and optimizes the utilization of hardware register communication resources for arbitrary thread-granularity reduction.

[0061] Based on any of the above embodiments, step S31 specifically includes: Step S311: Based on the first value, generate a main reduction step sequence applicable to the subset of threads.

[0062] Specifically, to drive multi-step parallel communication, the system first needs to plan the span (i.e., step size) of each data exchange. In this step, the hardware logic will determine the first value (i.e., 2) based on the previously determined first value. k The main reduction step sequence is generated dynamically or statically and applied within a subset of threads. Specifically, this sequence exhibits standard doubling properties, with a length of the main reduction step number k. The step sequence can be represented by the number of rounds as {1, 2, 4, ..., 2}. k-1 This sequence precisely guides the logical thread distance required for each data exchange step during the standard butterfly reduction performed within the subsequent thread subset.

[0063] Step S312: According to the step size indicated by the main reduction step size sequence, the first type of register-level thread communication instruction is used to control each thread in the thread subset to pair up and exchange data and perform reduction calculation, so that the data exchange range of each round is closed within the boundary of the thread subset, until the first thread in the thread subset obtains the main reduction result. Among them, the first type of register-level thread communication instructions has a subset closed topology characteristic based on XOR pairing.

[0064] Specifically, after generating the step size sequence, the system enters the execution loop. In multiple iterations, according to each step size (denoted as s) indicated by the main reduction step size sequence, the system cyclically triggers the first type of register-level thread communication instruction. During this process, the underlying mathematical and physical mechanism of this instruction, namely the subset closed topology based on XOR pairing, plays a core role.

[0065] Here, the subset-closed topology property of XOR pairing means that when controlling thread pairing, this instruction will force the thread with thread ID (also called thread index) t to be paired with the thread with thread ID t⊕s (⊕ represents the XOR operation). This is because the maximum step size in the main reduction sequence is 2. k-1Therefore, for any s < 2 k According to the mathematical closure law of the XOR operation, the pairing relationship always remains t / 2. k The divisibility value remains unchanged.

[0066] This means that no matter how the two paired threads jump within the subset, they will never go beyond a size of 2. k The boundaries of the thread subset are defined, thus avoiding the risk of out-of-bounds pollution from fetching data from adjacent tiles. Under this closed topology, each thread pairs up to exchange data and perform reduction calculations. After a complete k-step iteration, the data originally scattered across 2... k The data in each thread will be fully aggregated until the first thread in the thread subset (usually the first thread with an offset of 0 in the current target thread sub-block) finally obtains the main reduction result, preparing for any possible residual merging in the future.

[0067] The method provided in this invention utilizes the inherent mathematical topological properties of the first type of register-level thread communication instructions. It directly drives the threads within a subset to perform standard XOR butterfly reduction to achieve data exchange through a power-of-2 step size sequence. This scheme does not require calculating and configuring complex dynamic masks to isolate thread sub-block boundaries in the pre-step process. It achieves high-speed local reduction with zero out-of-bounds pollution solely by relying on the subset closure characteristics of the underlying instructions. This simplifies the execution logic and reduces the difficulty of compilation optimization, ensuring high computational efficiency in the register domain during the main reduction stage.

[0068] Based on any of the above embodiments, in order to accurately and safely aggregate the data of the residual threads into the main reduction result, step S32, which involves calling the second type of register-level thread communication instruction to perform a conditional downward reduction operation, to accumulate and merge the data of the residual threads corresponding to the second value and the main reduction result, specifically includes: Step S321: Initiate multiple rounds of iterative communication according to an increasing communication step size to transmit the data of the residual thread level by level.

[0069] Specifically, when processing residual data that is not a power of 2, the system cannot collect all data in a single communication; instead, a multi-round transmission mechanism needs to be designed. Specifically, the system will set a communication step size (e.g., sequence) that is initially 1 and doubles in each round. The total amount needed log2N step, · (This indicates rounding up), and multiple rounds of iterative communication are initiated according to an increasing communication step size. This tree-like, hierarchical transmission method ensures that even residual thread data distributed at the end of the target thread's sub-blocks can flow efficiently towards the first thread as the step size increases exponentially.

[0070] Step S322: In each round of iterative communication, the data to be transmitted by the target communication thread is obtained by calling the second type of register-level thread communication instruction. The target communication thread is the thread whose thread index is equal to the sum of the thread index of the currently executing thread and the current communication step size. The second type of register-level thread communication instruction has a unidirectional transmission topology characteristic based on the decreasing direction of the thread index.

[0071] Specifically, in each iteration, the hardware execution engine triggers a second type of register-level thread communication instruction (such as the down shuffle instruction `shfl_down_sync`). The key difference between this instruction and the first type (XOR pairing) lies in its topology characteristic; it features a unidirectional transfer topology based on the decreasing direction of the thread index. In other words, this instruction always forces data to flow unidirectionally from the higher-numbered thread to the lower-numbered thread.

[0072] Specifically, if the thread index (i.e., Lane ID) of the currently executing thread is t and the current communication step size is s, the system will automatically locate a target communication thread, whose index is precisely locked as t+s. By calling this instruction, the currently executing thread t can skip the step size s and directly retrieve the transmitted data held by the thread t+s from its register. Since the target thread's sub-tiles are usually arranged consecutively within a warp, this high-to-low directionality perfectly matches the data convergence requirements of consecutive tile boundaries. However, with small step sizes, this also introduces the risk of incorrectly retrieving the tail data of the next adjacent tile. Therefore, interception is required in conjunction with subsequent steps.

[0073] Step S323: Calculate the relative offset of the currently executing thread within the target thread sub-block, and sum the relative offset with the current communication step size to obtain the addressing span.

[0074] Specifically, to identify whether the captured data belongs to the legitimate data of this tile, the system introduces a dynamic boundary awareness mechanism. First, the system calculates the relative offset (denoted as t_offset) of the current executing thread within the target thread's sub-block. This offset can be obtained by taking the modulo of the current thread index t and the size parameter N (i.e., t_offset = t % N), representing the thread's logical relative position within its own tile (e.g., from 0 to N-1). Next, in the hardware logic calculation unit, the system adds this relative offset to the current communication step size (i.e., s) of this round of communication to obtain the addressing span. This addressing span physically represents the logical coordinates of its data source (i.e., the target communication thread) within the current tile if the current thread performs a capture action.

[0075] Step S324: Based on the comparison result between the addressing span and the size parameter, determine whether to accumulate and merge the transmitted data with the current data held by the currently executing thread, so as to gradually complete the data reduction between residual threads and the accumulation and merging with the main reduction result through multiple rounds of iterative communication.

[0076] Specifically, after obtaining the addressing span, the system uses this addressing span and the size parameter N as a basis for conditional judgment. Because the logical coordinates of any valid thread within a tile are strictly less than N, the hardware logic only needs to perform a simple numerical comparison, i.e., determine whether t_offset+s is less than N, to immediately detect whether the captured data comes from valid residual data within the same tile or has crossed over into the next adjacent tile. Based on this comparison result, the system ultimately determines whether to accumulate and merge the transmitted data with the current data held by the currently executing thread, thereby gradually completing the data reduction between residual threads through multiple rounds of iterative communication. After obtaining the residual reduction result, it can be accumulated and merged with the main reduction result.

[0077] The method provided in this invention solves the data aggregation problem of residual irregular threads by introducing a second type of register-level thread communication instruction with natural directional adaptability and combining it with a step-by-step transmission mechanism driven by incremental step size. In particular, the method incorporates address span calculation based on the sum of relative offset and step size during communication, replacing complex dynamic mask calculations with low-overhead register-level addition and comparison operations. This allows each thread to autonomously perceive boundaries without external coordination, significantly reducing the complexity of control logic and effectively ensuring efficient instruction flow in a parallel architecture.

[0078] Based on any of the above embodiments, step S324 specifically includes: If the addressing span is less than the size parameter, the transmitted data is accumulated and merged with the current data held by the currently executing thread; If the addressing span is greater than or equal to the size parameter, the accumulation and merging of the transmitted data is not performed, and the current data held by the currently executing thread remains unchanged.

[0079] Specifically, when the system determines that the addressing span is less than the size parameter (i.e., t_offset+s<N), it indicates that although the data source end (i.e., the target communication thread) that the current execution thread attempts to acquire has a higher index number, its logical span does not exceed the limit of N, so the data source end definitely belongs to the same current target thread sub-block (tile). This means that the data transmitted through the second type of register-level thread communication instruction comes from the high-order residual thread within the current tile (or the intermediate residual value accumulated in the previous step), which is valid data that needs to be included in the sum. In this case, the system triggers the arithmetic logic unit to safely accumulate and merge the transmitted data with the current data held by the current execution thread.

[0080] Conversely, when the system determines that the addressing span is greater than or equal to the size parameter (i.e., t_offset+s≥N), it indicates that the current communication step is too large, or the current thread is originally located at a rear position of the tile, resulting in the calculated data source end having crossed the valid boundary of the current tile and falling into the range of the next adjacent tile. If the merging is forcibly performed at this time, the data of the next tile will seriously pollute the reduction result of the current tile. To avoid such data pollution, the system will perform conditional masking at this time, and will not perform the accumulation and merging of the transmitted data. In other words, the hardware level will discard or ignore the captured transmitted data of the target communication thread, and keep the current data held by the current execution thread unchanged.

[0081] The method provided by the embodiments of the present invention constructs a safe virtual boundary protection for the non-power-of-two granularity reduction process by setting an accurate threshold blocking condition (i.e., the numerical comparison between the addressing span and the size parameter) at the hardware level. When the thread subset uses the down-pass instruction shfl_down_sync to collect residual data, there is no need to perform boundary division at the physical memory level, and cross-tile data out-of-bounds pollution can be avoided through logical judgment. While ensuring the correctness of the final reduction result, the performance potential of multi-thread parallel computing is maximally released.

[0082] Based on any of the above embodiments, in the case that the target reduction strategy is an identity element filling strategy, step S30 specifically includes: Step S31', determining a double value of the first numerical value as an extended numerical value, and determining a corresponding reduction identity element according to the type of the target reduction operation; Step S32', for invalid threads in the target thread sub-block whose thread index is greater than or equal to the size parameter and less than the extended numerical value, applying the reduction identity element for data filling; Step S33', based on the extended value, call the first type of register-level thread communication instruction to perform complete butterfly reduction communication on the filled thread sub-block, and determine the data obtained by the first thread after the communication ends as the reduction result.

[0083] Specifically, when the system determines that the size parameter N is not a power of 2, in addition to the aforementioned conditional downward reduction strategy, an identity element filling strategy based on mathematical equivalent substitution may also be selected. When the identity element filling strategy is adopted, its core is to reconstruct an irregular target thread sub-block logically into a regular virtual thread block with a size that is a power of 2 by filling the missing thread positions.

[0084] Specifically, the system will take the first value decomposed previously (i.e., the largest power of 2 not exceeding the size parameter N 2 k ) and multiply it by 2, and determine twice the first value as the extended value (denoted as K, i.e., K=2×2 k ). Mathematically, this extended value K is exactly equal to the smallest power of 2 that is not less than N.

[0085] At the same time, to ensure that the filling operation does not change the final calculation result, the system will determine the corresponding reduction identity element in advance according to the target reduction operation type. The reduction identity element (denoted as e) herein refers to a special value such that in a specific binary reduction operation, the result of any valid data operating with this element remains unchanged.

[0086] After the virtual extended value K and the reduction identity element e are determined, the system will create a logical filling area at the end of the current target thread sub-block (tile). Specifically, for a thread sub-block originally of size N, the range of thread indices (relative offsets) of threads that actually hold valid data inside is 0 to N-1. Therefore, the system forcibly executes a data replacement operation for invalid threads in the target thread sub-block whose thread index is greater than or equal to the size parameter N and less than the extended value K (there are a total of K-N such threads), that is, applies the reduction identity element for data filling. Specifically, for a thread t that satisfies N≤t<K, its register data value is forcibly set to e; while for valid threads with t<N, their original data remains unchanged.

[0087] After the above padding operation, the system logically obtains a regularized thread subblock of size K containing some identities. Since K is a standard power of 2, the system no longer needs to introduce any complex boundary checks or mask calculations. It directly calls the first type of register-level thread communication instruction (such as the shfl_xor_sync instruction) based on the extended value to perform a complete butterfly reduction communication on the padded thread subblock. The system will perform a complete log2K-step data exchange. After this series of standard butterfly communications is completed, the padded identity e will naturally be incorporated into and consumed in the multi-level reduction operation (due to its algebraic properties, it will not produce any substantial change to the valid data). Finally, the system determines the data obtained by the first thread after the communication is the reduction result, which is mathematically strictly equivalent to the result of reducing only the first N valid elements.

[0088] The method provided in this invention expands irregular thread blocks that are not powers of 2 upwards to powers of 2, and precisely injects reduction identities that will not interfere with the calculation results into blank and invalid thread positions. This allows reduction requests of any granularity to directly reuse the most basic and efficient pure butterfly topology network in the underlying hardware. This scheme is logically simple, eliminates the overhead of dynamic mask calculation and branch condition judgment, and achieves extremely low instruction execution latency while sacrificing a very small amount of redundant thread computing resources.

[0089] Based on any of the above embodiments, step S31', determining the corresponding reduction identity element according to the target reduction operation type, includes: When the target reduction operation type is addition or bitwise OR, the zero value is determined as the reduction identity. When the target reduction operation type is bitwise AND, the all-1 value is determined as the reduction identity; When the target reduction operation type is a maximum value operation, the negative infinity value is determined as the reduction identity element; When the target reduction operation type is a minimum value operation, the positive infinity value is determined as the reduction identity.

[0090] Specifically, when the system recognizes that the currently executed reduction operation is a summation or a logical bitwise OR, according to the laws of mathematical algebra, any value plus 0 still equals that value, and any binary value ORed with 0 also retains its original value. Therefore, the system extracts the constant 0, determines the zero value as the reduction identity, and injects it into the aforementioned invalid thread register.

[0091] When the system recognizes that the target reduction operation type is a bitwise AND operation, due to the characteristics of the bitwise AND operation, any binary bit will retain its original state when ANDed with 1. Therefore, the system will determine the all-1 value as the reduction identity, thereby ensuring that when this part of invalid data participates in bitwise operations, it does not change the valid data bits of adjacent valid threads.

[0092] When deep learning operators (such as Softmax forward propagation) need to perform a maximum value operation, the system needs to find a number that will never become the final maximum value to fill the gap. Since any real number is greater than or equal to negative infinity, the system automatically extracts the smallest lower bound that can be represented by the current data type, that is, the value of negative infinity (-inf) is determined as the reduction identity. In this way, during multi-step comparisons and swaps, these fake filler data will naturally be eliminated by the true maximum value.

[0093] Similarly, when an operator needs to perform a minimum (min) operation, to prevent invalid data from accidentally becoming the final extracted minimum value, the system employs a reverse extreme value protection strategy. Since any real number is less than or equal to positive infinity, the system extracts the maximum upper bound that the current data type can represent, that is, the positive infinity value (+inf) is determined as the reduction identity, so that the real valid data can be safely retained in the first thread after multiple comparisons.

[0094] The method provided in this invention ensures that the identity fill strategy maintains numerical equivalence under different operator types by precisely converting mathematical logic of different dimensions into constant filling rules at the hardware level. This deterministic mapping criterion not only avoids the contamination of results by invalid data in high-speed butterfly communication, but also reduces the burden on developers and the probability of code errors when adapting to different underlying operation types.

[0095] Based on any of the above embodiments, the method further includes: When the size parameter is a power of 2, a butterfly reduction step size sequence is generated based on the size parameter; Based on the butterfly reduction step sequence, the first type of register-level thread communication instruction is invoked within the target thread sub-block to execute the complete butterfly reduction communication, so as to obtain the reduction result corresponding to the target reduction operation type.

[0096] Specifically, when the system receives a reduction request and makes a decomposition decision, if it determines that the current size parameter N is itself a power of 2 (i.e., the second value r=0 and N=2 obtained through the aforementioned decomposition logic), kThe system will determine that the current target thread sub-block is a regular subset. In this case, the system does not need to perform any identity padding or initiate any conditional downward reduction logic to process residual data. The system will directly generate a butterfly reduction step sequence based on the size parameter. Specifically, the length of this step sequence is k (i.e., log2N), and its sequence content follows a standard doubling rule, i.e., {1, 2, 4, ..., 2^k}. k-1}

[0097] After obtaining the standard step size sequence, the system enters the high-speed execution phase. Based on the butterfly reduction step size sequence, the system iteratively calls the first type of register-level thread communication instruction (i.e., the shfl_xor_sync instruction) within the target thread sub-block. During the k iterations, all threads within the target thread sub-block utilize the high-speed register exchange channel provided by the underlying hardware to perform data exchange and target reduction operations in pairs.

[0098] Since the size parameter N is exactly equal to 2 at this time. k Based on the subset closure property of XOR topology, all data exchanges generated by these k-step communication steps are naturally and strictly confined within the logical boundary formed by these N threads, preventing any data broadcasting or out-of-bounds pollution across tile boundaries. Therefore, the system can execute complete butterfly reduction communication. After these k-step instructions are executed, the thread at the beginning of the target thread sub-block (the thread with a relative offset of 0) will naturally aggregate the calculated values ​​of all N elements, thus obtaining the reduction result corresponding to the target reduction operation type.

[0099] The method provided by this invention provides developers with a unified programming interface that shields the differences in underlying logic. When faced with regular thread sub-blocks, the system can automatically enter the most direct, lightweight, and zero-overhead pure butterfly reduction path. While ensuring backward compatibility with mainstream hardware and optimal implementation, it avoids any unnecessary logical judgments or resource waste, and achieves full coverage and efficient output of parallel computing architecture in various granular scenarios.

[0100] Based on any of the above embodiments Figure 3 This is a schematic diagram of the overall architecture of the arbitrary granularity thread sub-block reduction system provided by the present invention. Figure 4 This is the second flowchart of the arbitrary granularity thread sub-block reduction method provided by the present invention. The following will be combined with... Figure 3 and Figure 4 This paper describes the overall architecture and complete method flow process of the arbitrary granularity thread sub-block reduction system provided by the present invention.

[0101] like Figure 3As shown, the system adopts a hierarchical and modular design, specifically divided into five core layers from top to bottom: the input layer, the decomposition decision layer, the butterfly reduction execution layer, the conditional downward reduction layer, and the output layer. The input layer, serving as the system's unified entry point, is primarily responsible for receiving three key parameters from the upper-layer application or operator scheduling: the thread data held by each thread participating in the reduction, the size N of the thread sub-block indicating the number of threads participating in the reduction operation, and the reduction operation type specifying the specific operation type (such as sum, max, min, etc.).

[0102] The decomposition decision layer is the intelligent routing hub of the system. It first performs mathematical analysis on the received N to determine if N is a power of 2. If the result is yes, it indicates that the current thread sub-block is naturally regular, and the system directly routes it to an efficient butterfly reduction path. If the result is no, it indicates that there is irregular granularity, and the system routes it to an identity fill / residual reduction path, further selecting a specific strategy based on the specific scenario or user configuration.

[0103] The butterfly reduction execution layer primarily handles butterfly reduction tasks based on XOR topology and contains three important components. The step sequence generator automatically generates a power-of-2 step sequence based on the input size parameter N. The identity injector specifically serves the identity filling strategy, accurately injecting reduction identities at virtually extended invalid thread positions without altering the computation result. The Shuffle executor is the core execution unit for underlying hardware communication, responsible for receiving the step sequence and thread data (processed or unprocessed by the injector), and invoking XOR-based pairing instructions to complete high-speed data exchange. It should be understood that the Shuffle executor uses the `shfl_xor_sync` instruction during the main reduction phase and the `shfl_down_sync` instruction during the conditional down reduction phase.

[0104] The conditional down reduction layer is specifically designed for residual reduction strategies (also known as conditional down reduction strategies) that handle irregular tail data. Its core mechanism uses `shfl_down_sync` to iteratively pass data in an incremental step size, tightly coupled with a boundary checker. This boundary checker determines the validity of the currently passed data by simply summing the offset and the step size, thus preventing out-of-bounds pollution without using dynamic masks.

[0105] Once any path in the execution layer has been completed, the results will converge to the output layer, and the final reduction result will be output. Usually, the first thread in the tile holds the final reduction result.

[0106] Figure 4 The dynamic flow of data within the aforementioned system architecture is illustrated in detail, such as... Figure 4As shown, the process starts from the beginning node and includes the following execution steps: First, the system receives reduction parameters, including the aforementioned thread block size N, reduction operation type, and thread data. Then, it enters the parameter decomposition phase, where the system calculates the main reduction step number k and the residual r using logarithm and subtraction operations. Next, the process enters the core decision node, determining whether r is 0 (i.e., whether N is a power of 2).

[0107] If the determination is yes (i.e., r is 0), the system enters the simplest pure butterfly reduction path. Under this path, the system does not need any additional processing and directly performs k-step XOR butterfly reduction. After all threads complete data exchange within the closed subset, they directly flow to the bottom output reduction result node.

[0108] If the judgment is negative (i.e., r is not 0), the system faces irregular granularity and enters the node for selecting a reduction strategy. Based on the system scheduler, this can be differentiated into two execution paths: One path is the identity filling path. Upon entering this path, the system first fills the identity in the invalid thread, padding the original irregular sub-block of size N upwards to the nearest power of 2 (i.e., the expanded complete sub-block). Subsequently, the hardware performs XOR butterfly reduction on the expanded complete sub-block. Since the filling is with identity elements, the reduction does not affect the valid data, and the process smoothly proceeds to the output reduction result node.

[0109] Another path is the conditional downward reduction path, which is specifically divided into two execution phases: primary reduction and residual reduction. First, it enters the primary reduction phase (…). Figure 4(Not shown in the diagram) Within a subset of threads defined by a first value (i.e., a power of 2 not exceeding N), the system executes k-step butterfly reduction by calling an XOR communication instruction. This ensures that the threads within this subset efficiently exchange data within the closed boundary, yielding the main reduction result. Subsequently, the system enters the residual reduction phase, initiating multiple iterations to process the data from the residual threads and merge it with the main reduction result. In each iteration, the required communication step size is first configured (e.g., the iteration step size for local residual reduction and the communication step size for global merging) and downward communication data is acquired. The data then undergoes boundary checks, specifically checking if the sum of the offset and the current iteration step size (i.e., the addressing span) is less than N. If yes, it indicates the data originates within this tile, and the system performs reduction accumulation (updating the current value). If no, it indicates the step size is too large, causing data from adjacent tiles to be captured; the system triggers a protection mechanism, skipping the accumulation (preserving the original value). After completing one judgment and processing, the system continues to judge whether the current step size is less than N. If it is, it means that the entire tile has not been covered yet. The process reverts to the feedback loop, doubles the step size, and continues to configure the step size for the next round and obtain the downward communication data. If it is not, it means that all residual data has been successfully converged to the first thread. The process jumps out of the loop and enters the output reduction result node.

[0110] Ultimately, regardless of which path is taken, the process will uniformly reach the convergence point, which is the output of the reduction result. At this point, the first thread is guaranteed to hold the complete reduction result, and the entire multi-threaded parallel reduction task successfully reaches the end node.

[0111] To facilitate a clearer understanding of the technical solution of this invention, the following will describe it in conjunction with specific thread sub-block sizes (such as N=3, 6, 10, etc.) and using the most common addition reduction operation as an example.

[0112] In one embodiment, when it is necessary to sum the data from three threads (denoted as v0, v1, and v2 respectively), N=3 is first calculated, which can be decomposed into 2... 1+1, that is, the number of main reduction steps k=1, the first value (i.e., the size of the main reduction thread subset) is 2, and the second value (i.e., the number of residual threads) is r=1. When the selected target reduction strategy is the identity element filling strategy, upward expansion is performed to find the smallest power of 2 that is not less than 3, that is, K=4, which means that the expanded thread subset includes 4 threads, the first three of which are valid threads (assuming their indices are 0, 1, and 2 respectively), and the last one is an invalid thread (assuming its index is 3). The data of the invalid thread with index 3 is forcibly replaced with the additive identity element 0 (i.e., e=0), which is equivalent to performing a standard 2-step XOR butterfly reduction (with step sizes of 1 and 2 respectively) on the 4 threads, and this can be implemented through the shfl_xor_sync instruction. Since the result of v0+v1+v2+0 remains unchanged, thread 0 (i.e., the thread with index 0) finally obtains an accurate 3-element reduction value.

[0113] If the selected target reduction strategy is the conditional downward reduction strategy, the execution process is split into two phases: a main reduction phase and a residual reduction phase. In the first phase (i.e., the main reduction phase), within the target thread sub-block (assuming it includes thread 0, thread 1, and thread 2), the system invokes the shfl_xor_sync instruction to perform one step of butterfly main reduction in the thread subset defined by the first value (i.e., thread 0 and thread 1). Specifically, when the step size is 1 (i.e., s=1), thread 0 and thread 1 are paired two by two based on the XOR topology to exchange data, and thread 0 acquires and accumulates the data of thread 1 to obtain the main reduction result, that is, v0'=v0+v1. At this point, the regular reduction within the subset has been efficiently completed within the closed boundary. In the second phase (i.e., the conditional downward reduction phase), the system switches to call the shfl_down_sync instruction with the unidirectional increment characteristic, and cooperates with the boundary condition t_offset+s<N to perform the accumulation and merging of residual data. When the communication step size jumps to 2 (i.e., s=2), thread 0 attempts to fetch residual data from thread 2. Since the addressing span of thread 0 is 0+2=2<3 (the condition is satisfied), the accumulation and merging is executed, obtaining v0''=v0'+v2=v0+v1+v2; while for thread 1, its addressing span is 1+2=3≥3 (out-of-bounds interception), no accumulation is performed. In the end, thread 0 holds the complete reduction result, which not only maximizes the efficient use of XOR instructions, but also avoids the pollution of data in the next tile.

[0114] In another embodiment, when it is necessary to sum the data (denoted as v0, v1, v2, v3, v4, v5 respectively) of 6 threads (assuming their indices are 0, 1, 2, 3, 4, 5 respectively), N=6 is first calculated, which can be decomposed into 2 2+2, meaning the first value is 4 and the second value is 2. When using the conditional down-reduction strategy, in the first phase (i.e., the main reduction phase), the system calls the shfl_xor_sync instruction to perform a two-step butterfly main reduction within the thread subset (i.e., threads 0 to 3) limited by the first value. When performing the reduction with step size s=1, threads 0 and 1 are paired and added together to get v0'=v0+v1, and threads 2 and 3 are paired and added together to get v2'=v2+v3; when performing the reduction with step size s=2, threads 0 and 2 are paired, swapped, and added together to get v0''=v0'+v2'=v0+v1+v2+v3. At this point, thread 0 has obtained the main reduction results of the first four threads.

[0115] In the second phase (i.e., the conditional down-reduction phase), the system calls the `shfl_down_sync` instruction for the remaining threads 4 and 5 and initiates multiple iterations with an increasing communication step size. During the local residual reduction with a step size of s=1, thread 4 retrieves data from thread 5, whose address span 4+1=5<6 (satisfying the condition), and performs accumulation to obtain the residual merged value v4'=v4+v5; however, thread 5, which is at the boundary, does not perform accumulation because its address span 5+1=6≥6 (out-of-bounds interception). Subsequently, during the global merge with a step size of s=4 (i.e., the length spanning the entire subset of the main reduce threads), thread 0 grabs the residual reduce result held by thread 4. Its addressing span 0+4=4<6 (satisfying the condition), and performs the final cumulative merge, i.e., v0'''=v0''+v4'=v0+v1+v2+v3+v4+v5. Other threads keep the original values ​​unchanged when judging that the addressing span is out of bounds, so thread 0 finally gets the complete 6-element reduce result.

[0116] Understandably, when N is not a power of 2, the identity fill strategy requires rounding up the size of the thread sub-blocks. This may reduce the number of tiles that can be accommodated within the warp. For example, 32 threads could originally accommodate 5 tiles of size 6, but after rounding up to 8, it can only accommodate 4 tiles of size 8, wasting the reduction capacity of 2 threads. To utilize warp thread resources more efficiently, this embodiment of the invention prioritizes a conditional downward reduction strategy, i.e., it does not perform upward rounding, but instead decomposes N into 2... k +r, execute the first k steps of butterfly reduction to complete 2 k The data from the r threads is reduced, and then the remaining data from the r threads is merged with the main reduction result through an additional shuffle operation.

[0117] The method provided in this invention reduces the latency of reducing non-power-of-two thread sub-blocks from 20-40 clock cycles to 3-12 clock cycles (depending on the thread sub-block size) compared to shared memory schemes, achieving a speedup of 2-6 times. Furthermore, the reduction operation is performed entirely within the register domain using shuffle instructions, without data leaving the register file, eliminating the overhead of shared memory writes, synchronization barriers, and re-reads. Multiple warps can perform reduction simultaneously without competing for shared memory bandwidth, improving the overall computational throughput of the system.

[0118] Furthermore, unlike related technologies that only support tile reduction at the power of 2 granularity, this invention supports tile reduction at any positive integer granularity, covering common non-power-of-2 scenarios in practical applications such as 3, 5, 6, 7, 10, 12, and 24. This invention also provides an adaptive dual-strategy approach. The identity fill strategy is simple to implement, using the `shfl_xor_sync` instruction to complete the power-of-2 extended reduction. The conditional down-reduction strategy uses the `shfl_down_sync` instruction in conjunction with boundary checks, resulting in higher thread utilization and automatically selecting the optimal strategy based on the specific scenario.

[0119] The arbitrary granularity thread sub-block reduction device provided by the present invention will be described below. The arbitrary granularity thread sub-block reduction device described below can be referred to in correspondence with the arbitrary granularity thread sub-block reduction method described above.

[0120] Based on any of the above embodiments Figure 5 This is a schematic diagram of the arbitrary granularity thread sub-block reduction device provided by the present invention, as shown below. Figure 5 As shown, the device includes: The parameter acquisition unit 510 is used to acquire the size parameter of the target thread sub-block and the target reduction operation type, wherein the size parameter is used to indicate the number of threads participating in the reduction operation; The decomposition decision unit 520 is used to decompose the size parameter into a first value and a second value when the size parameter is not a power of 2, and to determine a target reduction strategy based on the first value and the second value, wherein the first value is no more than the largest power of 2 of the size parameter, and the second value is the difference between the size parameter and the first value. The reduction execution unit 530 is used to call register-level thread communication instructions to perform multi-step inter-thread data exchange according to the target reduction strategy, and obtain a reduction result corresponding to the target reduction operation type. The register-level thread communication instructions are used to perform hardware-level aggregation of data in the registers of each thread in the target thread sub-block through shared memory transfer at the chip physical hardware level.

[0121] The apparatus provided in this invention obtains the size parameter and reduction operation type of the target thread sub-block, and decomposes it into a maximum power of 2 (not exceeding the parameter) and a remaining difference (i.e., a first value and a second value) when the size parameter is not a power of 2. This allows for precise matching of a suitable target reduction strategy based on the decomposition result. This mechanism enables the system to overcome the technical limitations of relying on specific sizes in related technologies when processing non-standard granularities, eliminating dependence on high-latency shared memory. Furthermore, it directly invokes efficient register-level thread communication instructions to complete multi-step inter-thread data exchange according to the determined target reduction strategy. This pure register-level data exchange method not only avoids complex dynamic mask calculations but also eliminates the additional latency and resource contention issues caused by shared memory read / write operations, ensuring efficient and continuous execution of the reduction operation.

[0122] Based on any of the above embodiments, when the target reduction strategy is a conditional downward reduction strategy, the reduction execution unit includes: The main reduction subunit is used to call a first type of register-level thread communication instruction to perform a main reduction operation within the thread subset defined by the first value in the target thread subblock, and obtain the main reduction result. The residual reduction subunit is used to call the second type of register-level thread communication instruction to perform a conditional downward reduction operation, so as to accumulate and merge the data of the residual thread corresponding to the second value and the main reduction result to obtain the reduction result.

[0123] Based on any of the above embodiments, the main reduction subunit is specifically used for: Based on the first value, a main reduction step sequence is generated that is applied within the subset of threads; According to the step size indicated by the main reduction step size sequence, the first type of register-level thread communication instructions control each thread in the thread subset to pair up and exchange data and perform reduction calculations, so that the data exchange range of each round is closed within the boundary of the thread subset, until the first thread in the thread subset obtains the main reduction result.

[0124] Based on any of the above embodiments, the residual reduction subunit is specifically used for: Multiple rounds of iterative communication are initiated according to an increasing communication step size to transmit the data of the residual thread level by level; In each round of iterative communication, the data to be transmitted by the target communication thread is obtained by calling the second type of register-level thread communication instruction. The target communication thread is the thread whose thread index is equal to the sum of the thread index of the currently executing thread and the current communication step size. Calculate the relative offset of the currently executing thread within the target thread sub-block, and sum the relative offset with the current communication step size to obtain the addressing span; Based on the comparison result between the addressing span and the size parameter, it is determined whether to accumulate and merge the transmitted data with the current data held by the currently executing thread, so as to gradually complete the data reduction between residual threads and the accumulation and merging with the main reduction result through multiple rounds of iterative communication.

[0125] Based on any of the above embodiments, the residual reduction subunit is specifically used for: If the addressing span is less than the size parameter, the transmitted data is accumulated and merged with the current data held by the currently executing thread; If the addressing span is greater than or equal to the size parameter, the accumulation and merging of the transmitted data is not performed, and the current data held by the currently executing thread remains unchanged.

[0126] Based on any of the above embodiments, when the target reduction strategy is an identity fill strategy, the reduction execution unit includes: An extended subunit is used to determine twice the value of the first value as the extended value, and to determine the corresponding reduction identity element according to the target reduction operation type; A filling subunit is used to fill data for invalid threads in the target thread subblock whose thread index is greater than or equal to the size parameter and less than the extended value, by applying the reduction identity element. The reduction subunit is used to call a first type of register-level thread communication instruction to perform a complete butterfly reduction communication on the filled thread subblock based on the extended value, and to determine the data obtained by the first thread after the communication is completed as the reduction result.

[0127] Based on any of the above embodiments, the extended subunit is specifically used for: When the target reduction operation type is addition or bitwise OR, the zero value is determined as the reduction identity. When the target reduction operation type is bitwise AND, the all-1 value is determined as the reduction identity; When the target reduction operation type is a maximum value operation, the negative infinity value is determined as the reduction identity element; When the target reduction operation type is a minimum value operation, the positive infinity value is determined as the reduction identity.

[0128] Based on any of the above embodiments, the device further includes a butterfly reduction unit, which is used for: When the size parameter is a power of 2, a butterfly reduction step size sequence is generated based on the size parameter; Based on the butterfly reduction step sequence, the first type of register-level thread communication instruction is invoked within the target thread sub-block to execute the complete butterfly reduction communication, so as to obtain the reduction result corresponding to the target reduction operation type.

[0129] Figure 6 This is a schematic diagram of the structure of the electronic device provided by the present invention, such as... Figure 6 As shown, the electronic device may include: a processor 610, a communication interface 620, a memory 630, and a communication bus 640, wherein the processor 610, the communication interface 620, and the memory 630 communicate with each other through the communication bus 640. The processor 610 can invoke logical instructions in the memory 630 to execute an arbitrary-granularity thread sub-block reduction method. This method includes: obtaining a size parameter of the target thread sub-block and a target reduction operation type, wherein the size parameter indicates the number of threads participating in the reduction operation; if the size parameter is not a power of 2, decomposing the size parameter into a first value and a second value, and determining a target reduction strategy based on the first value and the second value, wherein the first value is no more than a maximum power of 2 of the size parameter, and the second value is the difference between the size parameter and the first value; and invoking register-level thread communication instructions to perform multi-step inter-thread data exchange according to the target reduction strategy, obtaining a reduction result corresponding to the target reduction operation type. The register-level thread communication instructions are used to perform hardware-level aggregation of data in the registers of each thread in the target thread sub-block via shared memory transfer at the chip physical hardware level.

[0130] Furthermore, the logical instructions in the aforementioned memory 630 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to related technologies, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0131] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the arbitrary-granularity thread sub-block reduction method provided by the above methods. The method includes: obtaining the size parameter of the target thread sub-block and the target reduction operation type, wherein the size parameter is used to indicate the number of threads participating in the reduction operation; if the size parameter is not a power of 2, decomposing the size parameter into a first value and a second value, and determining a target reduction strategy based on the first value and the second value, wherein the first value is no more than the maximum power of 2 of the size parameter, and the second value is the difference between the size parameter and the first value; and calling register-level thread communication instructions to perform multi-step inter-thread data exchange according to the target reduction strategy to obtain a reduction result corresponding to the target reduction operation type. The register-level thread communication instructions are used to perform hardware-level aggregation of data in the registers of each thread in the target thread sub-block through shared memory transfer at the chip physical hardware level.

[0132] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon. When executed by a processor, the computer program implements a method for reducing thread sub-blocks of arbitrary granularity provided by the methods described above. The method includes: obtaining a size parameter of a target thread sub-block and a target reduction operation type, wherein the size parameter indicates the number of threads participating in the reduction operation; if the size parameter is not a power of 2, decomposing the size parameter into a first value and a second value, and determining a target reduction strategy based on the first value and the second value, wherein the first value is a power of 2 not exceeding the largest power of the size parameter, and the second value is the difference between the size parameter and the first value; and executing multi-step inter-thread data exchange by calling register-level thread communication instructions according to the target reduction strategy to obtain a reduction result corresponding to the target reduction operation type. The register-level thread communication instructions are used to perform hardware-level aggregation of data in the registers of each thread in the target thread sub-block through shared memory transfer at the chip physical hardware level.

[0133] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0134] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the parts that contribute to the related technology, can be embodied in the form of software products. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0135] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for reducing thread sub-blocks with arbitrary granularity, characterized in that, include: Obtain the size parameter of the target thread sub-block and the target reduction operation type, wherein the size parameter is used to indicate the number of threads participating in the reduction operation; When the size parameter is not a power of 2, the size parameter is decomposed into a first value and a second value, and a target reduction strategy is determined from a conditional downward reduction strategy and an identity fill strategy. The first value is a power of 2 that does not exceed the size parameter, the second value is the difference between the size parameter and the first value, the conditional downward reduction strategy is to first perform a main reduction on the subset of threads covered by the first value, and then perform a one-way accumulation and merging strategy on the residual threads corresponding to the second value, and the identity fill strategy is to use the characteristic of the first value to expand by multiples to construct a virtual boundary for identity fill. According to the target reduction strategy, based on the first value, or based on the first value and the second value, register-level thread communication instructions are invoked to perform multi-step inter-thread data exchange to obtain a reduction result corresponding to the target reduction operation type. The register-level thread communication instructions are used to perform hardware-level aggregation of data in the registers of each thread in the target thread sub-block through shared memory transfer at the chip physical hardware level.

2. The arbitrary granularity thread sub-block reduction method according to claim 1, characterized in that, The step of invoking register-level thread communication instructions to perform multi-step inter-thread data exchange based on the first value, or based on the first value and the second value, according to the target reduction strategy, to obtain a reduction result corresponding to the target reduction operation type, includes: When the target reduction strategy is the conditional downward reduction strategy, within the subset of threads defined by the first value in the target thread sub-block, the first type of register-level thread communication instruction is invoked to perform the main reduction operation and obtain the main reduction result; The second type of register-level thread communication instruction is invoked to perform a conditional downward reduction operation, so as to accumulate and merge the data of the residual thread corresponding to the second value and the main reduction result to obtain the reduction result.

3. The arbitrary granularity thread sub-block reduction method according to claim 2, characterized in that, The step of invoking the first type of register-level thread communication instruction to perform the main reduction operation and obtain the main reduction result includes: Based on the first value, a main reduction step sequence is generated that is applied within the subset of threads; According to the step size indicated by the main reduction step size sequence, the first type of register-level thread communication instructions control each thread in the thread subset to pair up and exchange data and perform reduction calculations, so that the data exchange range of each round is closed within the boundary of the thread subset, until the first thread in the thread subset obtains the main reduction result.

4. The arbitrary granularity thread sub-block reduction method according to claim 2, characterized in that, The invocation of the second type of register-level thread communication instruction to perform a conditional downward reduction operation, in order to accumulate and merge the data of the residual thread corresponding to the second value and the main reduction result, includes: Multiple rounds of iterative communication are initiated according to an increasing communication step size to transmit the data of the residual thread level by level; In each round of iterative communication, the data to be transmitted by the target communication thread is obtained by calling the second type of register-level thread communication instruction. The target communication thread is the thread whose thread index is equal to the sum of the thread index of the currently executing thread and the current communication step size. Calculate the relative offset of the currently executing thread within the target thread sub-block, and sum the relative offset with the current communication step size to obtain the addressing span; Based on the comparison result between the addressing span and the size parameter, it is determined whether to accumulate and merge the transmitted data with the current data held by the currently executing thread, so as to gradually complete the data reduction between residual threads and the accumulation and merging with the main reduction result through multiple rounds of iterative communication.

5. The arbitrary granularity thread sub-block reduction method according to claim 4, characterized in that, The step of determining whether to accumulate and merge the transmitted data with the current data held by the currently executing thread based on the comparison result of the addressing span and the size parameter includes: If the addressing span is less than the size parameter, the transmitted data is accumulated and merged with the current data held by the currently executing thread; If the addressing span is greater than or equal to the size parameter, the accumulation and merging of the transmitted data is not performed, and the current data held by the currently executing thread remains unchanged.

6. The arbitrary granularity thread sub-block reduction method according to claim 1, characterized in that, The step of invoking register-level thread communication instructions to perform multi-step inter-thread data exchange based on the first value, or based on the first value and the second value, according to the target reduction strategy, to obtain a reduction result corresponding to the target reduction operation type, includes: When the target reduction strategy is the identity filling strategy, twice the value of the first value is determined as the extended value, and the corresponding reduction identity is determined according to the target reduction operation type. For invalid threads in the target thread sub-block whose thread index is greater than or equal to the size parameter and less than the extended value, the reduction identity is applied to fill in the data. Based on the extended value, the first type of register-level thread communication instruction is invoked to perform a complete butterfly reduction communication on the filled thread sub-block, and the data obtained by the first thread after the communication is completed is determined as the reduction result.

7. The arbitrary granularity thread sub-block reduction method according to claim 6, characterized in that, The step of determining the corresponding reduction identity element based on the target reduction operation type includes: When the target reduction operation type is addition or bitwise OR, the zero value is determined as the reduction identity. When the target reduction operation type is bitwise AND, the all-1 value is determined as the reduction identity; When the target reduction operation type is a maximum value operation, the negative infinity value is determined as the reduction identity element; When the target reduction operation type is a minimum value operation, the positive infinity value is determined as the reduction identity.

8. The arbitrary granularity thread sub-block reduction method according to any one of claims 1 to 7, characterized in that, Also includes: When the size parameter is a power of 2, a butterfly reduction step size sequence is generated based on the size parameter; Based on the butterfly reduction step sequence, the first type of register-level thread communication instruction is invoked within the target thread sub-block to execute the complete butterfly reduction communication, so as to obtain the reduction result corresponding to the target reduction operation type.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the arbitrary granularity thread subblock reduction method as described in any one of claims 1 to 8.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the arbitrary granularity thread subblock reduction method as described in any one of claims 1 to 8.

Images