A thread bundle construction apparatus, method and processor

Abstract

The application discloses a kind of thread bundle construction device, method and processor, thread block dimension register and mode configuration register are respectively stored thread block dimension information and mode configuration information in thread bundle construction request;Loop controller responds thread bundle construction request, reads thread block dimension information and mode configuration information to calculate total cycle number, and controls thread bundle calculation construction array to execute and stop thread coordinate calculation;Thread bundle calculation construction array is under the control of loop controller, according to mode configuration information, enable specified thread bundle calculation construction unit, to execute thread coordinate calculation according to clock cycle loop, and output the thread coordinate calculated;Wherein, specified thread bundle calculation construction unit is according to thread block dimension information in the process of thread coordinate calculation Boundary judgment and thread coordinate adjustment are carried out.The application can support linear and zig-zag thread arrangement mode, and realize continuous coordinate calculation across Z axis, to further improve hardware utilization efficiency.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence chips, and in particular to a thread bundle construction device, method and processor. BACKGROUND

[0002] Before an artificial intelligence chip executes a computing task, a set of threads (i.e., a thread block) needs to be grouped to construct a thread bundle (usually 32 threads in a group), and each thread is assigned a unique coordinate (i.e., a thread ID) to achieve the mapping between the computing task and the hardware execution resources. The existing thread bundle construction device has the following limitations when processing irregular thread blocks or non-continuous thread arrangement modes:

[0003] (1) When the size of the thread block in a certain dimension is not a fixed value, or the XY plane size is not an integer multiple of 32, thread coordinate calculation is difficult, and since Z-axis continuous coordinate calculation cannot be achieved, incomplete thread bundles (thread number < 32) are easily generated, reducing hardware utilization efficiency.

[0004] (2) Only linear arrangement is supported, Zig-Zag arrangement cannot be achieved, and it is difficult to meet the needs of computing scenarios such as block data access required by texture sampling. SUMMARY

[0005] The purpose of the embodiments of the present application is to provide a thread bundle construction device, method and processor, which supports linear and Zig-Zag thread arrangement modes through a configurable thread bundle construction array, can efficiently and continuously complete multi-dimensional thread coordinate calculation and boundary processing in each clock cycle, and thus realizes continuous coordinate calculation across the Z-axis.

[0006] The first embodiment of the first aspect of the present application provides a thread bundle construction device, comprising:

[0007] A thread block dimension register for storing thread block dimension information carried in a thread bundle construction request;

[0008] A mode configuration register for storing mode configuration information carried in the thread bundle construction request;

[0009] A loop controller for reading the thread block dimension information and the mode configuration information in response to the thread bundle construction request, calculating the total number of loops, and controlling the thread bundle calculation construction array to perform thread coordinate calculation after entering the thread bundle construction state;

[0010] The thread bundle computation construction array is configured to enable a specified thread bundle computation construction unit to perform thread coordinate computation in a clock cycle loop and output the computed thread coordinates under the control of the loop controller according to the mode configuration information; wherein the specified thread bundle computation construction unit performs boundary judgment and thread coordinate adjustment according to the thread block dimension information in the thread coordinate computation process;

[0011] The loop controller is further configured to control the specified thread bundle computation construction unit to stop thread coordinate computation when the number of loops of the specified thread bundle computation construction unit reaches the total number of loops.

[0012] Optionally, the thread bundle construction request is initiated by an external task allocation unit.

[0013] Optionally, the mode configuration information includes a thread arrangement mode and a number of arithmetic logic units called per clock cycle; wherein the thread arrangement mode includes a linear mode, an N-shaped mode or a Z-shaped mode; and the number of arithmetic logic units is used to represent a computation granularity.

[0014] Optionally, the thread bundle computation construction array includes M1 thread bundle computation construction units of the linear mode, M2 thread bundle computation construction units of the N-shaped mode, M3 thread bundle computation construction units of the Z-shaped mode and a multiplexer; wherein the multiplexer is used to enable a specified thread bundle computation construction unit and output the computed thread coordinates according to the mode configuration information; M1≥1; M2≥1; M3≥1.

[0015] Optionally, the enabling of the specified thread bundle computation construction unit to perform thread coordinate computation in a clock cycle loop includes:

[0016] The specified thread bundle computation construction unit computes all thread coordinates in the current clock cycle based on the thread coordinate initial value of the current clock cycle and the incremental rule defined by the corresponding thread arrangement mode;

[0017] In the computation process of each thread coordinate, the dimension coordinate values of the currently computed thread coordinate are compared with the dimension sizes determined by the thread block dimension information to obtain corresponding comparison results;

[0018] If the comparison result is that no dimension coordinate value is out of bounds, the currently computed thread coordinate is maintained;

[0019] If the comparison result is that there is a dimension coordinate value out of bounds, the out-of-bounds dimension coordinate value is reset to an initial value and a carry is performed to a higher dimension coordinate value;

[0020] The last thread coordinate calculated in the current clock cycle is used as the initial thread coordinate value for the next clock cycle, until the total number of loops is reached.

[0021] Optionally, resetting the out-of-bounds dimension coordinate values ​​to their initial values ​​and carrying over to higher-dimensional coordinate values ​​includes:

[0022] Reset the out-of-bounds dimension coordinates to 0, and increment the coordinates of the next higher dimension by 1;

[0023] If the coordinate value of the next higher dimension exceeds the limit after being incremented by 1, the coordinate value of the next higher dimension is reset to 0, and the process continues to increment the coordinate value of the next higher dimension by 1.

[0024] Optionally, the reuse of arithmetic logic units is supported among the thread bundle computation building units.

[0025] Optionally, when the thread arrangement mode is an N-shaped mode or a Z-shaped mode, the dimensions corresponding to the X and Y dimensions in the thread block dimension information are integer multiples of the dimensions of the corresponding thread arrangement granularity in the X and Y dimensions, respectively.

[0026] Optionally, the calculated thread coordinates are output to an external thread bundle coordinate buffer.

[0027] A second aspect of the present invention provides a thread bundle construction method, applicable to the thread bundle construction apparatus described in any of the first aspects above, the method comprising:

[0028] The thread block dimension information carried in the thread bundle construction request is stored in the thread block dimension register;

[0029] The mode configuration information carried in the thread bundle construction request is stored in the mode configuration register;

[0030] The loop controller responds to the thread bundle construction request, enters the thread bundle construction state, reads the thread block dimension information and the mode configuration information, calculates the total number of loops, and controls the thread bundle calculation and construction array to perform thread coordinate calculation.

[0031] Under the control of the loop controller, the array constructed by thread beam computation enables a designated thread beam computation construction unit according to the mode configuration information to perform thread coordinate calculation cyclically according to the clock cycle and output the calculated thread coordinates; wherein, the designated thread beam computation construction unit performs boundary judgment and thread coordinate adjustment according to the thread block dimension information during the thread coordinate calculation process;

[0032] When the loop controller controls the designated thread bundle calculation building unit to stop calculating thread coordinates when the number of loops in the designated thread bundle calculation building unit reaches the total number of loops, the loop controller controls the designated thread bundle calculation building unit to stop calculating thread coordinates.

[0033] A third aspect of the present invention provides a processor including the thread bundle construction apparatus described in any of the first aspects above.

[0034] Compared with the prior art, the embodiments of the present invention provide a thread bundle construction device, method and processor, which have the following beneficial effects: The embodiments of the present invention support multiple linear and Zig-Zag thread arrangement modes through a configurable thread bundle calculation and construction array, and can efficiently and flexibly handle thread blocks of different dimensions and sizes; at the same time, the specified thread bundle calculation and construction unit performs thread coordinate calculation cyclically according to the clock cycle, which can realize continuous iteration of thread coordinates, and perform boundary judgment and thread coordinate adjustment according to the thread block dimension information during the thread coordinate calculation process, thereby realizing continuous coordinate calculation across the Z-axis, so as to build as many complete thread bundles as possible, thereby improving hardware utilization efficiency. Attached Figure Description

[0035] Figure 1 This is a schematic diagram of an embodiment of thread bundle partitioning provided by related technologies;

[0036] Figure 2 This is a schematic diagram of another embodiment of thread bundle partitioning provided by related technologies;

[0037] Figure 3(a) is a schematic diagram of an embodiment of the linear arrangement pattern provided by the present invention;

[0038] Figure 3(b) is a schematic diagram of an embodiment of the N-type Zig-Zag mode provided by the present invention;

[0039] Figure 3(c) is a schematic diagram of an embodiment of the Z-type Zig-Zag mode provided by the present invention;

[0040] Figure 4 This is a schematic diagram of an embodiment of the thread beam construction device provided by the present invention;

[0041] Figure 5 This is a schematic diagram of the structure of an embodiment of the thread beam calculation building unit provided by the present invention;

[0042] Figure 6 This is a schematic diagram of another embodiment of the thread beam computing building unit provided by the present invention;

[0043] Figure 7 This is a schematic diagram of an embodiment of thread X-coordinate calculation provided by the present invention;

[0044] Figure 8 This is a flowchart illustrating an embodiment of the thread bundle construction method provided by the present invention. Detailed Implementation

[0045] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0046] The artificial intelligence processor involved in this invention can be any one of CPU (Central Processing Unit), GPU (Graphics Processing Unit), TPU (Tensor Processing Unit), NPU (Neural Network Processing Unit), DPU (Deep Learning Processing Unit), APU (Accelerated Processing Unit), and GPGPU (General-Purpose computing on Graphics Processing Unit), depending on its application to a specific product or technology in the embodiments of this invention.

[0047] It should be noted that, as Figure 1 The diagram shown is a schematic representation of an embodiment of thread bundle partitioning provided by related technologies. Figure 1 In this model, each small cube represents a thread, and a collection of threads is called a threadblock. Each thread has corresponding coordinates, and 32 threads form a warp. In the existing programming model of GPGPU, the coordinates of each thread need to be calculated before the computation task can begin.

[0048] On the one hand, related technologies typically assume that the number of threads in each direction (X, Y, Z dimensions) of a thread block is a fixed value. If the size in a certain dimension is not a fixed value, it is difficult to construct a thread bundle. Furthermore, because related technologies cannot calculate thread coordinates continuously along the Z-axis, when the number of threads in the XY plane is not equal to 32×N (N is a positive integer), an incomplete thread bundle (number of threads < 32) will be generated, such as... Figure 2 The diagram shown is a schematic diagram of another embodiment of thread bundle partitioning provided by related technologies.

[0049] On the other hand, the related technologies only support linear arrangements (as shown in Figure 3(a)). However, in computational scenarios such as texture sampling, where texture data is typically stored in the form of pixel blocks, the threads within a thread block should also be arranged in a similar tile pattern, such as the N-shaped (N-type Zig-Zag) arrangement shown in Figure 3(b) or the Z-shaped (Z-type Zig-Zag) arrangement shown in Figure 3(c). The lack of a Zig-Zag arrangement in the related technologies limits their applicability and performance in different computational tasks.

[0050] See Figure 4 This is a schematic diagram of an embodiment of the thread bundle construction device provided by the present invention.

[0051] To address the aforementioned technical problems, a first aspect of the present invention provides a thread bundle construction apparatus, comprising:

[0052] Thread block dimension register 100 is used to store thread block dimension information carried in the thread bundle construction request;

[0053] Mode configuration register 200 is used to store the mode configuration information carried in the thread bundle construction request;

[0054] The loop controller 300 is used to respond to the thread bundle construction request, and after entering the thread bundle construction state, read the thread block dimension information and the mode configuration information, calculate the total number of loops, and control the thread bundle calculation and construction array 400 to perform thread coordinate calculation.

[0055] The thread beam calculation construction array 400 is used, under the control of the loop controller 300, to enable a specified thread beam calculation construction unit according to the mode configuration information, so as to perform thread coordinate calculation cyclically according to the clock cycle and output the calculated thread coordinates; wherein, the specified thread beam calculation construction unit performs boundary judgment and thread coordinate adjustment according to the thread block dimension information during the thread coordinate calculation process;

[0056] The loop controller 300 is further configured to control the designated thread bundle calculation building unit to stop thread coordinate calculation when the number of loops of the designated thread bundle calculation building unit reaches the total number of loops.

[0057] It should be noted that, as Figure 4 As shown, the thread bundle construction device mainly consists of a thread block dimension register 100, a mode configuration register 200, a loop controller 300, and a thread bundle computation and construction array 400. The working principle of the thread bundle construction device is as follows:

[0058] The external task allocation unit initiates a thread bundle construction request to the thread bundle construction device, and stores the thread block dimension information and related mode configuration information carried in the thread bundle construction request into the thread block dimension register 100 and the mode configuration register 200, respectively.

[0059] The thread block dimension information includes the size of the thread block to be divided in each dimension (i.e. the number of threads in each dimension, such as thread_block_x_size, thread_block_y_size, thread_block_z_size), which is used by the thread bundle construction computing unit to perform boundary judgment during the calculation process, and to provide the loop controller 300 with the total number of loops to calculate.

[0060] The mode configuration information includes: thread arrangement mode and the number of Arithmetic Logic Units (ALUs) called per clock cycle; among which, the thread arrangement mode includes: linear mode, N-shaped mode (N-type Zig-Zag mode) or Z-shaped mode (Z-type Zig-Zag mode); the number of arithmetic logic units is used to characterize the computation granularity (i.e. the number of thread coordinates calculated per cycle); in other words, the number of ALUs directly determines the number of thread coordinates that the thread bundle computation building unit can calculate per clock cycle.

[0061] The loop controller 300 contains a state machine. Upon receiving a thread bundle construction request and entering the thread bundle construction state, it inputs thread block dimension information through the thread block dimension register 100 to obtain the total number of threads and inputs the number of ALUs through the mode configuration register 200. Further, it calculates the total number of loops by dividing the total number of threads by the number of ALUs and rounding up. The loop controller 300 also controls designated thread bundle computation construction units in the thread bundle computation construction array 400 to perform thread coordinate calculations, and when the number of loops reaches the total number of loops, it controls the designated thread bundle computation construction unit to stop computation and return to the idle state.

[0062] The warp computing construction array 400 consists of multiple configurable warp computing units (warp computing construction units in one-dimensional / two-dimensional / three-dimensional linear mode, two-dimensional / three-dimensional N-type Zig-Zag mode warp computing construction units, and two-dimensional / three-dimensional Z-type Zig-Zag mode warp computing construction units) and a multiplexer. Each warp construction computing unit adapts to different mode configuration information. The warp computing construction array 400 automatically enables the matching specified warp computing construction unit through the multiplexer according to the mode configuration information transmitted by the mode configuration register 200; under the control of the loop controller 300, the specified warp computing construction unit performs loop calculations with the thread coordinates of the number of ALUs calculated per clock cycle. During the calculation process, according to the thread block dimension information in the thread block dimension register 100, boundary judgment and coordinate adjustment are performed in real time.

[0063] Exemplarily, when calculating the X-axis coordinate of a three-dimensional thread block, if the current X-axis coordinate is the size W of the X dimension, the current X-axis coordinate is reset to 0 (reset to 0), and the Y-axis coordinate is triggered to carry (add 1); if the Y-axis coordinate is the size H of the Y dimension, the Y-axis coordinate is reset to 0, and the Z-axis coordinate is triggered to carry (add 1), ensuring that the thread coordinate values (x, y, z) are always within the valid range of "0 ≤ x < W, 0 ≤ y < H, 0 ≤ z < D", avoiding out-of-bounds errors, and enabling continuous coordinate calculations across the Z-axis to construct as many complete warps as possible; where D is the size of the Z dimension.

[0064] In addition, the specified warp computing construction unit is also used to output the calculated thread coordinates to the thread coordinate buffer outside the warp construction device for storage, to be used for subsequent operation scheduling.

[0065] In an optional embodiment, the mode configuration information includes: the thread arrangement mode and the number of arithmetic logic units called per clock cycle; where the thread arrangement mode includes: linear mode, N-type mode or Z-type mode; the number of arithmetic logic units is used to characterize the calculation granularity.

[0066] Furthermore, when the thread arrangement mode is N-type mode or Z-type mode, the sizes corresponding to the X dimension and Y dimension in the thread block dimension information have an integer multiple relationship with the sizes of the corresponding thread arrangement granularity in the X dimension and Y dimension respectively.

[0067] It should be noted that the N-type mode includes the conventional N-type Zig-Zag mode or the reverse N-type Zig-Zag mode; the Z-type mode includes the conventional Z-type Zig-Zag mode or the reverse Z-type Zig-Zag mode.

[0068] The thread arrangement granularity corresponding to the N-shaped pattern is 2×k (threads are arranged in small blocks of 2×k units). As shown in Figure 3(b), the conventional N-shaped Zig-Zag pattern has a thread arrangement granularity of 2×2. The calculation unit is constructed by using 4 ALUs (calculating 4 thread coordinates per clock cycle) and the thread bundle of the conventional N-shaped Zig-Zag pattern for coordinate calculation; where k≥2.

[0069] The thread arrangement granularity corresponding to the Z-shaped pattern is m×2 (threads are arranged in small blocks of m×2). As shown in Figure 3(c), the inverse Z-shaped Zig-Zag pattern has a thread arrangement granularity of 2×2. The calculation unit is constructed by using a thread bundle with an ALU of 4 (calculating 4 thread coordinates per clock cycle) and the inverse Z-shaped Zig-Zag pattern for coordinate calculation; where m≥2.

[0070] Furthermore, in Zig-Zag mode, the thread arrangement granularity and the number of thread coordinates calculated per clock cycle correspond to the number of ALUs called by the thread bundle computation building unit in Zig-Zag.

[0071] Specifically, for the Zig-Zag pattern, the dimensions of the thread block in the X and Y dimensions are subject to the following constraints: the number of threads in the thread block in the X and Y directions must be divisible by the corresponding thread arrangement granularity's X and Y dimensions, respectively, and the total number of threads in the thread block in the XY plane must be even. Since the thread arrangement granularity determines the number of ALUs called in the Zig-Zag thread bundle computation building unit, the total number of threads in the thread block in the XY plane must also be divisible by the number of ALUs called.

[0072] In an optional embodiment, the thread bundle computation building array 400 includes: M1 linear mode thread bundle computation building units, M2 N-shaped mode thread bundle computation building units, M3 Z-shaped mode thread bundle computation building units, and a multiplexer; wherein, the multiplexer is used to enable a specified thread bundle computation building unit and output the calculated thread coordinates according to the mode configuration information; M1≥1; M2≥1; M3≥1.

[0073] Furthermore, the reuse of arithmetic logic units is supported among the various thread bundle computation building units.

[0074] It should be noted that, as Figure 4As shown, the thread beam computation building array 400 includes three types of thread beam computation building units (linear mode, N-type Zig-Zag mode, and Z-type Zig-Zag mode). Each thread beam computation building unit consists of several ALUs, responsible for performing thread coordinate calculation tasks; the number of ALUs directly determines the number of thread coordinates that the unit can calculate per clock cycle. During thread beam construction, the corresponding thread beam computation building unit is activated according to the working mode selected based on the mode configuration information in the mode configuration register 200, and after performing coordinate calculations, the results are stored in the thread coordinate buffer.

[0075] In this embodiment of the invention, "configurable" means that the thread bundle computation building array 400 can determine the number of ALUs (i.e. the number of thread coordinates calculated per clock cycle) of different thread bundle computation building units during the design phase; in other words, the thread bundle computation building array 400 can pre-integrate multiple different types and multiple different computation granularities of thread bundle computation building units during the design phase, so as to select them through mode configuration information.

[0076] Specifically, if the thread bundle computation building unit in linear mode contains j ALUs, then j thread coordinates will be calculated each clock cycle; if the threads in the thread block are arranged non-linearly and are arranged in small blocks of 2×k or m×2, then the thread bundle computation building unit will also perform coordinate calculations with a computation granularity of 2×k or m×2, that is, 2×k or m×2 ALUs will be called for calculation each clock cycle.

[0077] The values ​​of j, k, and m mentioned above are determined and fixed during the design phase of each thread-beam computing building unit. However, different thread-beam computing building units can reuse ALUs. For example, a linear-mode thread-beam computing building unit configured with 4 ALUs (j=4) and an N-type Zig-Zag-mode thread-beam computing building unit configured with 6 ALUs (k=3, 2×3=6) can share all or part of the computing components in the 4 ALUs, thereby reducing the consumption of hardware resources.

[0078] like Figure 5 The diagram shown is a structural schematic of an embodiment of the thread beam computing building unit provided by the present invention. Figure 5 It includes four ALUs (i.e., coordinate calculation granularity is 4), namely ALU0, ALU1, ALU2, and ALU3; the mode configuration register is used to determine the key parameters of the specified thread bundle calculation building unit, such as the number of ALUs and the incrementing rule defined by the thread arrangement mode; the thread block dimension register is used to store thread block dimension information, providing a basis for boundary judgment during the coordinate calculation process. In linear mode, Figure 5Each ALU in the system receives the X, Y, and Z coordinates of the previous thread, processes them internally, outputs the X, Y, and Z coordinates of the current thread, and passes them to the next ALU, thereby enabling continuous calculation of coordinates for multiple threads.

[0079] like Figure 6 The diagram shows a structural schematic of another embodiment of the thread bundle computation building unit provided by this invention. Each ALU includes at least three sets of adders (corresponding to the increment operation of coordinates in the X, Y, and Z dimensions, respectively), a comparator, and a selector for each dimension. The comparator is used to determine whether the coordinates in the corresponding dimension are out of bounds, such as whether the X coordinate is equal to the size tg_size_x of the X dimension. The selector is used to select "increment coordinates by 1" or "reset coordinates to 0" based on the comparison result. The activation of each computation component of the ALU is determined by an incremental rule based on the corresponding thread arrangement mode, and the coordinate transfer between ALUs also follows an incremental rule. Clearly, the linear mode transfers coordinates forward one ALU at a time, while the Zig-Zag mode may also involve transfers across ALUs.

[0080] Furthermore, whether each ALU uses the corresponding dimension's computational components for computation is determined by the mode configuration register and the thread block dimension register. For simplified illustration, in... Figure 6 The diagram only schematically shows the adders and selectors for each dimension of the six ALUs, while the comparators are not explicitly shown.

[0081] In this embodiment of the invention, ALU reuse is performed between scenarios with the same dimension but different computational granularities. The following are some specific embodiments of ALU reuse between thread bundle computation building units for reference:

[0082] Example 1: Figure 7 The diagram shown is a schematic representation of an embodiment of thread X-coordinate calculation provided by the present invention. Figure 7 (a) A thread bundle computation building block for calculating 4 thread coordinates per cycle in a one-dimensional linear pattern. Figure 7 (b) A thread bundle computation building block for calculating 6 thread coordinates per cycle in a one-dimensional linear pattern. This is because both use X-dimensional linear coordinate computation links, and Figure 7 The calculated trajectory (incrementing rule) of the four thread coordinates in (a) and Figure 7 In (b), the coordinates of the first four threads are completely identical, therefore Figure 7 (a) Can be directly reused Figure 7(b) All computational components of the first four ALUs. In this reuse scenario, each ALU only activates the computational components in the X dimension (i.e., only the adders, comparators, and selectors in the X dimension participate in the operation). Of course, in the two-dimensional linear mode, each enabled ALU activates the computational components in the Y dimension to perform the Y-coordinate increment operation whenever the X coordinate goes out of bounds.

[0083] Figure 7 The X-coordinate calculation process is as follows: The adder increments the X-coordinate of the previous thread by 1, generating candidate increasing coordinates (e.g., performing p0_x+1 on p0_x yields a candidate value for p1_x); the comparator compares the current candidate increasing coordinate with the thread block's X-dimensional size tg_size_x, outputting a "whether it's out of bounds" logic signal; the selector performs a binary choice based on the comparator's logic signal: if the current candidate increasing coordinate equals tg_size_x (out of bounds), it outputs 0 (X-coordinate reset to its initial value); otherwise, it outputs the candidate increasing coordinate (not out of bounds, X-coordinate increases normally). During the coordinate calculation, calculate_num controls the total number of loops for the thread bundle calculation building unit; calculation stops when the loop count reaches calculate_num. To ensure the continuity of multi-cycle calculations, at the end of each cycle, the clock signal will trigger a D-type flip-flop to store the X coordinate of the last ALU output of the current cycle (denoted as pixel_offset_x_old). pixel_offset_x_old will be used as the initial X coordinate for the next cycle.

[0084] Example 2: For Zig-Zag patterns such as Z-type and N-type, the reusability of the ALU is limited by the differences in computational trajectories. For portions with completely identical computational trajectories, all computational components of the ALU can be reused, while for portions with differing computational trajectories, typically only some computational components of the ALU (such as adders) are reused. If... Figure 6 This is thread bundle computation building unit 1 in the inverse Z-shaped Zig-Zag pattern, with a thread arrangement granularity of 3×2 and 6 ALUs (ALU0 to ALU5 in sequence). When the mode configuration register selects to enable thread bundle computation building unit 2 in the inverse Z-shaped Zig-Zag pattern with a thread arrangement granularity of 2×2, because the computation trajectory of "first thread coordinate -> second thread coordinate" is consistent in the 2×2 and 3×2 inverse Z-shaped Zig-Zag patterns, while the computation trajectories of "last coordinate of the previous cycle -> first thread coordinate" and the subsequent "second thread coordinate -> third thread coordinate -> fourth thread coordinate" are different, thread bundle computation building unit 2 can reuse all the computation components of ALU1 in thread bundle computation building unit 1, as well as the required computation components (such as adders) of ALU0, ALU2 and ALU3 corresponding to the dimensions.

[0085] Of course, if Figure 6 This is thread bundle computation building unit 3 in linear mode, with a thread arrangement granularity of 6×1. When the mode configuration register selects to enable thread bundle computation building unit 4 in linear mode with a thread arrangement granularity of 4×1, thread bundle computation building unit 4 can reuse all computation components of ALU0~ALU3 in thread bundle computation building unit 3 because the computation trajectory of thread coordinates in linear mode thread bundle computation building unit 4 is consistent with the computation trajectory of the first 4 thread coordinates in thread bundle computation building unit 3.

[0086] In an optional embodiment, enabling a specified thread beam computation building unit to perform thread coordinate calculations cyclically according to a clock cycle includes:

[0087] By specifying the thread bundle calculation building unit, based on the initial value of the thread coordinates in the current clock cycle, and according to the incrementing rule defined by the corresponding thread arrangement pattern, the coordinates of all threads in the current clock cycle are calculated.

[0088] During the calculation of each thread coordinate, the coordinate values ​​of each dimension of the currently calculated thread coordinate are compared with the dimension size determined by the thread block dimension information to obtain the corresponding comparison result.

[0089] If the comparison result is that the coordinates of the dimensionless coordinates are out of bounds, then the currently calculated thread coordinates are maintained.

[0090] If the comparison result indicates that a dimension coordinate value is out of bounds, then the out-of-bounds dimension coordinate value is reset to its initial value and carried over to the coordinate value of the higher dimension.

[0091] The last thread coordinate calculated in the current clock cycle is used as the initial thread coordinate value for the next clock cycle, until the total number of loops is reached.

[0092] Furthermore, resetting the out-of-bounds dimensional coordinate values ​​to their initial values ​​and carrying over to higher-dimensional coordinate values ​​includes:

[0093] Reset the out-of-bounds dimension coordinates to 0, and increment the coordinates of the next higher dimension by 1;

[0094] If the coordinate value of the next higher dimension exceeds the limit after being incremented by 1, the coordinate value of the next higher dimension is reset to 0, and the process continues to increment the coordinate value of the next higher dimension by 1.

[0095] It should be noted that, in the thread coordinate calculation of this embodiment of the invention, the cyclic logic of "thread coordinate initial value setting - batch calculation within clock cycle - boundary judgment and coordinate adjustment - thread coordinate initial value update" is followed to ensure the continuity and accuracy of thread coordinate calculation under different dimensions and different arrangement modes.

[0096] Taking a three-dimensional thread block as an example, the initial value of the thread coordinates of the specified thread bundle calculation building unit is (0,0,0) in the first loop. Subsequent loops directly use the last thread coordinate calculated in the previous clock cycle as the initial value of the thread coordinates to achieve continuous iterative calculation of the thread coordinates.

[0097] This invention supports three types of thread arrangement modes, and the coordinate increment rules for each mode are as follows:

[0098] The linear pattern increment rule: Taking Figure 3(a) as an example, the threads are generated linearly in the X→Y→Z dimension (that is, the X-axis is incremented first, the Y-axis is incremented second, and the Z-axis is incremented last).

[0099] The incremental rule of the N-type Zig-Zag pattern: Taking Figure 3(b) as an example, according to... → → → The conventional "N"-shaped path increases incrementally, that is, by first expanding along the Y-axis and then extending along the X-axis, a "zigzag" arrangement is formed in the local area.

[0100] The incremental rule of the Z-type Zig-Zag pattern: Taking Figure 3(c) as an example, according to... → → → The reverse "Z" shaped path increases, that is, by first expanding along the X-axis and then extending along the Y-axis, another type of "zigzag" arrangement is formed in the local area.

[0101] The specified thread bundle computation building unit will synchronously complete the parallel computation of multiple thread coordinates within one clock cycle, based on its own ALU count.

[0102] During the calculation of each thread coordinate, the calculated coordinate values ​​of each dimension are compared one by one with the corresponding dimension boundary values ​​to obtain the comparison result for each dimension (out of bounds or not out of bounds). If the comparison results for all dimensions are within bounds, the currently calculated thread coordinates are maintained and can be directly output to the thread coordinate buffer as valid calculation results. If the coordinate value of a certain dimension is out of bounds (e.g., the X-axis coordinate value equals thread_block_x_size), then the operation of "resetting the out-of-bounds dimension coordinate + carrying over the higher dimension coordinate" is performed, specifically as follows:

[0103] Reset the out-of-bounds dimension coordinates to their initial values ​​(usually 0), and increment the coordinates of the next higher dimension by 1 (e.g., if the X-axis is out of bounds, reset the X-axis to 0 and increment the Y-axis by 1).

[0104] If the coordinate value of a higher-level dimension also exceeds the limit after being incremented by 1 (e.g., the Y-axis value equals thread_block_y_size after being incremented by 1), then the coordinate value of that higher-level dimension is reset to 0, and the coordinate value of the next higher-level dimension is incremented by 1 (e.g., the Y-axis is reset to 0, and the Z-axis is incremented by 1), and so on, until the coordinate values ​​of all dimensions are within the valid range.

[0105] Taking thread_block_x_size=8 as an example, when calculating the coordinates of thread7 (index starts from 0, corresponding to the 8th position on the X-axis), the X-axis coordinate of thread7 is 7, which is less than thread_block_x_size. At this time, the comparison result is that the X-axis is not out of bounds. However, when calculating the coordinates of thread8, the X-axis coordinate of thread8 is 8, which is equal to thread_block_x_size. At this time, the comparison result is that the X-axis is out of bounds, and the X-axis coordinate needs to be reset to 0, while the Y-axis coordinate is incremented by 1.

[0106] In summary, the thread bundle construction apparatus provided by the embodiments of the present invention has the following beneficial effects:

[0107] (1) It is highly flexible and can adapt to thread blocks of different sizes and dimensions; for three-dimensional thread blocks, it can perform continuous calculation of thread coordinates across the Z-axis.

[0108] (2) The logic is simple and the implementation cost is small.

[0109] (3) Supports multiple thread arrangement modes: It not only supports linear construction of thread bundles, but also adapts to special thread arrangements such as N-type Zig-Zag and Z-type Zig-Zag.

[0110] (4) It has high computational efficiency and can support the calculation of different number of thread coordinates per clock cycle by flexibly configuring the number of ALUs (such as 1, 2, 4, 8, etc.), thereby dynamically expanding according to actual hardware resources and performance requirements.

[0111] See Figure 8 This is a flowchart illustrating an embodiment of the thread bundle construction method provided by the present invention.

[0112] A second aspect of the present invention provides a thread bundle construction method, applicable to the thread bundle construction apparatus described in any of the first aspects above. The method includes steps S1 to S5, as follows:

[0113] Step S1: Store the thread block dimension information carried in the thread bundle construction request through the thread block dimension register;

[0114] Step S2: Store the mode configuration information carried in the thread bundle construction request through the mode configuration register;

[0115] Step S3: Respond to the thread bundle construction request through the loop controller, enter the thread bundle construction state, read the thread block dimension information and the mode configuration information, calculate the total number of loops, and control the thread bundle calculation and construction array to perform thread coordinate calculation;

[0116] Step S4: Under the control of the loop controller, the array is constructed by thread beam calculation. According to the mode configuration information, a specified thread beam calculation construction unit is enabled to perform thread coordinate calculation cyclically according to the clock cycle and output the calculated thread coordinates. The specified thread beam calculation construction unit performs boundary judgment and thread coordinate adjustment according to the thread block dimension information during the thread coordinate calculation process.

[0117] Step S5: When the loop controller controls the designated thread bundle calculation building unit to stop calculating thread coordinates when the loop count of the designated thread bundle calculation building unit reaches the total loop count.

[0118] It should be noted that the thread bundle construction method provided in this embodiment of the invention can implement all the processing flows of the thread bundle construction device described in any of the embodiments of the first aspect above. The specific implementation schemes and technical effects of the thread bundle construction method correspond to the same specific implementation schemes and technical effects of the thread bundle construction device described in any of the embodiments of the first aspect above, and will not be repeated here.

[0119] A third aspect of the present invention provides a processor including the thread bundle construction apparatus described in any of the first aspects above.

[0120] The processor can be any of the following: CPU (Central Processing Unit), GPU (Graphics Processing Unit), TPU (Tensor Processing Unit), NPU (Neural Network Processing Unit), DPU (Deep Learning Processing Unit), APU (Accelerated Processing Unit), and GPGPU (General-Purpose computing on Graphics Processing Unit). The thread bundle construction device is a dedicated hardware module within the processor used to generate the coordinates of each thread within a thread bundle for use by the processor's computing units during scheduling.

[0121] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the technical principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A thread bundle construction device, characterized in that, include: The thread block dimension register is used to store the thread block dimension information carried in the thread bundle construction request; A mode configuration register is used to store the mode configuration information carried in the thread bundle construction request; A loop controller is used to respond to the thread bundle construction request, and after entering the thread bundle construction state, read the thread block dimension information and the mode configuration information, calculate the total number of loops, and control the thread bundle calculation and construction array to perform thread coordinate calculation. A thread beam computation building array is used, under the control of the loop controller, to enable a specified thread beam computation building unit according to the mode configuration information, so as to perform thread coordinate calculation cyclically according to the clock cycle and output the calculated thread coordinates; wherein, the specified thread beam computation building unit performs boundary judgment and thread coordinate adjustment according to the thread block dimension information during the thread coordinate calculation process; The loop controller is further configured to control the designated thread bundle calculation building unit to stop calculating thread coordinates when the number of loops of the designated thread bundle calculation building unit reaches the total number of loops.

2. The thread bundle construction apparatus as described in claim 1, characterized in that, The thread bundle construction request is initiated by an external task allocation unit.

3. The thread bundle construction apparatus as described in claim 1, characterized in that, The mode configuration information includes: thread arrangement mode and the number of arithmetic logic units called per clock cycle; wherein, the thread arrangement mode includes: linear mode, N-shaped mode or Z-shaped mode; the number of arithmetic logic units is used to characterize the computational granularity.

4. The thread bundle construction apparatus as described in claim 1, characterized in that, The thread bundle computation array includes: M1 linear mode thread bundle computation units, M2 N-shaped mode thread bundle computation units, M3 Z-shaped mode thread bundle computation units, and a multiplexer; wherein, the multiplexer is used to enable a specified thread bundle computation unit and output the calculated thread coordinates according to the mode configuration information; M1≥1; M2≥1; M3≥1.

5. The thread bundle construction apparatus as described in claim 3, characterized in that, The step of enabling a specified thread beam computation building unit to perform thread coordinate calculations cyclically according to a clock cycle includes: By specifying the thread bundle calculation building unit, based on the initial value of the thread coordinates in the current clock cycle, and according to the incrementing rule defined by the corresponding thread arrangement pattern, the coordinates of all threads in the current clock cycle are calculated. During the calculation of each thread coordinate, the coordinate values ​​of each dimension of the currently calculated thread coordinate are compared with the dimension size determined by the thread block dimension information to obtain the corresponding comparison result. If the comparison result is that the coordinates of the dimensionless coordinates are out of bounds, then the currently calculated thread coordinates are maintained. If the comparison result indicates that a dimension coordinate value is out of bounds, then the out-of-bounds dimension coordinate value is reset to its initial value and carried over to the coordinate value of the higher dimension. The last thread coordinate calculated in the current clock cycle is used as the initial thread coordinate value for the next clock cycle, until the total number of loops is reached.

6. The thread bundle construction apparatus as described in claim 5, characterized in that, The step of resetting the out-of-bounds dimensional coordinate values ​​to their initial values ​​and carrying over to higher-dimensional coordinate values ​​includes: Reset the out-of-bounds dimension coordinates to 0, and increment the coordinates of the next higher dimension by 1; If the coordinate value of the next higher dimension exceeds the limit after being incremented by 1, the coordinate value of the next higher dimension is reset to 0, and the process continues to increment the coordinate value of the next higher dimension by 1.

7. The thread bundle construction apparatus as described in claim 4, characterized in that, The arithmetic logic units can be reused among the various thread bundle computation building units.

8. The thread bundle construction apparatus as described in claim 3, characterized in that, When the thread arrangement pattern is an N-shaped pattern or a Z-shaped pattern, the dimensions corresponding to the X and Y dimensions in the thread block dimension information are integer multiples of the dimensions of the corresponding thread arrangement granularity in the X and Y dimensions, respectively.

9. The thread bundle construction apparatus as described in claim 1, characterized in that, The calculated thread coordinates are output to an external thread beam coordinate buffer.

10. A method for constructing a thread bundle, characterized in that, The method, applicable to any one of claims 1 to 9, comprises: The thread block dimension information carried in the thread bundle construction request is stored in the thread block dimension register; The mode configuration information carried in the thread bundle construction request is stored in the mode configuration register; The loop controller responds to the thread bundle construction request, enters the thread bundle construction state, reads the thread block dimension information and the mode configuration information, calculates the total number of loops, and controls the thread bundle calculation and construction array to perform thread coordinate calculation. Under the control of the loop controller, the array constructed by thread beam computation enables a designated thread beam computation construction unit according to the mode configuration information to perform thread coordinate calculation cyclically according to the clock cycle and output the calculated thread coordinates; wherein, the designated thread beam computation construction unit performs boundary judgment and thread coordinate adjustment according to the thread block dimension information during the thread coordinate calculation process; When the loop controller controls the designated thread bundle calculation building unit to stop calculating thread coordinates when the number of loops in the designated thread bundle calculation building unit reaches the total number of loops, the loop controller controls the designated thread bundle calculation building unit to stop calculating thread coordinates.

11. A processor, characterized in that, Includes the thread bundle construction apparatus as described in any one of claims 1 to 9.

Images