Systolic arithmetic array device and control method

Abstract

To prevent deterioration in calculation performance of an arithmetic array.SOLUTION: An arithmetic array device comprises: a systolic arithmetic array 2 comprising a plurality of arithmetic units 4 connected in an array; and an adjustment unit 3 connected to one or more arithmetic units 4 in a final stage. Each arithmetic unit 4 shifts a plurality of bits of an exponent part in a calculation result for input data, according to a value of one or more bits of the exponent part in the calculation result and updates shift information indicating a cumulative amount of shift by at least one arithmetic unit 4 according to the shift and outputs data obtained by shifting the plurality of bits of the exponent part as the input data to another arithmetic unit 4 which is a subsequent stage unit of the arithmetic unit 4 or the adjustment unit 3 and outputs the shift information to the subsequent stage unit, and the adjustment unit 3 adjusts the input data inputted from each of the one or more arithmetic units 4 on the basis of the shift information inputted from each of the one or more arithmetic units 4.SELECTED DRAWING: Figure 4

Description

[Technical Field]

[0001] The present invention relates to a systolic arithmetic array device and a control method thereof. [Background technology]

[0002] In general-purpose matrix multiplication calculations or calculations for AI (Artificial Intelligence), systolic-type arithmetic arrays (hereinafter sometimes simply referred to as "arithmetic arrays") that have multiple arithmetic elements (arithmetic units) connected in an array (matrix) are sometimes used.

[0003] Each arithmetic element in the arithmetic array performs unit arithmetic processing such as addition or multiplication on input data and outputs the arithmetic result as input data to the subsequent arithmetic element. With this configuration, the arithmetic array is superior in terms of power performance compared to general GPUs (Graphics Processing Units) or CPUs (Central Processing Units), because it does not require registers to store arithmetic results.

[0004] In general AI calculations, the precision of numbers expressed in 32-bit floating-point format (FP (Floating Point)32) is sometimes sufficient. Therefore, it is conceivable to improve computational performance and power performance by having the arithmetic array perform calculations using a floating-point format with lower precision than FP32, such as 16-bit floating-point format (FP16) or BF (Brain Float)16.

[0005] However, compared to FP32, FP16 has a narrower representation range and lower resolution, so there is a possibility that the absolute value of the calculation result (operation result) will overflow, exceeding the upper limit of the representation range of FP16, or underflow, falling below the lower limit.

[0006] To prevent such overflow or underflow, a method is known in which the expression range of FP16 is made variable by multiplying the calculation result by a scaling factor (SF) as the calculation (e.g., matrix calculation) progresses. [Prior art documents] [Patent documents]

[0007] [Patent Document 1] US Patent Application Publication No. 2018 / 0322607 Summary of the Invention [Problem to be solved by the invention]

[0008] The calculation for adjusting the SF includes obtaining the maximum value of the calculation result of the matrix calculation and calculating the SF based on the maximum value, and is performed serially between the matrix calculation and the next matrix calculation. Since the obtaining of the maximum value and the calculation of the SF are often performed by the CPU, for example, when a calculation such as a matrix calculation is performed by an arithmetic array, the calculation for adjusting the SF by the CPU becomes an overhead, and the arithmetic performance of the arithmetic array may be reduced.

[0009] In one aspect, an object of the present invention is to suppress a decrease in the arithmetic performance of an arithmetic array. [Means for solving the problem]

[0010] In one aspect, an arithmetic array device may include a systolic arithmetic array including a plurality of arithmetic units connected in an array, and an adjustment unit connected to one or more arithmetic units in a final stage of the arithmetic array. Each of the plurality of arithmetic units may shift multiple bits of an exponent part of a result of an arithmetic operation on input data input to the arithmetic unit, depending on the value of one or more bits of the exponent part of the result of the arithmetic operation. Each of the plurality of arithmetic units may update shift information indicating an accumulated amount of shifts by at least one arithmetic unit, depending on the shift. Each of the plurality of arithmetic units may output data obtained by shifting the multiple bits of the exponent part as the input data to another arithmetic unit that is a subsequent stage unit connected subsequent to the arithmetic unit, or to the adjustment unit, and may also output the shift information to the subsequent stage unit. The adjustment unit may adjust the input data input from each of the one or more arithmetic units based on the shift information input from each of the one or more arithmetic units. [Effects of the Invention]

[0011] In one aspect, it is possible to suppress a decrease in the computational performance of the computational array. [Brief explanation of the drawings]

[0012] [Figure 1] FIG. 10 is a diagram illustrating an example of a representation range according to the number of bits in the floating-point format. [Figure 2] FIG. 10 is a diagram illustrating an example of adjusting SF. [Figure 3] FIG. 10 is a diagram illustrating an example of application of SF to a matrix multiplication operation. [Figure 4] 1 is a block diagram showing an example of the configuration of an arithmetic array device according to an embodiment; [Figure 5] FIG. 2 is a diagram for explaining an example of the operation of each PE (Processing Element) in the arithmetic array. [Figure 6] FIG. 2 is a block diagram illustrating an example of the configuration of a PE according to an embodiment. [Figure 7] FIG. 10 is a block diagram showing an example of the configuration of an SF determination update unit. [Figure 8] FIG. 2 is a block diagram showing an example of the configuration of a data reconstruction unit. [Figure 9] FIG. 2 is a block diagram showing a configuration example of an SF adjustment unit. [Figure 10] FIG. 10 is a diagram illustrating an example of a table. [Figure 11] 10 is a flowchart illustrating an example of the operation of each PE in an arithmetic array according to an embodiment. [Figure 12] 10 is a flowchart illustrating an example of the operation of an SF adjustment unit according to an embodiment. [Figure 13] FIG. 10 is a block diagram showing the configuration of a arithmetic array device according to a modified example. [Figure 14] FIG. 10 is a block diagram showing a configuration of an SF determination update unit according to a modified example. [Figure 15] 10 is a flowchart illustrating the operation of each PE in the arithmetic array according to the modification. DETAILED DESCRIPTION OF THE INVENTION

[0013] Hereinafter, embodiments of the present invention will be described with reference to the drawings. However, the embodiments described below are merely examples, and are not intended to exclude various modifications or application of techniques not explicitly described below. For example, the present embodiment can be implemented with various modifications within the scope of the spirit thereof. In the drawings used in the following description, parts with the same reference numerals represent the same or similar parts unless otherwise specified.

[0014] [A] Explanation of low-precision operations 1 is a diagram showing an example of the representation range (representable range, hereinafter sometimes simply referred to as "range") according to the number of bits in the floating-point format. Reference symbol A1 indicates the FP32 numeric format, reference symbol A2 indicates the FP16 numeric format, and reference symbol A3 indicates the maximum value and minimum positive number (resolution) for each of FP32 and FP16.

[0015] FP32 includes a 1-bit sign part indicating positive or negative, an 8-bit exponent part indicating the number of decimal point shifts, and a 23-bit mantissa part indicating the value after the decimal point shift. The maximum representation range is 3.4 x 10 38 ,The smallest positive number is 1.18×10 -38 FP16 includes a 1-bit sign part, a 5-bit exponent part, and a 10-bit mantissa part, and the representation range is 65554 (maximum) and 6.10×10 (minimum positive number). -5 is.

[0016] Thus, FP16 is 10 10 Although it has a wide value range of about 6.10×10, compared to FP32, the expression range is narrower and the resolution is lower. For example, if there is a bias in the calculation result value in the expression range of FP16, the calculation result may be less than 6.10×10 -5 10 to 65554, which is likely to result in overflow or underflow. For example, in FP16, 70000 and 80000 cannot be distinguished, and both are rounded to 65554 due to overflow. Also, in FP16, 0.0000001 and 0.0000005 cannot be distinguished, and both are rounded to 65554 due to underflow. -5 is rounded to.

[0017] Hereinafter, operations using a floating-point format with lower precision (e.g., FP16, BF16, FP8, etc.) than the precision sufficient for performing the operation (e.g., FP32) may be referred to as operations with low precision or low-precision operations.

[0018] To achieve low-precision calculations, it is possible to bring the calculation result into the range of FP16 by multiplying the calculation result by SF. SF is the value by which the calculation result is multiplied to bring the calculation result into the range of FP16, in other words, the number of decimal point shifts.

[0019] For example, in the low-precision calculation of a certain neural network (NN), the calculation result of the first layer is 10 -7 ~10 -0(outside the range of FP16), and the calculation result of the second layer is 10 -1 ~10 5 The calculation results for the first layer are assumed to be in the range of 10 (within the range of FP16). 2 By multiplying this, the calculation result of the first layer can be within the range of FP16. As a result, the maximum value of the calculation result of the second layer is 10 7 This is outside the range of FP16. Therefore, we set SF to 10 for the entire second layer calculation results. -2 By multiplying this, the calculation result of the second layer can be made within the range of FP16.

[0020] FIG. 2 shows an example of SF adjustment. The symbol B1 indicates the range of 10 that can be expressed by FP32. -38 ~10 38 The code B2 indicates the range of FP16 representation, 10 -5 ~10 5 In the symbol B3, SF is 10 as the calculation progresses. -2 When updated to FP16 (*10 -2 ) is expressed in the range of 10. 4 When updated to FP16 (*10 4 ) represents the range of representation possible for FP16. In this way, by using SF, the range of representation possible for FP16 can be made variable.

[0021] FIG. 3 shows an example of applying SFs to matrix multiplication. Reference symbol C1 shows an example in which elements of matrices A, B, and C in the matrix product A×B=C are expressed in FP16, and each element of matrices A, B, and C is multiplied by SF0, SF1, and SF2. Reference symbol C2 shows an example in which elements of matrices A, B, and C are multiplied by different SFs. For example, elements A0, A1, A2, and A3 of matrix A are multiplied by SF00, SF01, SF02, and SF03, respectively. Elements B0, B1, B2, and B3 of matrix B are multiplied by SF10, SF11, SF12, and SF13, respectively. Elements C0, C1, C2, and C3 of matrix C are multiplied by SF20, SF21, SF22, and SF23, respectively.

[0022] In this way, by adjusting the SF for each predetermined unit in the calculation, it is possible to achieve pseudo-high accuracy. The predetermined unit may be, for example, a matrix calculation, a tile or chunk into which a matrix is ​​divided, or a layer of a neural network. For example, by dividing the convolution calculation of a neural network into chunks and using a different SF for each chunk, it is possible to reduce accuracy degradation in floating-point formats smaller than FP32, such as FP16 or FP8.

[0023] The calculation for adjusting the SF includes obtaining the maximum value of the matrix calculation results and calculating the SF based on the maximum value, and is performed serially by the CPU between multiple matrix calculations performed by the arithmetic array. Therefore, the calculation for adjusting the SF by the CPU becomes overhead, and there is a possibility that the arithmetic performance of the arithmetic array will be reduced.

[0024] Therefore, in one embodiment, a method for suppressing the degradation of the arithmetic performance of an arithmetic array that performs low-precision arithmetic will be described.

[0025] [B] Example of the configuration of an arithmetic array device according to one embodiment 4 is a block diagram showing an example of the configuration of a arithmetic array device 1 according to one embodiment. The arithmetic array device 1 is a systolic arithmetic array device and an example of an accelerator. As shown in FIG. 4, the arithmetic array device 1 includes, as its circuit (hardware) configuration, a arithmetic array 2 and an SF adjustment unit 3 connected to the arithmetic array 2.

[0026] The arithmetic array 2 includes a plurality of PEs (Processing Elements) 4 connected in an array. Examples of the arithmetic array 2 include systolic-type arithmetic arrays such as a systolic array and a specific CGRA (Coarse-Grained Reconfigurable Architecture). A specific CGRA is, for example, a CGRA in which systolic array-type operations are implemented.

[0027] The PE4 is an example of an arithmetic unit, arithmetic device, or arithmetic circuit that executes various arithmetic operations, such as unit operations such as addition and multiplication. Hereinafter, a specific PE4 among the multiple PE4 may be referred to as PE#00 using the numbers #00 to #03, #10 to #13, #20 to #23, and #30 to #33 shown in FIG. 4 . The multiple PE4 may have the same configuration. The multiple PE4 are connected in a matrix along the left-right direction (horizontal axis) and the up-down direction (vertical axis) of the page. The arithmetic array 2 may include two or more PE4s, with at least one column in the horizontal axis direction and at least two rows in the vertical axis direction.

[0028] Input data sequence #0 is input from one side of the horizontal axis (the left side of the paper in FIG. 4) and input data sequence #1 is input from one side of the vertical axis (the top of the paper in FIG. 4) to the operational array 2. The multiple PEs 4 propagate data in one direction between adjacent PEs 4 in each axis direction (see the solid arrows in FIG. 4).

[0029] In the example of FIG. 4, PE4 propagates data of data sequence #0 horizontally from left to right on the paper. PE4 also propagates the results of an operation using data of data sequence #0 and data of data sequence #1 vertically from top to bottom on the paper. Input data sequences #0 and #1 are input data to be operated on by the arithmetic array device 1, and may be matrix data, for example. Input data sequence #0 is divided into row or column elements, and input to PEs #00, #10, #20, and #30, each corresponding to one of the elements. Input data sequence #1 is divided into row or column elements, and input to PEs #01, #02, #03, and #04, each corresponding to one of the elements.

[0030] Each of the plurality of PEs 4 (PEs #30 to #33) at the bottom (final) stage in the vertical direction of the arithmetic array 2 is connected to the SF adjustment unit 3, and outputs the calculation result propagated in the vertical direction to the SF adjustment unit 3.

[0031] Hereinafter, for a certain PE4, another PE4 or SF adjustment unit 3 connected in the propagation direction (subsequent stage) of the data of the operation result from the certain PE4 may be referred to as a "subsequent stage PE4," an SF adjustment unit 3, a "subsequent stage," or simply as a "subsequent stage." Note that the subsequent stage is the SF adjustment unit 3 when the certain PE4 is located at the bottom stage in the operation array 2. Furthermore, when viewed from the subsequent stage, the certain PE4 may be referred to as a "previous stage PE4" or simply as a "previous stage."

[0032] In an arithmetic array 2 according to one embodiment, each PE4 shifts multiple bits of the exponent part of the arithmetic result according to the value of one or more bits of the exponent part of the arithmetic result, and propagates (outputs) the shifted arithmetic result to a subsequent stage as data. The PE4 also updates the value of SF according to the shift, and propagates (outputs) the value of SF together with the data to a subsequent stage (see the dashed arrow in FIG. 4). SF is an example of shift information indicating the cumulative amount of shifts by at least one PE4.

[0033] The SF adjustment unit 3 is an example of an adjustment unit connected to one or more PEs 4 in the final stage of the arithmetic array 2, and adjusts input data input from each of the plurality of PEs 4 in the lowest stage of the arithmetic array 2 based on the SF value input from each of the plurality of PEs 4, and outputs the adjusted data as an output data string. Adjusting the input data may mean, for example, adjusting the value of the input data that is different from the original value to the "true value" by shifting the bits of the exponent part. The output data string is output data that is the result of arithmetic from the arithmetic array device 1, and is, for example, matrix (matrix multiplication) data. The output data string may be stored in a memory such as a RAM (Random Access Memory) and used for processing by a processor such as a CPU.

[0034] Fig. 5 is a diagram for explaining an example of the operation of each PE 4 in the arithmetic array 2. For convenience, Fig. 5 shows PEs #00, #10, #20, and #30 of the arithmetic array device 1 shown in Fig. 4 and the SF adjustment unit 3, and does not show the other PEs 4, input data strings, output data strings, etc.

[0035] As shown in Fig. 5, PE4 shifts multiple bits of the exponent part in accordance with the values ​​of one or more bits of the exponent part in the operation result. For example, PE4 multiplies the operation result by a predetermined value and updates the value of SF. PE4 outputs the operation result, in which the bits of the exponent part have been shifted, to the subsequent stage as data to be output to the subsequent stage (denoted as "C" in Fig. 5), and outputs the updated value of SF to the subsequent stage as the value of SF to the subsequent stage (denoted as "SF" in Fig. 5). In other words, PE4 propagates the updated SF to the subsequent stage together with data C.

[0036] For example, as shown by symbol D1, when the value of the most significant bit (MSB) of the exponent part in the result of the operation executed by PE#00 is 1, PE#00 shifts the exponent part one bit to the right, thereby setting the value of the most significant bit of the exponent part to 0. PE#00 also adds 1 to the value of SF to record that the exponent part has been shifted one bit to the right, i.e., the value of the operation result has been increased by one digit. PE#00 then outputs the operation result including the exponent part after the shift (update) to the subsequent stage, and also outputs the value of SF after the addition (update) to the subsequent stage. This makes it possible to prevent overflow from occurring.

[0037] Furthermore, as shown by symbol D2, if the value of the most significant bit of the exponent part in the calculation result executed by PE#10 and the value of the bit one bit lower than the most significant bit are both 0, PE#10 shifts the exponent part one bit to the left, thereby setting the value of the least significant bit of the exponent part to 0. PE#10 also subtracts 1 from the value of SF to record that the exponent part has been shifted one bit to the left, i.e., that the value of the calculation result has been reduced by one digit. PE#10 then outputs the calculation result including the exponent part after the shift (update) to the subsequent stage, and also outputs the value of SF after the addition (update) to the subsequent stage. This makes it possible to prevent underflow from occurring.

[0038] Furthermore, as shown by reference symbol D3, when the value of the most significant bit of the exponent part in the calculation result executed by PE#20 is 0 and the value of the bit one bit lower than the most significant bit is 1, PE#20 maintains the current value of the exponent part. PE#20 also maintains the current value of SF. PE#10 then outputs the calculation result executed by PE#20 to the subsequent stage, and also outputs the current value of SF to the subsequent stage. When the values ​​of the most significant bit of the exponent part and the bit one bit lower than the most significant bit are 01, it can be said that the calculation result is in an appropriate position within the representable range of the low-precision floating-point format (e.g., FP16) based on the value of the current SF. In other words, it can be said that the possibility of both overflow and underflow occurring is low for the value of the current SF. Therefore, PE#20 maintains the current value of SF.

[0039] PE#30 performs the same process as any one of the patterns of PE#00, #10, and #20 (codes D1 to D3), and outputs the calculation result and SF to the SF adjustment unit 3.

[0040] The SF adjustment unit 3 adjusts the data C (computation result) input from the PE #30 to a true value based on the value of SF input from the PE #30. For example, the SF adjustment unit 3 updates the data C according to the following equation (1). The updated data C may be output as the processing result of the computational processing by the computational array device 1. C = C * 2 SF (1)

[0041] [C] PE configuration example Next, a configuration example of the PE 4 according to an embodiment will be described with reference to FIGS.

[0042] 6 is a block diagram showing an example of the configuration of a PE 4 according to an embodiment. As shown in the example of FIG. 6, each of the plurality of PEs 4 may include, as a circuit (hardware) configuration, a computing unit 5, an SF determination update unit 6, and a data reconstruction unit 7.

[0043] The arithmetic unit 5 performs arithmetic processing on input data (denoted as "Input0" and "Input1") input to the PE4, and outputs the arithmetic result (denoted as "Result"). Examples of the input data include data of any element in the input data sequence #0, data of any element in the input data sequence #1 (when the PE4 is the topmost stage), or input data from the data reconstruction unit 7 of the previous stage PE4 (when the PE4 is not the topmost stage). In the example of FIG. 4, the topmost PE4 is PE#00, #01, #02, and #03. The arithmetic unit 5 may be, for example, any of various known arithmetic units that perform various arithmetic processing.

[0044] The SF determination update unit 6 determines and updates the SF based on the calculation result (Result) by the arithmetic unit 5 and the SF (denoted as "InputSF") output from the preceding PE 4. As a result of determining and updating the SF, the SF determination update unit 6 outputs information on the updated exponent part (denoted as "ModifiedRe") to the data reconstruction unit 7, and outputs the updated SF (denoted as "OutputSF") to the succeeding stage. Note that the SF determination update unit 6 provided in the highest PE 4 in the arithmetic array 2 may be set to fixedly detect 0 as the InputSF.

[0045] The data reconstruction unit 7 reconstructs the data of the calculation result based on the calculation result (Result) by the calculator 5 and the information (ModifiedRe) of the exponent part updated by the SF determination update unit 6, and outputs the reconstructed output data (denoted as "Output") to the subsequent stage. Note that if the subsequent stage is PE4, the output data (Output) becomes the input data (Input0 or Input1) of the PE4, and if the subsequent stage is SF adjustment unit 3, it becomes the input data of the SF adjustment unit 3.

[0046] In the following explanation, it is assumed that the arithmetic array device 1 executes low-precision FP16 arithmetic, and that the input data (Input), arithmetic result (Result), and output data (Output) are each 16-bit data represented as [15:0], where

[15] indicates the most significant bit in the data (the leftmost bit of FP16 shown in FIG. 1), and [0] indicates the least significant bit in the data (the rightmost bit of FP16 shown in FIG. 1).

[0047] 7 is a block diagram showing an example of the configuration of the SF determination update unit 6. The SF determination update unit 6 may illustratively include an SF determination unit 61 and an SF update unit 62. In the following description, of the calculation result Result[15:0], the sign part Result

[15] may be represented as Rs, the exponent part Result[14:10] as Re[4:0] or Re, and the mantissa part Result[9:0] as Rm[9:0] or Rm.

[0048] The SF determination unit 61 determines whether to perform a right shift, a left shift, or maintain as processing on the exponent part based on the bit value of the exponent part Re[4:0] in the calculation result Result[15:0], and outputs information indicating the determination result to the SF update unit 62. This information may be a predetermined signal such as a flag.

[0049] For example, when Re[4]="1" (see symbol D1 in Figure 5), the SF determination unit 61 outputs information (right) indicating a right shift to guard against overflow since up to the most significant bit of the exponent part is used.

[0050] Furthermore, when Re[4]="0" and Re[3]="0" (see symbol D2 in Figure 5), the SF determination unit 61 outputs information (left) indicating a left shift to guard against underflow since there is room in the upper two bits of the exponent part.

[0051] Furthermore, if the situation does not correspond to a right shift or a left shift, for example, if Re[4]="0" and Re[3]="1" (see symbol D3 in Figure 5), the SF determination unit 61 outputs information indicating maintain (keep) to maintain the current state since the situation of the exponent part is balanced.

[0052] The SF update unit 62 updates (including maintains) Re[4:0] according to the determination result by the SF determination unit 61, and outputs the information ModifiedRe of the updated exponent part as the processing result of the update to the data reconstruction unit 7. Furthermore, the SF update unit 62 updates (including maintains) InputSF from the preceding PE4 according to the determination result by the SF determination unit 61, and outputs OutputSF to the succeeding stage as the processing result of the update.

[0053] For example, when the determination result of the SF determination unit 61 indicates a right shift, the SF update unit 62 sets the result of shifting Re[4:0] to the right (expressed as "Re >> 1") to ModifiedRe, and sets the result of adding 1 to InputSF to OutputSF. Shifting Re[4:0] to the right may mean, for example, setting Re[4:1] to ModifiedRe[3:0] and setting 0 to ModifiedRe[4].

[0054] Furthermore, when the determination result of the SF determination unit 61 indicates a left shift, the SF update unit 62 sets the result of shifting Re[4:0] to the left (denoted as "Re << 1") to ModifiedRe, and sets the result of subtracting 1 from InputSF to OutputSF. Shifting Re[4:0] to the left may mean, for example, setting Re[3:0] to ModifiedRe[4:1] and setting 0 to ModifiedRe[0].

[0055] Furthermore, if the determination result of the SF determination unit 61 indicates keep, the SF update unit 62 sets Re[4:0] to ModifiedRe and sets InputSF to OutputSF.

[0056] 8 is a block diagram showing an example of the configuration of the data reconstruction unit 7. The data reconstruction unit 7 may illustratively include a data replacement unit 71. In the following description, of the output data Output[15:0], the sign part Output

[15] may be represented as Os, the exponent part Output[14:10] as Oe[4:0] or Oe, and the mantissa part Output[9:0] as Om[9:0] or Om.

[0057] The data substitution unit 71 generates output data Output[15:0] by substituting the exponent part Re[4:0] of the operation result Result[15:0] with the updated exponent part information ModifiedRe, and outputs Output[15:0] to the subsequent stage. For example, the data substitution unit 71 may generate Output[15:0] by setting Os to Rs, Oe to ModifiedRe instead of Re, and Om to Rm as the data of Output[15:0].

[0058] 9 is a block diagram showing an example of the configuration of the SF adjustment unit 3, and FIG. 10 is a diagram showing an example of the table 32a. The SF adjustment unit 3 may include, for example, a multiplication unit 31 and a storage area 32.

[0059] The storage area 32 is a storage area realized by storage elements (storage devices) such as registers, SRAM (Static RAM), ROM (Read Only Memory), etc., and stores a table 32a illustrated in FIG. 10. The table 32a converts the integer value of SF (InputSF) into a power of 2 (2 InputSF For example, the storage area 32 is a table for converting the value of 2 into the value of 32a according to the input of InputSF (for convenience, it is written as Table(InputSF)). InputSF Alternatively, instead of the storage area 32 and the table 32a, the input / output circuit may be configured to output (acquire) the following data in accordance with Table(InputSF). InputSF The SF adjusting unit 3 may be provided with a circuit logic that outputs (acquires) the following.

[0060] The multiplication unit 31 multiplies InputSF input from the preceding PE4 by 2 using a table 32a in the storage area 32 or circuit logic. InputSF For example, the multiplication unit 31 converts the output of Table (InputSF) into InputSF Then, the multiplication unit 31 multiplies the Temp value 2 by the Input[15:0] input from the previous stage PE4. InputSF By multiplying this, the output data Output[15:0] is generated.

[0061] 9 may be provided corresponding to one of the multiple PEs 4 (PEs #30, #31, #32, and #33 in the example of FIG. 4) in the stage preceding the SF adjustment unit 3 (the lowest stage of the arithmetic array 2). Therefore, the SF adjustment unit 3 may include, for example, a set of the multiplication unit 31 and the storage area 32 for each PE 4 in the stage preceding the SF adjustment unit 3.

[0062] The multiplication unit 31 may be capable of processing inputs from two or more PEs 4 in parallel, and in this case, the SF adjustment unit 3 may be provided with a number of multiplication units 31 corresponding to the number of parallel processes that the multiplication units 31 can execute and the number of PEs 4 preceding the SF adjustment unit 3. Furthermore, the memory area 32 (or the table 32a) may be capable of outputting results in parallel for two or more references (inputs to InputSF), and in this case, the SF adjustment unit 3 may be provided with a number of memory areas 32 (or tables 32a) corresponding to the number of parallel results that the memory area 32 (or the table 32a) can output and the number of PEs 4 preceding the SF adjustment unit 3.

[0063] [D] Operation example Next, an example of operation of the arithmetic array device 1 according to one embodiment will be described. Fig. 11 is a flowchart for explaining an example of operation of each PE 4 in the arithmetic array 2 according to one embodiment, and Fig. 12 is a flowchart for explaining an example of operation of the SF adjustment unit 3 according to one embodiment.

[0064] [D-1] Example of PE operation 11 shows an example of the operation of one arithmetic process in each of the multiple PEs 4 included in the arithmetic array 2. As shown in the example of FIG. 11, in step S1, a PE4 obtains Input0[15:0] and Input1[15:0] from the previous PE4 or from the input data string, and obtains InputSF from the previous PE4 or as a fixed value 0.

[0065] In step S2, the arithmetic unit 5 executes an operation using Input0[15:0] and Input1[15:0], and outputs Result[15:0] as the operation result.

[0066] In step S3, the SF determination unit 61 determines whether Result

[14] (Re[4]) of Result[15:0] is "1". If Result

[14] (Re[4]) is "1" (YES in step S3), the SF determination unit 61 outputs information indicating a right shift as the determination result, and the process proceeds to step S4. If Result

[14] (Re[4]) is not "1" (is "0") (NO in step S3), the process proceeds to step S5.

[0067] In step S4, the SF update unit 62 generates ModifiedRe by shifting Re to the right, generates OutputSF by adding 1 to InputSF, and outputs ModifiedRe and OutputSF. Then, the process proceeds to step S8.

[0068] In step S5, the SF determination unit 61 determines whether Result

[13] (Re[3]) of Result[15:0] is "0". If Result

[13] (Re[3]) is "0" (YES in step S5), the SF determination unit 61 outputs information indicating a left shift as the determination result, and the process proceeds to step S6. If Result

[13] (Re[3]) is not "0" (is "1") (NO in step S5), the process proceeds to step S7.

[0069] In step S6, the SF update unit 62 generates ModifiedRe by shifting Re to the left, generates OutputSF by subtracting 1 from InputSF, and outputs ModifiedRe and OutputSF. Then, the process proceeds to step S8.

[0070] In step S7, the SF update unit 62 sets Re to ModifiedRe, sets InputSF to OutputSF, and outputs ModifiedRe and OutputSF. Then, the process proceeds to step S8.

[0071] In step S8, the data replacement unit 71 replaces Result[14:10] (Re[4:0]) of Result[15:0] with ModifiedRe generated in step S4, S6 or S7, thereby generating Output[15:0].

[0072] In step S9, the PE 4 outputs Output[15:0] generated by the data reconstruction unit 7 and OutputSF generated by the SF determination update unit 6 to the subsequent stage, and the process ends.

[0073] [D-2] Example of operation of SF adjustment unit 12 shows an example of operation when the results of one calculation process are input from each of the multiple PEs 4 in the previous stage in the SF adjustment unit 3. As shown in the example of FIG. 12, in step S11, the SF adjustment unit 3 acquires Input[15:0] and InputSF from each PE 4 in the previous stage.

[0074] In step S12, the multiplication unit 31 multiplies the numerical values ​​of each InputSF by a power of 2 (2 InputSF )

[0075] In step S13, the multiplication unit 31 multiplies the 2 nd inputs obtained from each PE4 by the Input[15:0] obtained from each PE4. InputSFBy multiplying the above, Output[15:0] for each PE4 is generated.

[0076] In step S14, the multiplication unit 31 outputs each of the generated Output[15:0], and the process ends.

[0077] [E] Effects of one embodiment As described above, the arithmetic array device 1 according to one embodiment includes a systolic arithmetic array 2 including a plurality of PEs 4 connected in an array, and an SF adjustment unit 3 connected to one or more PEs 4 in the final stage included in the arithmetic array 2. Each PE 4 shifts multiple bits of the exponent part of the arithmetic result in accordance with the value of one or more bits of the exponent part in the arithmetic result for input data input to the PE 4. Each PE 4 also updates an SF indicating the cumulative amount of shifts by at least one PE 4 in accordance with the shifts. Each PE 4 also outputs data obtained by shifting the multiple bits of the exponent part as input data to a subsequent stage unit connected subsequent to the PE 4, and outputs the SF to the subsequent stage unit. The SF adjustment unit 3 adjusts the input data input from each of the one or more PEs 4 based on the SF input from each of the one or more PEs 4.

[0078] In this way, in the arithmetic array device 1 according to one embodiment, SF adjustment (update) is performed in addition to arithmetic processing on input data in each PE 4. In other words, SF adjustment can be performed within the arithmetic array device 1. Therefore, compared to when calculations for adjusting SF are performed by the CPU, it is possible to suppress the occurrence of overhead due to communication between CPUs, etc., and to suppress a decrease in the arithmetic performance of the arithmetic array 2.

[0079] Furthermore, according to the arithmetic array device 1 of one embodiment, each PE 4 can perform an adjustment process of the SF for each operation. This enables fine adjustment of the SF, so that the operation result of each operation can be flexibly adjusted so that it is located within the representable range of the low-precision floating-point format, regardless of the contents of the input data string, the contents of the operation, etc. In other words, the representable range of the low-precision floating-point format can be flexibly shifted so that the operation result of each operation is included in an appropriate position.

[0080] Furthermore, each of the plurality of PEs 4 according to an embodiment shifts the plurality of bits of the exponent part to the right when the value of the most significant bit of the exponent part is 1. This makes it possible to prevent overflow from occurring.

[0081] Furthermore, each of the multiple PEs 4 according to one embodiment shifts multiple bits of the exponent part to the left when the value of the most significant bit of the exponent part and the value of the bit one bit lower than the most significant bit are both 0. This makes it possible to prevent underflow from occurring.

[0082] Furthermore, SF adjustment unit 3 according to one embodiment performs adjustment by converting the SF input from each of one or more PEs 4 into a power of 2 value and multiplying the converted value by the input data input from each of one or more PEs 4. This allows data output from arithmetic array 2 to be converted into a true value, thereby enabling appropriate arithmetic results to be output from arithmetic array device 1.

[0083] [F] Variation Next, a description will be given of a variation of the arithmetic array device 1 according to an embodiment. In the variation of the embodiment, a method for storing a history of updates (changes) to the SFs in each PE 4 will be described.

[0084] 13 is a block diagram showing the configuration of a arithmetic array device 1A according to a modified example. The arithmetic array device 1A includes an arithmetic array 2A, an SF adjustment unit 3, and a history storage unit 8. The SF adjustment unit 3 is the same as in the first embodiment, except that it can also output output data Output to a debugger (not shown).

[0085] The arithmetic array 2A includes a plurality of PE4A, and also includes paths (routes) connecting each PE4A to the history storage unit 8. Other configurations in the arithmetic array 2A, such as the connection configuration of the PE4A for propagating data C and SF, are the same as those in the arithmetic array 2 shown in Fig. 4. For convenience, Fig. 13 shows only some of the PE4A included in the arithmetic array 2A (for example, PE4A corresponding to PEs #00, #10, #20, and #30 included in the arithmetic array 2 shown in Fig. 4), and does not show the other PE4A, input data strings, output data strings, etc.

[0086] In addition to the configuration of the PE4 according to one embodiment, the PE4A includes a register that holds difference information of the SF. The difference information is an example of information indicating the difference before and after updating the SF. After a predetermined unit of arithmetic processing in the arithmetic array 2A is completed, the PE4A outputs the difference information (denoted as "D") held in the register to the history holding unit 8 via a path, starting from the lowest PE4A.

[0087] 13 shows a propagation path between PE4A and the history storage unit 8 that is similar to the propagation path of data C and SF. Similar to the propagation path of data C and SF, difference information D may be propagated via each PE4A from the top PE4A to the bottom PE4A in the arithmetic array 2A. In this case, each of the multiple PE4A in the bottom tier may be connected to the history storage unit 8. The same applies to other columns of PE4A (not shown) (e.g., columns of PE#01, #11, #21, #31, etc.). Note that the path is not limited to the example shown in FIG. 13, and may be, for example, a path that individually connects each of the multiple PE4A to the history storage unit 8.

[0088] The history holding unit 8 is a storage element such as a register, SRAM, or ROM, and is an example of a storage device. The history holding unit 8 holds a history (internal information) of SF updates in each PE4A of the arithmetic array 2A. For example, the history holding unit 8 holds difference information D for each column of PE4A as a history of SF updates in each PE4A. As shown in FIG. 13, the history holding unit 8 is a four-stage storage element, and holds the difference information D received from the bottommost PE4A in the column of PE4A in the deepest stage. In the example of FIG. 13, the number of the PE4A that sent the difference information is added to each piece of difference information in the history holding unit 8.

[0089] 14 is a block diagram showing the configuration of an SF determination update unit 6A according to a modified example. The SF determination update unit 6A includes an SF determination unit 61 similar to that of the SF determination update unit 6 shown in FIG. 7, and also includes an SF update unit 62A instead of the SF update unit 62. The SF determination update unit 6A also includes a register 63 and a transfer unit 64.

[0090] In addition to the processing of the SF updating unit 62, the SF updating unit 62A stores difference information (denoted as "Diff"), which is the added value performed on the InputSF to generate the OutputSF, in the register 63. The added value is, for example, "+1" added to the InputSF when a right shift is performed on the bits of the exponent part, "-1" added to the InputSF when a left shift is performed on the bits of the exponent part, and "+-0" added to the InputSF when the exponent part is maintained.

[0091] The register 63 holds the difference information Diff input from the SF update unit 62 A. Furthermore, the register 63 of a PE 4 A other than the topmost one may hold the difference information Diff (difference information D) input from the preceding PE 4 A after outputting the difference information Diff input from the SF update unit 62 A.

[0092] The transfer unit 64 outputs the difference information Diff held in the register 63 to the subsequent PE4A or the history holding unit 8 as difference information D. For example, the transfer unit 64 repeatedly executes a transfer process in which it reads the difference information Diff from the register 63 and outputs (stores) it to the register 63 of the subsequent PE4A, or to the history holding unit 8 if the subsequent stage is the history holding unit 8.

[0093] The transfer process may be started, for example, when the arithmetic process of a predetermined unit in the arithmetic array 2A, for example, the arithmetic process of a tile unit, is completed, and may be ended when the difference information D held in the register 63 of the topmost PE 4A is stored in the history holding unit 8. In the example of FIG. 13, the transfer process is repeatedly executed four times.

[0094] Next, an example of the operation of the arithmetic array device 1A according to the modified example will be described. Note that an example of the operation of the SF adjustment section 3 according to the modified example is the same as that shown in FIG.

[0095] Fig. 15 is a flowchart for explaining the operation of each PE 4A in an arithmetic array 2A according to the modification. In the flowchart shown in Fig. 15, in addition to steps S1 to S9 shown in Fig. 11, steps S21 to S24 are added, and each process of steps S1 to S9 is the same as the process of the same reference numerals shown in Fig. 11.

[0096] 15, step S21 is executed between step S4, S6 or S7 and step S8. In step S21, the SF update unit 62A stores difference information Diff, which is an added value when generating OutputSF, in the register 63. Note that step S21 may be executed after step S8 or S9, or may be executed in parallel with step S8 or S9.

[0097] In step S22 after step S9, the transfer unit 64 determines whether or not the calculation processing for a predetermined unit, for example, the calculation processing for a tile unit, has been completed. If the calculation processing for the predetermined unit has not been completed (NO in step S22), the processing ends. If the calculation processing for the predetermined unit has been completed (YES in step S22), the processing proceeds to step S23.

[0098] In step S23, the transfer unit 64 reads the difference information Diff from the register 63 and transfers it to the register 63 of the subsequent PE 4A, or to the subsequent history holding unit 8 if its own PE 4A is the lowest stage.

[0099] In step S24, the transfer unit 64 determines whether the transfer has finished, for example, whether the difference information Diff stored in the register 63 of the top-stage PE4A has been stored in the history holding unit 8. If the transfer has not finished (NO in step S24), the process proceeds to step S23. If the transfer has finished (YES in step S24), the process ends.

[0100] As described above, the arithmetic array device 1A according to the modified example includes a history holding unit 8. Each PE 4A also includes a register 63 that stores difference information Diff indicating the difference before and after updating the SF, and a transfer unit 64 that outputs difference information Diff(D) to the history holding unit 8 after a predetermined unit of arithmetic processing in the arithmetic array 2A is completed.

[0101] As a result, for example, a debugger (not shown) can reverse-calculate the operations performed by each PE 4A in the arithmetic array 2A during debugging by linking the difference information D stored in the history storage unit 8 with the output data Output output from the SF adjustment unit 3. This allows for easy debugging and efficient use of the arithmetic array device 1A.

[0102] [G] Other The techniques according to the embodiment and the modified examples described above can be implemented with the following modifications and variations.

[0103] For example, the number of PEs 4, and the number of columns and rows of PEs 4 in the arithmetic array 2 are not limited to the numbers shown in Fig. 4, and may be various numbers. Also, the blocks (function blocks, circuit blocks, etc.) shown in Fig. 6 to Fig. 9 and Fig. 14 may be merged or divided in any combination.

[0104] Furthermore, in the embodiment and the modified example, FP16 is given as an example of a low-precision floating-point format, but this is not limiting, and various floating-point formats such as BF16 and FP8, in which the number of bits (number of digits) of the exponent and mantissa parts are different from FP16, may be used. When a floating-point format other than FP16 is used, it is sufficient to change the number of bits (number of digits) of the exponent part to be shifted in the method described in the embodiment and the modified example.

[0105] [H] Note The following additional notes are provided regarding the above embodiment and modifications.

[0106] (Appendix 1) a systolic arithmetic array having a plurality of arithmetic units connected in an array; an adjustment unit connected to one or more arithmetic units in a final stage of the arithmetic array; Each of the plurality of calculation units shifting a plurality of bits of the exponent part of the operation result in accordance with a value of one or more bits of the exponent part of the operation result for the input data input to the operation unit; updating shift information indicating an accumulated amount of shifts by at least one calculation unit in accordance with the shift; outputting data obtained by shifting the plurality of bits of the exponent part as the input data to another arithmetic unit or the adjustment unit, which is a subsequent stage unit connected to a subsequent stage of the arithmetic unit, and outputting the shift information to the subsequent stage unit; the adjustment unit adjusts the input data input from each of the one or more calculation units based on the shift information input from each of the one or more calculation units. A systolic computing array device.

[0107] (Appendix 2) Each of the plurality of calculation units If the value of the most significant bit of the exponent is 1, shifting the bits of the exponent to the right. 2. The systolic arithmetic array device according to claim 1.

[0108] (Appendix 3) Each of the plurality of calculation units shifting the plurality of bits of the exponent part to the left when both the value of the most significant bit of the exponent part and the value of the bit one bit lower than the most significant bit are 0; 3. The systolic arithmetic array device according to claim 1 or 2.

[0109] (Appendix 4) a storage device; Each of the plurality of calculation units a register for storing difference information indicating a difference before and after updating the shift information; outputting the difference information to the storage device after completion of a predetermined unit of arithmetic processing in the arithmetic array; 4. The systolic arithmetic array device according to claim 1.

[0110] (Appendix 5) The adjustment unit converting the shift information input from each of the one or more arithmetic units into a value of a power of two, and multiplying the input data input from each of the one or more arithmetic units by the converted value, thereby performing the adjustment. 10. The systolic arithmetic array device according to claim 1, wherein the first and second arithmetic array devices are connected to one another.

[0111] (Appendix 6) A method for controlling a systolic arithmetic array device, comprising: In a systolic arithmetic array having a plurality of arithmetic units connected in an array, each of the plurality of arithmetic units is shifting a plurality of bits of the exponent part of the operation result in accordance with a value of one or more bits of the exponent part of the operation result for the input data input to the operation unit; updating shift information indicating an accumulated amount of shifts by at least one calculation unit in accordance with the shift; outputting data obtained by shifting the plurality of bits of the exponent part as the input data to another arithmetic unit that is a subsequent stage unit connected to a subsequent stage of the arithmetic unit, or to an adjustment unit connected to one or more arithmetic units in a final stage in the arithmetic array, and outputting the shift information to the subsequent stage unit; the adjustment unit adjusts the input data input from each of the one or more calculation units based on the shift information input from each of the one or more calculation units. Control method.

[0112] (Appendix 7) Each of the plurality of calculation units If the value of the most significant bit of the exponent is 1, shifting the bits of the exponent to the right. 6. The control method according to claim 6.

[0113] (Appendix 8) Each of the plurality of calculation units shifting the plurality of bits of the exponent part to the left when both the value of the most significant bit of the exponent part and the value of the bit one bit lower than the most significant bit are 0; 8. The control method according to claim 6 or 7.

[0114] (Appendix 9) Each of the plurality of calculation units storing difference information indicating a difference between before and after the update of the shift information in a register; outputting the difference information to a storage device after completion of a predetermined unit of arithmetic processing in the arithmetic array; The control method according to any one of Supplementary notes 6 to 8.

[0115] (Appendix 10) The adjustment unit: converting the shift information input from each of the one or more arithmetic units into a value of a power of two, and multiplying the input data input from each of the one or more arithmetic units by the converted value, thereby performing the adjustment. A control method according to any one of Supplementary notes 6 to 9. [Explanation of symbols]

[0116] 1,1A Computing Array Device 2,2A Computational Array 3 SF adjustment section 31 Multiplication section 32 Storage area 32a Table 4,4A PE 5 Arithmetic unit 6,6A SF judgment update section 61 SF Judgment Department 62 SF update section 63 registers 64 Transfer Unit 7 Data Reconstruction Unit 71 Data replacement section 8 History storage unit

Claims

1. a systolic arithmetic array having a plurality of arithmetic units connected in an array; an adjustment unit connected to one or more arithmetic units in a final stage of the arithmetic array; Each of the plurality of calculation units shifting a plurality of bits of an exponent part of the operation result in accordance with a value of one or more bits of the exponent part of the operation result for input data input to the operation unit; updating shift information indicating an accumulated amount of shifts by at least one calculation unit in response to the shift; outputting data obtained by shifting the plurality of bits of the exponent part as the input data to another arithmetic unit or the adjustment unit, which is a subsequent stage unit connected to a subsequent stage of the arithmetic unit, and outputting the shift information to the subsequent stage unit; the adjustment unit adjusts the input data input from each of the one or more calculation units based on the shift information input from each of the one or more calculation units. A systolic computing array device.

2. Each of the plurality of calculation units If the value of the most significant bit of the exponent is 1, shifting the bits of the exponent to the right.

2. The systolic arithmetic array device according to claim 1.

3. Each of the plurality of calculation units shifting the plurality of bits of the exponent part to the left when the value of the most significant bit of the exponent part and the value of the bit one bit lower than the most significant bit are both 0; 3. The systolic arithmetic array device according to claim 1.

4. a storage device; Each of the plurality of calculation units a register for storing difference information indicating a difference before and after updating the shift information; outputting the difference information to the storage device after completion of a predetermined unit of arithmetic processing in the arithmetic array; 3. The systolic arithmetic array device according to claim 1.

5. The adjustment unit converting the shift information input from each of the one or more arithmetic units into a value of a power of two, and multiplying the input data input from each of the one or more arithmetic units by the converted value, thereby performing the adjustment.

3. The systolic arithmetic array device according to claim 1.

6. A method for controlling a systolic arithmetic array device, comprising: In a systolic arithmetic array having a plurality of arithmetic units connected in an array, each of the plurality of arithmetic units is shifting a plurality of bits of an exponent part of the operation result in accordance with a value of one or more bits of the exponent part of the operation result for input data input to the operation unit; updating shift information indicating an accumulated amount of shifts by at least one calculation unit in response to the shift; outputting data obtained by shifting the plurality of bits of the exponent part as the input data to another arithmetic unit that is a subsequent stage unit connected to a subsequent stage of the arithmetic unit, or to an adjustment unit connected to one or more arithmetic units in a final stage in the arithmetic array, and outputting the shift information to the subsequent stage unit; the adjustment unit adjusts the input data input from each of the one or more calculation units based on the shift information input from each of the one or more calculation units. Control method.

Images