Processing Unit

Abstract

To efficiently, at a low cost, execute arithmetic processing which requires time, such as calculation of fully-connected layers and calculation of convolution layers that require many product-sum operations.SOLUTION: An arithmetic processing apparatus includes: data supply units (110, 120) which supply input data based on calculation of convolution layers and calculation of fully-connected layers, and weight data; a parallel processor 130 equipped with a plurality of processor sets 140 each having a plurality of processors 150 and executing calculation at different timings based on the input data and the weight data, for output; and addition units (160, 170). The plurality of processors 150 are included in the same processor set 140, and execute calculation at the same timing based on the input data and the weight data, for output. The addition units (160, 170) output the sum of results of the calculations executed at the same timing in the plurality of processors 150 of the processor sets 140 during executing calculation of the fully-connected layers.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a processing device that executes operations of a fully connected layer and an operation of a convolutional layer. [Background technology]

[0002] In recent years, in order to provide users with applications using AI, there has been a surge in proposals for processing devices (circuits) capable of performing AI-used arithmetic operations at high speeds, with the aim of accelerating the processing of these applications and improving user convenience. Particularly time-consuming arithmetic operations include fully connected layer operations and convolutional layer operations, which require the execution of many product-sum operations, and there have been many proposals for processing devices to perform these operations at high speeds. Among these, many proposals have been made for product-sum circuits using systolic arrays, as they are advantageous for constructing ultra-large-scale parallel product-sum operators (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2017-21483 A (page 9, FIG. 6) Summary of the Invention [Problem to be solved by the invention]

[0004] However, in an arithmetic circuit using a systolic array, the result of the multiplication and accumulation is propagated between adjacent multiplication and accumulation units, so that the timing of input data must be shifted between adjacent units, and a buffer is required for delay. Also, since it is necessary to set weight data according to the arrangement of the multiplication and accumulation units arranged side by side, it may not be possible to use all the arranged multiplication and accumulation units.

[0005] Furthermore, due to the above, the input data must be input according to the weight data set in the multiply-add calculator, and in the calculation of the convolution layer, the data may have to be rearranged from the original input data order (raster order). As described above, the conventional calculation circuit using the systolic array has problems such as increased costs and reduced processing performance due to the increase in circuit size. [Means for solving the problem]

[0006] The processing device according to the present invention is a processing device that executes a first operation mode and a second operation mode, a data supplying unit that supplies input data and weight data based on the first operation mode and the second operation mode, a parallel arithmetic unit including a plurality of arithmetic unit sets each including a plurality of arithmetic units, each performing an operation based on the input data and the weight data supplied by the data supplying unit at different timings and outputting an operation result, and a sum-aggregation unit; the plurality of arithmetic units are in the same arithmetic unit set, and execute arithmetic operations at the same timing based on the input data and the weight data supplied by the data supply unit, and output arithmetic results; The sum-sum unit outputs a sum of operation results that are executed and output at the same timing by the plurality of arithmetic units in the arithmetic unit set when the second arithmetic mode is executed. Effect of the Invention

[0007] According to the present invention, a buffer for delaying input data is not required, more of the arranged arithmetic units can be utilized, and in the operation of the first operation mode, there is no need to rearrange the data from the original input data order, so that an arithmetic processing device can be provided that has improved processing performance while keeping the circuit size small. [Brief description of the drawings]

[0008] [Figure 1] 1 is a configuration diagram of a processing device according to a first embodiment of the present invention. [Diagram 2]FIG. 2 is a circuit diagram showing a configuration common to each computing unit. [Diagram 3] FIG. 13 is a diagram illustrating the execution of a multiply-and-accumulate operation in the calculation of a convolution layer. [Figure 4] FIG. 13 is a diagram showing the input state to the parallel computing unit in cycle (TIM1) and cycle (TIM2) in the calculation of the convolution layer. [Diagram 5] 13A and 13B are diagrams illustrating the input state to the parallel computing unit in cycle (TIM3) and cycle (TIM3) in the calculation of the convolution layer. [Figure 6] FIG. 13 is a diagram showing the input state to the parallel computing unit in cycle (TIM5) and cycle (TIM6) in the calculation of the convolution layer. [Figure 7] FIG. 13 is a diagram showing the input state to the parallel computing unit in cycle (TIM7) and cycle (TIM8) in the calculation of the convolution layer. [Figure 8] FIG. 13 is a diagram showing the input / output state of the parallel computing unit in a cycle (TIM9) in the computation of the convolutional layer. [Figure 9] FIG. 13 is a diagram showing input / output data for the parallel computing unit in cycles (TIM1 to TIM36) in the computation of the convolutional layer. [Figure 10] FIG. 13 is a diagram showing, in the form of a determinant, a product-sum operation performed in the calculation of a fully connected layer. [Figure 11] FIG. 13 is a diagram showing the input state to the parallel computing unit 130 in cycle (TIM1) and cycle (TIM4) in the calculation of the fully connected layer. [Figure 12] FIG. 13 is a diagram showing the input state to the parallel computing unit 130 in cycle (TIM3) and cycle (TIM2) in the computation of the fully connected layer. [Figure 13] FIG. 13 is a diagram showing the input state to the parallel computing unit 130 in cycle (TIM5) and cycle (TIM6) in the calculation of the fully connected layer. [Figure 14] FIG. 13 is a diagram showing an input state to the parallel computing unit 130 in a cycle (TIM7) in the computation of the fully connected layer. [Figure 15] FIG. 15(a) is a schematic diagram of the calculation of the fully connected layer in the reference example, and FIG. 15(b) shows the calculation formula. [Figure 16] FIG. 16(a) is a schematic diagram of the calculation of the convolution layer in the reference example, and FIG. 16(b) shows the calculation formula. [Figure 17] 1 shows an example of the configuration of a product-sum calculation circuit using a systolic array as a reference example. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Embodiment 1 Before describing the arithmetic processing device of the first embodiment according to the present invention, we will provide an overview of the fully connected layer operations, which are often used in AI and are the subject of processing by the arithmetic processing device here, the product-sum operations in the convolutional layer operations, and the arithmetic circuit using a systolic array as a reference example.

[0010] Fig. 15(a) is a schematic diagram of the operation of the fully connected layer, and Fig. 15(b) shows the calculation formula. As shown in the figure, in the calculation of the fully connected layer, a multiply-and-accumulate operation is performed in which multiple input data (X1 to Xn) are multiplied by weight data (W11 to Wmn) prepared for each node (Y1 to Ym) in the fully connected layer in the same number as the input data, and all the results are added together. The multiply-and-accumulate operation here corresponds to the operation surrounded by the dotted line in the calculation formula in Fig. 15(b).

[0011] FIG. 16(a) is a schematic diagram of the convolution layer operation, and FIG. 16(b) shows the operation formula. As shown in the figure, in the convolution layer operation, input data (X001 to Xlkc) having width, height, and depth (= number of input channels (hereinafter referred to as CH)) is scanned and processed using box filters (filter 1 to filter m) for the depth of the output data (number of output CHs) to obtain an operation result. As shown in the operation formula, each box filter has a weight for width k, height l, and depth (number of input CHs), and a multiply-and-accumulate operation is performed in which input data (Xxyc) in a local range is multiplied by the corresponding weight (Wxycm) and all are added together. The multiply-and-accumulate operation here corresponds to the operation surrounded by the dotted line in the operation formula in FIG. 16(b).

[0012] As described above, both the fully connected layer and the convolutional layer operations execute many multiply-accumulate operations. Therefore, a processing device (circuit) is required that can speed up the processing by arranging many multiply-accumulators and executing the operations in parallel. In addition, since both the fully connected layer operations and the convolutional layer operations execute multiply-accumulate operations, a processing device (circuit) that can be shared is required to reduce the circuit scale (cost).

[0013] 17 shows an example of the configuration of a multiply-add circuit using a systolic array as a reference example. As shown in the figure, in the multiply-add circuit using a systolic array, the output of each multiply-adder is connected so as to be input to the adjacent multiply-adder (the multiply-adder on the right in the figure), and each multiply-adder is configured to perform a multiply-add operation in which the sum of the value input from the adjacent multiply-adder is taken for the product of the value of the weight data supplied from the weight data supply unit set for that multiply-adder and the value of the input data supplied from the input buffer.

[0014] The input timing of the input data is adjusted using an input buffer, and the inputs to the multiple multiplier-accumulators lined up on the left and right are staggered so that the input to the leftmost multiplier-accumulator is the first. This allows the multiplier-accumulator to be repeatedly executed from the leftmost multiplier-accumulator to the rightmost multiplier-accumulator, and the final result is obtained from the rightmost multiplier-accumulator.

[0015] In addition, because of this configuration, it is necessary to set weight data according to the arrangement of the multiply-and-accumulate units arranged side by side, and therefore it may not be possible to use all the arranged multiply-and-accumulate units. For example, when considering a case where a convolutional layer operation in a 3×3 filter (9 weight data) is performed using 4×4=16 multiply-and-accumulate units in FIG. 17, since 9 weight data are set, only 9 of the 16 multiply-and-accumulate units can be used. In addition, the input in this case needs to be a certain 3×3 range in the input data when viewed at a certain timing, and it is troublesome to rearrange the data from the original input data arrangement (raster arrangement).

[0016] Next, a description will be given of a processing device according to the first embodiment of the present invention. Fig. 1 is a configuration diagram of a processing device 100 according to the present embodiment.

[0017] As shown in the figure, the arithmetic processing device 100 is composed of an input data supply unit 110, a weight data supply unit 120, a parallel arithmetic unit 130 having a plurality of arithmetic units 150, a multiplexer 160, a row sum circuit 170, and an arithmetic control unit 180. The input data supply unit 110 and the weight data supply unit 120 correspond to a data supply unit.

[0018] The input data supply unit 110 and the weight data supply unit 120 are components that supply input data and weight data to be used in the fully connected layer operation in the second operation mode and the convolutional layer operation in the first operation mode to the parallel operation unit 130. The input data supply unit 110 can read data from an external memory in which the input data is stored, and the weight data supply unit 120 can read data from an external memory in which the weight data is stored. In addition, the operation for supply is started in response to an instruction from the operation control unit 180, and at that time, the method of supplying the input data and weight data to the parallel operation unit 130 is changed according to the selected operation mode.

[0019] The parallel computing unit 130 is a component that performs arithmetic processing in parallel using a plurality of computing units 150 based on the input data supplied from the input data supply unit 110 and the weight data supplied from the weight data supply unit 120, and executes some of the product-sum operations in the fully connected layer operations and the convolutional layer operations, or all of the product-sum operations.

[0020] FIG. 2 is a circuit diagram showing a configuration common to each arithmetic unit 150. As shown in FIG.

[0021] As shown in the figure, the calculator 150 is a multiply-accumulate calculator composed of flip-flops 151, 152, 156, a multiplier 153, an adder 154, and a multiplexer 155, which multiplies the input data and weight data input in synchronization with a clock, adds the resulting multiplied value to an augend, and outputs the result. The augend value is selected by the multiplexer 155 from the calculation result of the previous cycle stored in the flip-flop 156 or 0 (at reset). For example, when A and B are input as input data and C and D are input as weight data in sequence, OUT1=A×C+0 (reset) is executed in the first cycle, and OUT2=B×D+OUT1 is executed in the second cycle to obtain the result of A×C+B×D.

[0022] 1, the parallel computing unit 130 is configured by preparing a plurality of sets (stages) of computing unit groups 140, each of which is a set of a plurality of the above-mentioned computing units 150. The computing unit 150 in the computing unit group 140 corresponding to the top stage (first stage) is connected so that input data is directly input from the input data supply unit 110, while the computing units 150 in the next stage and onwards are connected so that input data input to the adjacent (previous stage) computing unit 150 above is input with a one-cycle delay. In addition, the input of weight data is connected so that it is directly supplied to each computing unit 150 from the weight data supply unit 120. The outputs of each computing unit 150 in each computing unit group 140 are all input to a multiplexer 160.

[0023] Regarding the output of the multiplexer 160, if the input timing to the multiple arithmetic units 150 of the first-stage arithmetic unit group 140 of the parallel arithmetic unit 130 is the same, then naturally the outputs of the multiple arithmetic units 150 of a certain group are all output at the same timing. Also, since the input to each group is delayed by one cycle, the output of each group is output with a shift of one cycle and without overlapping timing. Therefore, the multiplexer 160 can select the valid output of the arithmetic unit group 140 from the input operation results every cycle, and input it to the row sum-of-collection circuit 170 as the subsequent stage sum-of-collection unit. The multiplexer 160 and the row sum-of-collection circuit 170 correspond to the sum-of-collection unit.

[0024] The row sum circuit 170 is a part that sums up multiple calculation results inputted in each cycle and outputs the sum. Note that the operation of summing up multiple calculation results may not be performed depending on the calculation mode selected by the calculation control unit 180, and multiple calculation results may be output as they are.

[0025] In the above configuration, specific operations when performing the multiply-and-accumulate operation in the convolution layer operation and the fully connected layer operation will be described in detail using examples.

[0026] (About the execution of multiply-and-accumulate operations in convolutional layers) Fig. 3 is a diagram for explaining the execution of the product-sum operation in the convolution layer operation, Fig. 4 to Fig. 8 are diagrams showing the input state to the parallel computing unit 130 for each cycle in cycle (TIM1) to cycle (TIM9), and Fig. 9 is a diagram showing all the product-sum operation results in the convolution layer operation shown in the example of Fig. 3. Specific operations for the execution of the product-sum operation in the convolution layer operation will be described below with reference to Figs. 3 to 9.

[0027] As shown in Fig. 3, an example will be described in which input data with a width of 6, height of 6, and number of CHs (depth) of 1 is convolution-operated using four filters (filters 1-4) equal to the number of output CHs (1-4). The numbers attached to each element in the figure indicate the order in which the data is arranged. Figs. 4-8 show the input of input data and weight data in the above example to the parallel computing unit 130, assuming that the parallel computing unit 130 is composed of 4 x 4 computing units 150. Cycle (TIM1) to cycle (TIM9) in each figure show the input state to the parallel computing unit 130 for each cycle.

[0028] That is, for example, in the computing unit 150 shown in Fig. 4, the boxed part in the upper center corresponds to the input part of flip-flop 151 which inputs input data in Fig. 2 and displays the number of the input data input in each cycle, while the boxed part in the left center corresponds to the input part of flip-flop 152 which inputs weighting data in Fig. 2 and displays a filled-in pattern corresponding to the number of the weighting data of each filter input in each cycle. Furthermore, in the computing unit 150 shown in Fig. 8, the boxed part in the right center corresponds to the output part of flip-flop 156 which outputs output data in Fig. 2 and displays the number of the output data output in each cycle.

[0029] As shown in each of Figures 4 to 8, in the calculation of the convolutional layer, the calculation unit set 140 of each stage of the parallel calculation unit 130 corresponds to execute the calculation of each output CH of the convolutional layer, and the weight data supply unit 120 inputs one data in the filter corresponding to each stage here as common weight data to the multiple calculation units 150 of each stage set in each cycle, and the input data supply unit 110 inputs input data corresponding to the weight data collectively for each of the calculation units 150 existing in the first stage calculation unit set 140 in each cycle.

[0030] As shown in FIG. 4, in the first cycle (TIM1), the weighting data supplying unit 120 inputs data No. 1 of filter 1 as common weighting data for the first-stage arithmetic unit group 140, and the input data supplying unit 110 inputs four pieces of data No. 1 to No. 4 in FIG. 3, which correspond to the weighting data No. 1, as input data to the first-stage arithmetic unit group 140.

[0031] In the next cycle (TIM2), the weighting data supplying unit 120 inputs the second data of filter 1 as common weighting data for the first-stage arithmetic unit group 140, and the input data supplying unit 110 inputs the four pieces of data, numbered 2 to 5 in Figure 3, corresponding to the second weighting data, as input data to the first-stage arithmetic unit group 140.

[0032] Thereafter, in cycles (TIM3) to (TIM9), weight data and input data are input sequentially until the final weight data for filter 1, number 9, is input, thereby completing the input corresponding to the first output of the first-stage arithmetic unit group 140.

[0033] When the input is completed up to (TIM9), the input data is represented as A*, the weight data is represented as W* (* is a number indicating the order of the data), and the outputs of the calculators 150 of the first stage calculator group 140 are represented as Y1, Y2, Y3, and Y4 from the left. Y1=A1×W1+A2×W2+A3×W3+A7×W4+A8×W5 +A9×W6+A13×W7+A14×W8+A15×W9 Y2=A2×W1+A3×W2+A4×W3+A8×W4+A9×W5 +A10×W6+A14×W7+A15×W8+A16×W9 Y3=A3×W1+A4×W2+A5×W3+A9×W4+A10×W5 +A11×W6+A15×W7+A16×W8+A17×W9 Y4=A4×W1+A5×W2+A6×W3+A10×W4+A11×W5 +A12×W6+A16×W7+A17×W8+A18×W9 The output will be as follows.

[0034] Furthermore, the input data to the arithmetic unit sets 140 other than the first stage arithmetic unit set 140 is input with the input data input to the adjacent arithmetic unit set 140 above delayed by one cycle, so the weighting data supplying unit 120 similarly delays the weighting data to be input to the arithmetic unit sets 140 of the subsequent stages, and inputs the weighting data of the filters (filters 2 to 4) corresponding to the subsequent stages one by one, as in the first stage, for each cycle, thereby obtaining the calculation results of all output CHs (1 to 4).

[0035] Furthermore, as shown in FIG. 9, by changing the input data (data surrounded by dotted lines) in cycles (TIM10) to (TIM18), cycles (TIM19) to (TIM27), and cycles (TIM28) to (TIM36) and processing the remaining input data in the same manner, it is possible to obtain all the product-sum calculation results in the convolution layer calculations shown in the example of FIG. 3.

[0036] In addition, the row sum circuit 170 in the calculation of this convolution layer is set by the calculation control unit 180 to an operation mode in which the input data is output as is, without performing the calculation process of summing and outputting the sum of multiple calculation results inputted in each cycle.

[0037] (Performing multiplication and accumulation operations in the fully connected layer) Fig. 10 is a diagram showing the product-sum calculation performed in the calculation of the fully connected layer in the form of a determinant, and Figs. 11 to 14 are diagrams showing the input state to the parallel computing unit 130 for each cycle in cycle (TIM1) to cycle (TIM7). A specific operational example of the execution of the product-sum calculation in the calculation of the fully connected layer will be described below with reference to Figs. 10 to 14.

[0038] Fig. 10 shows, in the form of a determinant, the product-sum operation performed in a fully connected layer having 16 pieces of input data and 4 output nodes. The numbers attached to each element in the figure indicate the order in which the data is arranged. Figs. 11 to 14 show the input of input data and weight data to the parallel computing unit 130, assuming that the parallel computing unit 130 is composed of 4 x 4 computing units 150. Cycles (TIM1) to (TIM7) in each figure show the input state to the parallel computing unit 130 for each cycle.

[0039] That is, for example, in the computing unit 150 shown in Fig. 11, the boxed part in the upper center corresponds to the input part of flip-flop 151 which inputs input data in Fig. 2 and displays the number of the input data input in each cycle, while the boxed part in the left center corresponds to the input part of flip-flop 152 which inputs weight data in Fig. 2 and displays the number of the weight data input in each cycle. Furthermore, for example, in the computing unit 150 shown in Fig. 12, the boxed part in the right center corresponds to the output part of flip-flop 156 which outputs output data in Fig. 2 and displays the number of the output data output in each cycle.

[0040] As shown in each of Figures 11 to 14, in the calculations of the fully connected layer, a set of calculators 140 at each stage of the parallel calculator 130 corresponds to execute calculations for each output node of the fully connected layer, and the weight data supply unit 120 inputs weight data corresponding to each stage (output node) here to each of the multiple calculators 150 in each stage set in each cycle, and the input data supply unit 110 inputs input data corresponding to the weight data to the first stage set of calculators 140 in each cycle.

[0041] As shown in FIG. 11, in the first cycle (TIM1), the input data supply unit 110 and the weight data supply unit 120 each input four pieces of data No. 1 to No. 4 (FIG. 10) to the first-stage operator group 140 of the parallel operator .

[0042] In the next cycle (TIM2), the input data supply unit 110 and the weight data supply unit 120 each input data No. 5 to No. 8 to the first-stage arithmetic unit set 140, the input data No. 1 to No. 4 input to the first stage in the previous cycle is input to the second-stage arithmetic unit set 140, and the weight data supply unit 120 inputs the corresponding weight data No. 17 to No. 20 to the second-stage arithmetic unit set 140.

[0043] 12, in the next cycle (TIM3), the input data supply unit 110 and the weight data supply unit 120 respectively input the 9th to 12th data to the first-stage arithmetic unit set 140, the 5th to 8th input data input to the first stage in the previous cycle is input to the second-stage arithmetic unit set 140, and the weight data supply unit 120 inputs the corresponding 21st to 24th weight data to the second-stage arithmetic unit set 140. Similarly, the 1st to 4th input data and the 33rd to 36th weight data are input to the third-stage arithmetic unit set 140.

[0044] In the next cycle (TIM4) (FIG. 12), the input data supply unit 110 and the weight data supply unit 120 respectively input data No. 13 to No. 16, which are the last inputs to the first-stage arithmetic unit set 140, to the first-stage arithmetic unit set 140. The second-stage arithmetic unit set 140 receives input data No. 9 to No. 12 that was input to the first stage in the previous cycle, and the weight data supply unit 120 inputs the corresponding weight data No. 25 to No. 28 to the second-stage arithmetic unit set 140. Similarly, the third-stage arithmetic unit set 140 receives input data No. 5 to No. 8 and weight data No. 37 to No. 40, and the fourth-stage arithmetic unit set 140 receives input data No. 1 to No. 4 and weight data No. 49 to No. 52.

[0045] In the next cycle (TIM5) (FIG. 13), the input data Nos. 13 to 16 input to the first stage in the previous cycle, which will be the last input to the second-stage arithmetic unit set 140, are input to the second-stage arithmetic unit set 140, and the weight data supply unit 120 inputs the corresponding weight data Nos. 29 to 32 to the second-stage arithmetic unit set 140. Similarly, the input data Nos. 9 to 12 and the weight data Nos. 41 to 44 are input to the third-stage arithmetic unit set 140, and similarly, the input data Nos. 5 to 8 and the weight data Nos. 53 to 56 are input to the fourth-stage arithmetic unit set 140.

[0046] Similarly, in the next cycle (TIM6) (FIG. 13), the final input data of No. 13 to No. 16 and weight data of No. 45 to No. 48 are input to the third-stage arithmetic unit set 140, and the input data of No. 8 to No. 12 and weight data of No. 57 to No. 60 are input to the fourth-stage arithmetic unit set 140. Then, in the next cycle (TIM7) (FIG. 14), the final input data of No. 13 to No. 16 and weight data of No. 61 to No. 64 are input to the fourth-stage arithmetic unit set 140.

[0047] As described above, each calculator 150 performs multiply-add by multiplying the input data input to each calculator 150 by the weight data and adding the result of the multiplication to the result of the previous cycle held in each calculator 150. That is, if the input data is represented as A* and the weight data is represented as W* (* is a number indicating the order of the data), and the outputs of the calculators 150 in the first stage calculator set 140 are represented as Y1, Y2, Y3, Y4 from the left, and the outputs of the calculators 150 in the subsequent sets are represented as Y5, Y6, ..., the outputs of each calculator 150 in each stage are as follows.

[0048] Output of the first stage computing unit group 140: Y1=A1×W1+A5×W5+A9×W9+A13×W13, Y2=A2×W2+A6×W6+A10×W10+A14×W14, Y3=A3×W3+A7×W7+A11×W11+A15×W15, Y4=A4×W4+A8×W8+A12×W12+A16×W16, It becomes.

[0049] Output of the second stage calculator 140: Y5=A1×W17+A5×W21+A9×W25+A13×W29, Y6=A2×W18+A6×W22+A10×W26+A14×W30, Y7=A3×W19+A7×W23+A11×W27+A15×W31, Y8=A4×W20+A8×W24+A12×W28+A16×W32, It becomes.

[0050] Output of the third stage calculator 140: Y9=A1×W33+A5×W37+A9×W41+A13×W45, Y10=A2×W34+A6×W38+A10×W42+A14×W46, Y11=A3×W35+A7×W39+A11×W43+A15×W47, Y12=A4×W36+A8×W40+A12×W44+A16×W48, It becomes.

[0051] Output of the fourth stage arithmetic unit group 140: Y13=A1×W49+A5×W53+A9×W57+A13×W61, Y14=A2×W50+A6×W54+A10×W58+A14×W62, Y15=A3×W51+A7×W55+A11×W59+A15×W63, Y16=A4×W52+A8×W56+A12×W60+A16×W64, It becomes.

[0052] Here, the output of the parallel computing unit 130 corresponds to each node of the target fully connected layer, but since the computation results of the multiple computing units 150 in each computing unit set 140 are output individually, it is understood that it is necessary to sum the computation results of the multiple computing units 150 in each computing unit set 140 to obtain the final output. Therefore, by operating the row aggregation-sum circuit 170 provided in the subsequent stage of the parallel computing unit 130 to sum and output the multiple computation results, the output of the row aggregation-sum circuit 170 corresponding to each computing unit set 140 becomes as follows.

[0053] Node output of the row sum circuit 170 corresponding to the first stage operator group 140: OUT1=Y1+Y2+Y3+Y4 = A1 x W1 + A5 x W5 + A9 x W9 +A13×W13+A2×W2+A6×W6+A10×W10 +A14×W14+A3×W3+A7×W7+A11×W11 +A15×W15+A4×W4+A8×W8+A12×W12 +A16×W16

[0054] Node output of the row sum circuit 170 corresponding to the second stage operator group 140: OUT2=Y5+Y6+Y7+Y8 = A1 x W17 + A5 x W21 + A9 x W25 + A13 x W29 +A2×W18+A6×W22+A10×W26+A14×W30 +A3×W19+A7×W23+A11×W27+A15×W31 +A4×W20+A8×W24+A12×W28+A16×W32

[0055] Node output of the row sum circuit 170 corresponding to the third stage arithmetic unit set 140: OUT3=Y9+Y10+Y11+Y12 = A1 x W33 + A5 x W37 + A9 x W41 + A13 x W45 +A2×W34+A6×W38+A10×W42+A14×W46 +A3×W35+A7×W39+A11×W43+A15×W47 +A4×W36+A8×W40+A12×W44+A16×W48

[0056] Node output of the row sum circuit 170 corresponding to the fourth-stage arithmetic unit set 140: OUT4=Y13+Y14+Y15+Y16 = A1 x W49 + A5 x W53 + A9 x W57 + A13 x W61 +A2×W50+A6×W54+A10×W58+A14×W62 +A3×W51+A7×W55+A11×W59+A15×W63 +A4×W52+A8×W56+A12×W60+A16×W64 As described above, the final value of the product-sum operation in the exemplified fully connected layer operation can be obtained.

[0057] As described above, in the arithmetic processing device of this embodiment, by providing the row sum circuit 170 at the rear stage of the multiple arithmetic units 150, input data can be input without shifting the timing between adjacent arithmetic units 150, and the product-sum operation can be performed for both the convolutional layer operation and the fully connected layer operation. Furthermore, input can be made according to the original input data sequence for each of the convolutional layer operation and the fully connected layer operation.

[0058] In addition, by preparing multiple operator groups 140 each consisting of multiple operator groups 150 and corresponding each operator group 140 to an output CH of the convolutional layer or an output node of the fully connected layer, it is possible to make maximum use of the arranged operator groups 150 for processing.

[0059] Therefore, according to the arithmetic processing device of this embodiment, a buffer for delaying input data is not required, more of the arranged arithmetic units can be utilized, and there is no need to rearrange data from the original input data order in the calculation of the convolutional layer, so that it is possible to provide a arithmetic processing device (circuit) that has improved processing performance while keeping the circuit scale smaller than before. [Industrial Applicability]

[0060] The present invention is applicable to a processing unit for executing applications using AI at high speed. [Explanation of symbols]

[0061] 100 arithmetic processing device, 110 input data supply unit, 120 weight data supply unit, 130 parallel arithmetic unit, 140 arithmetic unit group, 150 arithmetic unit, 151 flip-flop, 152 flip-flop, 153 multiplier, 154 adder, 155 multiplexer, 156 flip-flop, 160 multiplexer, 170 row sum circuit, 180 arithmetic control unit.

Claims

1. In a processing device that executes a first operation mode and a second operation mode, a data supply unit that supplies input data and weight data based on the first operation mode and the second operation mode; a parallel computing unit including a plurality of computing unit sets each including a plurality of computing units, which executes a computation at different timings based on the input data and the weight data supplied by the data supply unit, and outputs a computation result; It has a collection and sum section, the plurality of arithmetic units are in the same arithmetic unit set, and execute arithmetic operations at the same timing based on the input data and the weight data supplied by the data supply unit, and output arithmetic results; The sum-sum unit outputs a sum of the results of operations executed and output at the same timing by the plurality of arithmetic units of the arithmetic unit set during execution of the second arithmetic mode.

13. A processing device comprising:

2. 2. The processor according to claim 1, wherein the first operation mode is a convolution layer operation mode, and the second operation mode is a fully connected layer operation mode.

3. 3. The processor according to claim 1, wherein said data supplying section comprises an input data supplying section for outputting said input data, and a weighting data supplying section for supplying said weighting data.

Images