Semiconductor device and data generation method used therein
The semiconductor device optimizes deep learning processing by parallel data output, selective product-sum calculations, and data compression to enhance processing speed and reduce area requirements, addressing the challenges of existing technologies.
Patent Information
- Application Number
- JP2020114592
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- Filing Date
- 2020-07-02
- Publication Date
- 2024-12-12
- Estimated Expiration
- 2040-07-02
AI Technical Summary
Existing semiconductor devices face challenges in reducing processing time and minimizing the exclusive area required for large-scale arithmetic operations, particularly in deep learning processing, due to the need to simultaneously read multiple weight data from local memories and the increase in multipliers with 8-bit width.
The semiconductor device employs a configuration with a memory that outputs multiple data in parallel, utilizes product-sum calculators for each data, and includes selectors to perform product-sum calculations between different data sets, while compressing weight data corresponding to zero values and using multi-bit or ternary data to reduce memory and calculator requirements.
This approach reduces processing time and minimizes the increase in exclusive area, enabling efficient deep learning operations by optimizing data handling and calculator usage.
Smart Images

Figure 0007598714000001 
Figure 0007598714000002 
Figure 0007598714000003
Abstract
Description
[Technical field]
[0001] The present invention relates to a semiconductor device, a data generation method used therein, and a control method thereof, and more particularly to a semiconductor device that performs arithmetic processing such as deep learning processing, a data generation method used therein, and a control method thereof. [Background technology]
[0002] For example, a semiconductor device that performs large-scale arithmetic processing such as deep learning processing is described in Patent Document 1. For example, in Figure 10 of Patent Document 1, a parallel arithmetic unit (121) is described for performing arithmetic processing required for deep learning processing. [Prior art documents] [Patent documents]
[0003] [Patent Document 1] JP 2019-219753 A Summary of the Invention [Problem to be solved by the invention]
[0004] In Patent Document 1, as shown in FIG. 10, data is supplied from a local memory (122) to a parallel arithmetic unit (121), and the parallel arithmetic unit (121) performs multiply-and-accumulate operations in parallel. That is, processing is performed in a Single Instruction Multiple Data (SIMD) format. The present inventors have studied ways to shorten the processing time using the technology described in Patent Document 1.
[0005] Fig. 28 is a block diagram showing the configuration of a parallel arithmetic circuit of a comparative example examined by the present inventors, and Fig. 29 is a diagram showing the operation of the parallel arithmetic circuit shown in Fig. 28.
[0006] In FIG. 28, mem_a to mem_n indicate local memories, and mul indicates a multiplier. AD_a to AD_n indicate arithmetic units that perform addition and accumulation. Taking the local memory mem_a as an example, weight data A, B, and C output from the local memory mem_a are supplied to the corresponding multipliers mul, and are multiplied by the input data a, b, and c by the corresponding multipliers mul. Each multiplication result is supplied to the corresponding arithmetic unit AD_a, where it is added. By repeating the above-mentioned calculation while changing the input data and weight data, a product-sum calculation of the input data and weight data is performed, and a matrix calculation as shown in FIG. 29 can be performed.
[0007] By adopting such a configuration, it is possible to reduce the processing time of the multiplication and accumulation operation. However, there is a problem that it is required to read out multiple weight data a, b, and c substantially simultaneously from each of the local memories mem_a to mem_n. In addition, there is a problem that the number of multipliers mul that perform multiplication with an 8-bit width increases, and the occupied area increases. [Means for solving the problem]
[0008] A semiconductor device according to an embodiment is as follows.
[0009] That is, the semiconductor device includes a memory that outputs a plurality of first data in parallel, a plurality of multiply-accumulate operators corresponding to the plurality of first data, and a plurality of selectors corresponding to the plurality of multiply-accumulate operators, to which a plurality of second data are supplied in parallel, and which select and output second data from the plurality of second data supplied according to additional information indicating the position of the second data to be operated on the first data by the corresponding multiply-accumulate operator in the plurality of second data. Each of the plurality of multiply-accumulate operators performs a multiply-accumulate operation between the first data different from each other in the plurality of first data and the second data output from the corresponding selector.
[0010] Other objects and novel features will become apparent from the description of this specification and the accompanying drawings. Effect of the Invention
[0011] According to one embodiment, it is possible to provide a semiconductor device capable of shortening processing time, a data generating method used therein, and a control method thereof. [Brief description of the drawings]
[0012] [Figure 1] 1 is a block diagram showing a configuration of a semiconductor device according to a first embodiment. [Diagram 2] FIG. 2 is a diagram illustrating an example of a neural network structure. [Diagram 3] FIG. 2 is a schematic diagram showing the flow of computational processing of a neural network. [Figure 4] FIG. 2 is a diagram for explaining processing in a parallel computing unit according to the first embodiment. [Diagram 5] FIG. 2 is a diagram for explaining processing in a parallel computing unit according to the first embodiment. [Figure 6] 1 is a block diagram showing a configuration of a parallel arithmetic circuit according to a first embodiment. [Figure 7] 1 is a diagram for explaining the principle of a parallel arithmetic circuit according to a first embodiment. [Figure 8] FIG. 2 is a diagram for explaining a compression method according to the first embodiment. [Figure 9] FIG. 2 is a diagram for explaining a parallel arithmetic circuit according to the first embodiment. [Figure 10] FIG. 2 is a diagram for explaining a parallel arithmetic circuit according to the first embodiment. [Figure 11] FIG. 2 is a diagram for explaining a parallel arithmetic circuit according to the first embodiment. [Figure 12] 4 is a state diagram illustrating the operation of the parallel arithmetic circuit according to the first embodiment. [Figure 13] 4 is a state diagram illustrating the operation of the parallel arithmetic circuit according to the first embodiment. [Figure 14] 4 is a state diagram illustrating the operation of the parallel arithmetic circuit according to the first embodiment. [Figure 15]4 is a state diagram illustrating the operation of the parallel arithmetic circuit according to the first embodiment. [Figure 16] FIG. 1 is a diagram showing an overall flow of deep learning processing in accordance with embodiment 1. [Figure 17] FIG. 1 is a diagram for explaining compression considered by the present inventors. [Figure 18] FIG. 13 is a diagram illustrating a compression method according to a first modification of the first embodiment. [Figure 19] FIG. 11 is a diagram illustrating a compression method according to a second modification of the first embodiment. [Figure 20] FIG. 11 is a diagram illustrating a compression method according to a third modification of the first embodiment. [Figure 21] FIG. 11 is a block diagram showing a configuration of a parallel arithmetic circuit according to a second embodiment. [Figure 22] FIG. 13 is a block diagram showing a configuration of a parallel arithmetic circuit according to a first modification of the second embodiment. [Figure 23] FIG. 11 is a block diagram showing a configuration of a parallel arithmetic circuit using ternary weights according to a second embodiment. [Figure 24] FIG. 11 is a block diagram showing a configuration of a parallel arithmetic circuit according to a third embodiment. [Diagram 25] FIG. 11 is a block diagram showing a configuration of a parallel arithmetic circuit according to a third embodiment. [Figure 26] FIG. 13 is a block diagram showing a configuration of a parallel arithmetic circuit according to a first modification of the third embodiment. [Figure 27] FIG. 13 is a block diagram showing a configuration of a parallel arithmetic circuit according to a fourth embodiment. [Figure 28] FIG. 1 is a block diagram showing a configuration of a parallel arithmetic circuit of a comparative example. [Figure 29] FIG. 29 illustrates the operation of the parallel arithmetic circuit illustrated in FIG. 28. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0013] Hereinafter, each embodiment of the present invention will be described with reference to the drawings. Note that the disclosure is merely an example, and those skilled in the art can easily come up with appropriate modifications while maintaining the gist of the invention, which are naturally included in the scope of the present invention. In addition, in order to make the explanation clearer, the drawings may show the width, number, shape, etc. of each part in a schematic manner compared to the actual embodiment, but these are merely examples and do not limit the interpretation of the present invention.
[0014] In addition, in this specification and each figure, elements similar to those described above with respect to the previous figures are given the same reference numerals, and detailed descriptions may be omitted as appropriate.
[0015] In the following, a semiconductor device suitable for deep learning processing will be described as an example. However, the deep learning processing is only an example, and the semiconductor device and the control method described below may be used for performing large-scale arithmetic processing or the like.
[0016] The means for solving the two problems described in the "Problems to be Solved by the Invention" section differ between embodiment 1 and embodiments 2 and 3, so an overview of each will be provided here.
[0017] <Embodiment 1> In the first embodiment, among the weight data arranged in a matrix, compression is performed such that data that has little effect during multiplication, for example, data corresponding to the numerical value 0, is omitted, and the data is stored in a local memory. As a result, the supply of weight data that has little effect during multiplication is omitted, and significant weight data is supplied to the parallel arithmetic circuit. That is, a plurality of significant weight data are supplied to the parallel arithmetic circuit substantially simultaneously. In addition, the parallel arithmetic circuit is provided with a selector that selects input data to be used for the multiply-and-accumulate operation from a plurality of input data. As a result, it is possible to provide a common multiplication unit for a plurality of input data, and it is possible to suppress an increase in the number of multiplication units and an increase in the area occupied by the parallel arithmetic circuit.
[0018] Although not particularly limited, for example, in learning for a sparse model, weight data often corresponds to the numerical value 0, and therefore the parallel arithmetic circuit according to the first embodiment is effective.
[0019] <Embodiments 2 and 3> In the second and third embodiments, as in the conventional technology, multi-bit data is used as the weight data arranged in a matrix. On the other hand, the value used in the calculation has a narrow bit width, and the multi-bit is a bundle of multiple narrow bits. This makes it possible to increase the number of weight data simultaneously output to the parallel calculation circuit while the local memory has the same weight data width as in the conventional technology. Also, the calculation unit that performs calculation between narrow bit width data (weight) and multi-bit width input data can have a smaller occupied area than a multiplication unit that uses multi-bit width data. Therefore, it is possible to suppress an increase in the occupied area of the parallel calculation circuit.
[0020] In the second embodiment, an example of narrow bit width data is described as binary data. In this case, the weight data has a narrow bit width of 1 bit. In the third embodiment, an example of narrow bit width data is described as ternary data. Since at least 2 bits are required to represent 3 values, the weight data has a narrow bit width of 2 bits.
[0021] Of course, in the second and third embodiments, the compression operation of the weight data as described in the first embodiment is not required.
[0022] In this way, in any of the first to third embodiments, it is possible to solve the two problems described above, and it is possible to shorten the processing time while suppressing an increase in the occupied area. Moreover, since this can be realized by simply adding a small-scale circuit to the configuration of the conventional technology, it is a form that can easily coexist with the conventional configuration.
[0023] Next, semiconductor devices according to first to third embodiments will be described.
[0024] (Embodiment 1) <Configuration of Semiconductor Device> Fig. 1 is a block diagram showing a configuration of a semiconductor device according to a first embodiment. In Fig. 1, 1 indicates a semiconductor device, and 2 indicates an array-type processor mounted on the semiconductor device 1. Also in the figure, 10 indicates a memory. The semiconductor device 1 is also mounted with circuit blocks other than the array-type processor 2 and the memory 10, but these are omitted in the figure.
[0025] The array type processor 2 comprises an array unit 12 involved in processing, a management unit 9, and a transfer path unit 11. The management unit 9 is supplied with a descriptor string from a memory 10, and manages the array unit 12 in accordance with the descriptor string. The transfer path unit 11 is connected between the array unit 12 and a bus B_W that transfers weight data etc., and transfers the weight data etc. to the array unit 12. The array unit 12 processes input data from the bus B_DI, and outputs the processed results to the bus B_DO as output data. When performing this processing, the array unit 12 uses the weight data etc.
[0026] <<Array section 12>> The array section 12 includes a number of circuit blocks, which will be described below. Although it is schematic, the layout of the circuit blocks in Fig. 1 is drawn to correspond to the layout on an actual semiconductor chip. However, the layout of the circuit blocks is not limited to this.
[0027] 1, the array unit 12 includes, as circuit blocks, parallel arithmetic circuits 3_0 to 3_8, memories (Mem) 4_0 to 4_7, DMACs (Direct Memory Access Controllers) 5_0 to 5_7, 7_0 to 7_7, and processor elements (hereinafter also referred to as PEs) 6_0 to 6_7. Furthermore, the array unit 12 includes, as circuit blocks, programmable switches (hereinafter also referred to as P switches) P_SW1 to P_SW3.
[0028] In the following description, when PE6_0 to PE6_7 are collectively described, the PE reference number is 6. When parallel arithmetic circuits 3_0 to 3_8 are collectively described, the parallel arithmetic circuits reference number is 3. Similarly, when DMACs are collectively described, the reference number is 5 or 7, the reference number is 4, and the reference number is P_SW.
[0029] The PE6 is a circuit block capable of realizing various functions. The functions realized by the PE6 are determined by a control signal from the management unit 9. The P switch P_SW also electrically connects between the designated circuit blocks by a control signal from the management unit 9. For example, a function of a specific PE6 is set to perform a specific operation by a control signal from the management unit 9. A specific P switch P_SW is also set to connect, for example, a specific parallel operation circuit 3 and a specific PE6 by a control signal from the management unit 9. Here, an example in which a plurality of PE6 are provided in the array unit 12 has been described, but the present invention is not limited to this.
[0030] For example, among PE6_0 to PE6_7, a specific PE may be changed to a dedicated circuit block with a predetermined function. In FIG. 1, the PE with reference number 6_0 is changed to a dedicated circuit block BN, and the PE with reference number 6_2 is changed to a dedicated circuit block that performs activation processing. Here, BN indicates a batch normalization function. In this way, by changing a general-purpose PE to a dedicated circuit block, it is possible to improve processing efficiency and speed.
[0031] The DMAC 5 transfers the input data to, for example, a predetermined parallel arithmetic circuit 3. The parallel arithmetic circuit 3 executes a multiply-and-accumulate operation in parallel between the weight data and the input data transferred by the DMAC 5. A function set in the PE 6 is applied to the operation result calculated by the parallel arithmetic circuit 3, and the result is output as output data by the DMAC 7.
[0032] 1, it is possible to shorten the distance between the parallel arithmetic circuits 3 by arranging the parallel arithmetic circuits 3 in the center of the array section 12. Therefore, the arrangement shown in Fig. 1 is suitable for cases where the arithmetic performance of the parallel arithmetic circuits 3 affects the performance of the application to be processed by the array-type processor 2.
[0033] <<Management unit 9, memory 10, transfer path unit 11 and parallel arithmetic circuit 3>> A plurality of descriptor strings are stored in the memory 10. Each of the descriptor strings includes information for specifying the function of the PE 6 and information for determining the state of the P switch P_SW. When the descriptor string is supplied to the management unit 9, the management unit 9 decodes the descriptor string, generates a corresponding control signal, and supplies it to the array unit 12 and the like. This allows the array unit 12 to perform processing according to the descriptor string stored in the memory 10 by the user. In other words, the internal connection and state of the array unit 12 are determined by the management unit 9. The descriptor may include information for the parallel arithmetic circuit 3.
[0034] The transfer path unit 11 includes a DMAC 8. The DMAC 8 transfers the weight data on the bus B_W to the parallel arithmetic circuit 3. As will be described later, the parallel arithmetic circuit 3 includes a local memory. The weight data is stored in this local memory. In the parallel arithmetic circuit 3, a multiply-and-accumulate operation is performed between the weight data and data based on the input data. The parallel arithmetic circuit 3 will be described in detail later with reference to the drawings, and will not be described further here. The memory 4 is used when the array unit 12 performs processing.
[0035] <Calculation processing in deep learning processing> Next, an example of implementing deep learning processing using the array-type processor 2 shown in Fig. 1 will be described. Here, the processing of a neural network that is repeated to perform deep learning processing will be described with reference to Figs. 1 to 3. Here, Fig. 2 is a diagram showing an example of a neural network structure. Fig. 3 is a schematic diagram showing the flow of computation processing of a neural network.
[0036] The array-type processor 2 shown in Figure 1 is suitable for convolution processing executed in deep learning processing. In convolution processing, the same weight data is used many times. In the array-type processor 2, the weight data is stored in local memory, making it possible to perform processing efficiently. Figure 2 shows an example of full connection, which is not convolution processing but does not reuse the same weight data. An outline of the arithmetic processing in a neural network will be explained using this Figure 2.
[0037] As shown in FIG. 2, neural network operations involve performing a product-sum operation to multiply input data by weight data w (w'), then performing operations such as activation on the result of the product-sum operation, and outputting the operation result.
[0038] Array-type processor 2 according to the first embodiment executes the procedure shown in Fig. 2 with a calculation process flow as shown in Fig. 3. In Fig. 3, reference numeral 13 denotes, although not limited to, an external memory provided outside array-type processor 2. Input data is stored in this external memory 13, and the calculation results obtained in array-type processor 2 are written back into external memory 13. Furthermore, weight data w(w') is stored in local memory 14 of parallel calculation circuit 3.
[0039] In step S1, the array unit 12 reads out input data (feature amounts) required for the calculation from the external memory 13. In step S2, the configuration and state of the array unit 12 are set by the management unit 9, and the feature amounts from the external memory 13 are sequentially supplied to the parallel arithmetic circuit 3 in step S3. The parallel arithmetic circuit 3 executes a product-sum calculation process in step S4 by multiplying the sequentially supplied feature amounts by weight data w(w') stored in the local memory 14 in the order of reception. The results of the product-sum calculation by the parallel arithmetic circuit 3 are sequentially output in step S5.
[0040] In step S6, the array unit 12 performs calculations such as addition and activation as necessary on the data obtained by the parallel arithmetic circuit 3. In step S7, the array unit 2 writes the result of the calculation as another feature in the external memory 13. This process realizes neural network processing, and by repeating this process, the calculations required for deep learning processing are performed.
[0041] In this way, in the array-type processor 2 according to the first embodiment, among the necessary arithmetic processes, regular product-sum arithmetic processes are executed by the parallel arithmetic circuit 3, thereby making it possible to realize high speed. Moreover, arithmetic processes other than the product-sum arithmetic processes are executed by the PE 6, the circuit of which can be dynamically reconfigured by the management unit 9 or the like. This makes it possible to flexibly set processes such as activation in each layer described as the first layer (first layer processing), the second layer (second layer processing), and the output layer (third layer processing) in FIG. 2 and FIG. 3. Furthermore, it is possible to use the number of parallel arithmetic circuits according to the necessary arithmetic processes from the multiple parallel arithmetic circuits 3 provided in advance in the array-type processor 2, and it is also possible to use the number of PEs, etc. according to the necessary arithmetic processes for the PE 6. In other words, it is possible to improve flexibility.
[0042] In FIG. 3, 3a to 3m indicate product-sum calculators included in one parallel arithmetic circuit 3 (eg, 3_0).
[0043] <<Multiply and add operation>> Next, a specific example of the product-sum operation executed by the parallel arithmetic circuit 3 will be described with reference to the drawings. Figures 4 and 5 are diagrams for explaining the processing in the parallel arithmetic unit according to the first embodiment. Here, Figure 4 shows a schematic diagram of a matrix arithmetic expression, and Figure 5 shows a matrix arithmetic expression corresponding to Figure 4.
[0044] In FIG. 4, In indicates matrix data composed of elements of k rows and m columns, and W indicates matrix data composed of elements of m rows and n columns. The elements of the matrix data In correspond to input data input to the parallel arithmetic circuit 3 (e.g., 3_0). The elements of the matrix data W are data stored in the local memory 14 in the parallel arithmetic circuit 3_0, and correspond to the weight data w(w') described in FIG. 2. In the example of FIG. 4, multiplication is performed between the elements of the matrix data In and the elements of the matrix data W, and a bias value C is added to each element of the multiplication result, and the result is output as matrix data Out composed of elements of k rows and n columns. The bias value C is also stored in the local memory 14.
[0045] A case where a multiply-and-accumulate operation is performed on matrix data In related to the first layer processing will be described. First, the matrix data W and bias value C corresponding to the matrix data In are transferred to the local memory 14 as initial values. Next, the first row of data of the matrix data In is read from the external memory 13 and supplied to the array unit 12. At this time, since the array unit 12 is in a configuration and state defined by the management unit 9, a predetermined process determined by the configuration and state of the array unit 12 is performed on the read first row of data, and the result of the predetermined process is supplied to the parallel arithmetic circuit 3 as the first row of data.
[0046] In the parallel arithmetic circuit 3, the bias value C from the local memory 14 is set as an initial value in the product-sum calculators 3a to 3n. After that, the product-sum calculators 3a to 3n perform a multiplication operation between the data in the first row supplied from the array unit 12 (the result of execution of a predetermined process) and the data in the first column of the matrix data W read out from the local memory 14.
[0047] 5 shows an example of a multiply-and-accumulate operation in which the number of columns (m) of the matrix data In is 20, and the number of columns (n) and rows (m) of the matrix data W is 20. To explain using the example of FIG. 5, data b0 to b 19 A predetermined process is executed by the array unit 12 on the matrix data In. The parallel arithmetic circuit 3 performs a multiply-and-accumulate operation on the result of this process with the matrix data W. Similarly, the parallel arithmetic circuit 3 performs a multiply-and-accumulate operation on the matrix data In for the second row and onward with the matrix data W. In the parallel arithmetic circuit 3, in the process of performing the multiplication operation, the results of the multiplication operation are accumulated, and a bias value C is also added. In this way, each element of the matrix data Out is calculated.
[0048] <<Configuration of parallel arithmetic circuit 3>> 6 is a block diagram showing the configuration of the parallel arithmetic circuit according to embodiment 1. The parallel arithmetic circuit 3 includes a local memory 14, product-sum operators 3a to 3m, a selector 15, and a latch circuit 19.
[0049] The latch circuit 19 is serially supplied with input data (feature amount) DI that is generated in the array unit 12 and is to be subjected to a multiplication and accumulation operation with weight data (first data). Although not particularly limited, one piece of input data DI is composed of 16 bits in parallel. A plurality of pieces of input data DI are sequentially supplied to the latch circuit 19 with 16 bits as one unit. The latch circuit 19 outputs four pieces of input data DI in parallel as data DL0 to DL3 (second data) based on a control signal SCC from a string of supplied input data DI. For example, the first supplied input data DI is data DL0, the second supplied input data DI is data DL1, the third supplied data DI is data DL2, and the fourth supplied data DI is data DL3, and is held as such.
[0050] When the input data DI is parallel data with a large bit width, the latch circuit 19 may divide the parallel data with a large bit width being supplied thereto into data DL0 to DL3 and output them in parallel.
[0051] When the control signal SCC changes next, the latch circuit 19 outputs and holds the fifth supplied input data DI as data DL0, the sixth supplied input data DI as data DL1, the seventh supplied data DI as data DL2, and the eighth supplied data DI as data DL3. Similarly, every time the control signal SCC changes, the latch circuit 19 outputs and holds the four sequentially supplied input data DI in parallel. In other words, the latch circuit 19 is a conversion circuit that converts a serial data string into parallel data.
[0052] Of course, the latch circuit 19 is not limited to this. For example, the number of data items output in parallel is not limited to four, but may be two or more.
[0053] The selector 15 is connected to the local memory 14, the latch circuit 19, and the product-sum calculators 3a to 3m. The selector 15 includes unit selectors 15a to 15m corresponding to the product-sum calculators 3a to 3m, respectively. The unit selectors 15a to 15m are commonly supplied with data DL0 to DL3 from the latch circuit 19. Each of the unit selectors 15a to 15m selects data specified by additional information (described later) ADa to ADm supplied from the local memory 14 from the data DL0 to DL3, and supplies the selected data to the corresponding product-sum calculator.
[0054] Although the local memory 14 is composed of one memory, the output is divided so as to correspond to the multiply-and-accumulate units 3a to 3m. In order to show that the output is divided into a plurality of parts, the local memory 14 is depicted in the same figure as being composed of unit local memories 14a to 14m. However, since the local memory 14 is composed of one memory, the words W0 to Wn are common to the unit local memories 14a to 14m. Therefore, for example, when the word W0 is selected, the corresponding weight data WDa to WDm are simultaneously read out from the unit local memories 14a to 14m. Here, an example in which the local memory 14 is composed of one memory will be described, but the present invention is not limited to this, and the local memory 14 may be composed of a plurality of memories. However, in this case, a configuration for managing the addresses of each memory is required, so it is preferable to compose the local memory 14 with one memory.
[0055] Further, the unit local memories 14a to 14m store additional information ADa to ADm, and when the weight data WDa to WDm are read out, the additional information ADa to ADm is also read out at the same time.
[0056] In the first embodiment, the product-sum calculator, the unit selector, and the unit local memory correspond one-to-one. For example, the product-sum calculator 3a, the unit selector 15a, and the unit local memory 14a correspond to one another. The additional information ADa read from the unit local memory 14a is supplied to the corresponding unit selector 15a. Data selected by the unit selector 15a from the data DL1 to DL4 according to the additional information ADa and the weight data WDa read from the unit local memory 14a are supplied to the product-sum calculator 3a. Although not particularly limited, the weight data WDa is also data in units of 16 bits.
[0057] The product-sum calculator 3a includes a multiplier (product calculator) 16, an adder 17, and a register 18. The data selected by the unit selector 15a and the weight data WDa are supplied to the product calculator 16, and a product operation is performed between these data. The product operation result calculated by the product calculator 16 is added to the data stored in the register 18 by the adder 17, and the addition result is stored in the register 18. The data stored in the register 18 is output as the product-sum operation result SRa. The register 18 is supplied with a numerical value 0 or the bias value C described in FIG. 4 as an initial value from the corresponding unit local memory 14a.
[0058] Although the product-sum calculator 3a and the corresponding unit selector 15a and unit local memory 14a have been described as an example, the same applies to the remaining product-sum calculators 3b to 3m, unit selectors 15b to 15m, and unit local memories 14b to 14m.
[0059] <<Operation of parallel arithmetic circuits>> <<<Principle>>> First, the principle of the parallel arithmetic circuit according to the first embodiment will be described with reference to the drawings. Fig. 7 is a diagram for explaining the principle of the parallel arithmetic circuit according to the first embodiment. Fig. 7 shows four unit local memories 14a-14d, unit selectors 15a-15d corresponding to these unit local memories, and product-sum operators 3a-3d in the parallel arithmetic circuit 3 shown in Fig. 6. Of the product-sum operators 3a-3d, only product operator 16 is shown in Fig. 7.
[0060] In the multiply-and-accumulate operation executed by the parallel arithmetic circuit 3, weight data serving as a multiplication coefficient is stored in the local memory 14, but the weight data may include data corresponding to the numerical value 0. Even if the multiply-and-accumulate operation is executed using weight data corresponding to the numerical value 0, the result of the multiply-and-accumulate operation does not change.
[0061] In the first embodiment, a process of compressing the weight data is performed before storing the weight data in the local memory 14. That is, in the first embodiment, the weight data corresponding to the numerical value 0 is omitted, and the weight data compressed by the omission is stored in the local memory 14. 7, weight data a0, b1, c2, and d3 that are not equal to the numerical value 0 are stored in the bottom row of the unit local memories 14a to 14d. Furthermore, the latch circuit 19 (FIG. 6) outputs data A to D as input data DL0 to DL3 at time t1, and outputs data E to H as input data DL0 to DL3 at time t2.
[0062] At time t1, the unit selectors 15a to 15d select data A to D as shown in Fig. 7 according to the additional information ADa to ADd supplied to the unit selectors 15a to 15d. That is, the unit selector 15a selects data A, the unit selector 15b selects data B, the unit selector 15c selects data C, and the unit selector 15d selects data D. Also, at time t1, the weight data a0, b1, c2, and d3 are read from the unit local memories 14a to 14d by selecting the word W0 (Fig. 6). As a result, the product calculator 16 in the product-sum calculators 3a to 3d simultaneously performs product calculations between the weight data a0, b1, c2, and d3 and the data A to D, and outputs product calculation results A*a0, B*b1, C*c2, and D*d3.
[0063] Next, when word W1 is selected, the next weight data can be read out from unit local memories 14a-14d, while data A-D can continue to be used since they are held by latch circuit 19. This causes product calculators 16 in product-sum calculators 3a-3d to perform multiplication operations in parallel between the next weight data and data A-D. As a result, product-sum calculators 3a-3d output product-sum calculation results for data A-D in parallel, based on two weight data, respectively.
[0064] At time t2, the input data DL0 to DL3 output from the latch circuit 19 is changed to data E to H. In this case, as in the case of time t1, the product-sum operation results of the changed input data and weight data are output from the product-sum operators 3a to 3d.
[0065] <<<Compression method (data generation method)>>> Next, a compression method for compressing weight data will be described with reference to the drawings. By compressing the weight data, it is possible to compress the consumption of the local memory 14 and shorten the processing time. FIG. 8 is a diagram for explaining the compression method according to the first embodiment. FIG. 8 shows a state before compression (before compression) and a state after compression (after compression) by the compression method according to the first embodiment. Here, a case where weight data (original data) is arranged in a matrix in the local memory 14 will be described as an example. Each of the unit local memories 14a to 14d constituting the local memory 14 constitutes a column of the matrix. Also, it is assumed that the unit local memory has eight words W0 to W7. As a result, as shown before compression, a matrix of eight rows and four columns is formed. 32 pieces of weight data including data corresponding to the numerical value 0 are arranged in the matrix. Here again, it is assumed that each weight data is composed of 16 bits.
[0066] In the compression method according to the first embodiment, data arranged in a matrix is omitted so that data corresponding to the numerical value 0 is reduced in an omission step. In the matrix after the omission, data arranged above in the same column is moved to fill the blanks created by the omission, thereby filling the blanks (compression step). That is, the data arranged above is moved toward the bottom row, and the data is packed toward the bottom row, so that the blanks created by the omission are eliminated.
[0067] The following is an example of the column corresponding to the unit local memory 14a. First, data corresponding to the numerical value 0 is arranged in words W0, W2, W4, and W6 (before compression). By omitting these data, words W0, W2, W4, and W6 become blank. In order to fill these blanks with data other than the numerical value 0, data arranged above in the same column is moved. That is, data a1 is moved to word W0. As a result, words W1 and W2 become blank, so data a3 arranged in word W3 is moved to word W1. As a result, words W2 to W4 become blank, so data a5 arranged in word W5 is moved to word W2. Finally, data a7 arranged in word W7 is moved to the blank word W3. Similarly, data is moved for the remaining unit local memories 14b to 14d, so that 32 pieces of weight data are arranged in 4 rows and 4 columns, as shown after compression.
[0068] Moreover, the matrix before compression is divided into two groups G1 and G2. In the first embodiment, the data supplied to the unit selector is changed according to the group. That is, in group G1, data A to D are supplied as input data DL0 to DL3 to be operated with weight data. In contrast, in group G2, data E to H are supplied as input data DL0 to DL3 to be operated with weight data. The position of the boundary line (dotted line) CHL between groups G1 and G2 is determined by the number of input data input to the parallel arithmetic circuit. In the example of FIG. 7, the input data is in units of four (A to D and E to H), so four rows are divided into one group.
[0069] After compression, group G1 becomes group G1a with two rows, and group G2 becomes group G2a with two rows. Note that groups G1 and G2, and groups G1a and G2a can be regarded as two matrices arranged on either side of boundary line CHL.
[0070] When compressing toward the bottom row, data is prevented from moving downward across groups. For example, in the unit local memory 14c, data c4 arranged in word W4 before compression can be moved to word W1 by compression. However, if data c4 is moved to word W1, the compressed data c4 will move to group G1a and will be calculated with any of data A to D. As a result, data c4 will be calculated as erroneous input data. In the first embodiment, data corresponding to the numerical value 0 is added to the part of word W1 in the unit local memory 14c of group G1a. As a result, the compressed data c4 will not cross the boundary line CHL and will be arranged in group G2a and will be calculated with any of data E to H.
[0071] From another perspective, it can be considered that compression is performed in units of groups G1 and G2 before compression to omit data corresponding to the value 0, and groups G1a and G2a after compression are generated.
[0072] <<<Additional information (first additional information)>>> Before compression, the weight row Wn in the memory 14 indicates which input data each weight is to operate with. If data corresponding to the numerical value 0 is omitted from the weight data for compression, the position of the weight data in the matrix after compression changes. Therefore, the correspondence between the compressed weight data and the input data to be operated on becomes unclear. In the first embodiment, information indicating which input data the compressed weight data is to operate with is generated and added as additional information in an additional information generating step. In the following description, this additional information is also referred to as first additional information.
[0073] Next, the first additional information will be described with reference to Fig. 9. Fig. 9 is a diagram for explaining the parallel arithmetic circuit according to the first embodiment.
[0074] The matrix before compression shown in Fig. 9 is the same as the matrix before compression shown in Fig. 8. In the matrix after compression shown in Fig. 9, compressed weight data and corresponding first additional information are arranged. In Fig. 9, each of the unit local memories 14a to 14d has two memory columns, one memory column (data memory column) stores weight data, and the other memory column (addition memory column) stores first additional information. Taking the unit local memory 14a as an example, one memory column is indicated by reference numeral 14ad, and the other memory column is indicated by reference numeral 14aa.
[0075] In the matrix before compression, weight data corresponding to the numerical value 0 is omitted from the weight data 0, a1, 0, a3, 0, a5, 0, a7 arranged in the column (unit local memory 14a). In this case, the operation target of the weight data 0 arranged in the lowest word W0 is the input data DL0 (FIG. 7). The operation target of the weight data a1 arranged in the next word W1 is the input data DL1, and the operation target of the weight data 0 arranged in the word W2 is the input data DL2. Moreover, the operation target of the weight data a3 arranged in the word W3 is the input data DL3. The weight data corresponding to the numerical value 0 is omitted, and the remaining weight data is stored in the data storage column 14ad, and the first additional information corresponding to the remaining weight data is stored in the additional storage column 14aa.
[0076] Before compression, the weight data 0, a1, 0, a3 arranged in words W0 to W3 are calculated on input data DL0, DL1, DL2, DL3, and the position information of the input data DL0 to DL3 is the first additional information. Here, the numbers 0 to 3 attached to the code DL correspond to the positions in the four input data. Specifically, the weight data a1 in word W1 before compression is calculated on input data DL1, and the position information is 1. The weight data a3 in word W3 before compression is calculated on input data DL3, and the position information is 3. In the additional memory array 14aa after compression, the position information 1 corresponding to the weight data a1 is stored in the word W0 portion.
[0077] Similarly, in the compressed additional memory column 14aa, the position information 3 corresponding to the weight data a3 is stored in the portion of the word W1. This defines the correspondence between the compressed weight data and the input data to be operated on. As a result, when the word W0 is selected after compression, the data a0 is read out from the data memory column 14ad as the weight data WDa, and the value 1 indicating the position is read out from the additional memory column 14aa as the additional information ADa.
[0078] Although the unit local memory 14a has been described as an example, the remaining unit local memories 14b to 14d are similar.
[0079] Here, the groups G1 and G1a have been described as examples, but the same applies to the groups G2 and G2a. In the case of the groups G2 and G2a, the words W4 to W7 before compression and the words W2 to W3 after compression can be read as the above-mentioned words W0 to W3 and words W0 to W1.
[0080] In FIG. 9, an example is shown in which a decimal number indicating a position is used as the additional information, but the present invention is not limited to this. For example, the additional information may be expressed in binary (bitmap format) instead of decimal, as shown in FIG. 10. In FIG. 10, the binary numbers "0, 1, 0, 1" indicating that the data a1 and a3 arranged in the unit local memory 14a are the second and fourth in the group G1 before compression are stored in the unit local memory 14a as the additional information. In this case, the first bit of the binary number indicates the first input data DL0, the next bit indicates the second input data DL1, the next bit indicates the third input data DL2, and the last bit indicates the fourth input data DL3. To indicate that the second and fourth input data DL1 and DL3 are to be calculated, the second and fourth bits are set to the logical value "1", and since the input data DL0 and DL2 are not to be calculated, the first and third bits are set to the logical value "0". Since the additional information can be represented in 4 bits, it is possible to reduce the memory capacity required to represent the additional information.
[0081] <<<Additional information regarding timing (second additional information)>>> When weight data for a plurality of rows is read and used while maintaining the input data DL0 to DL3, it is required to control the timing at which the input data is held by the latch circuit 19 (FIG. 6) in order to perform compression. In the first embodiment, this timing is controlled by additional information (hereinafter also referred to as second additional information). The second additional information will be explained with reference to the drawings. FIG. 11 is a diagram for explaining a parallel arithmetic circuit according to the first embodiment.
[0082] The matrices before and after compression shown in Fig. 11 are the same as those shown in Fig. 9. In Fig. 11, a register GT used to generate a control signal SCC supplied to the latch circuit 19 is shown.
[0083] Each of the groups G1 and G2 of the matrix before compression has four rows. Therefore, by holding the values of the input data DL0 to DL3 for four cycles, it is possible to perform the product-sum operation while changing the weight data in a state in which the data A to D are maintained as the input data DL0 to DL3. However, by performing the compression process, the number of rows in each of the groups G1a and G2a after compression is reduced to two rows, which is less than before compression.
[0084] In the first embodiment, the register GT includes a plurality of register units GT1, GT2 corresponding to the compressed groups. The number of rows of the corresponding groups G1a and G2a are set in the register unit GT1 corresponding to the group G1a and the register unit GT2 corresponding to the group G2a, respectively. Thereby, the control signal SCC maintains the input data DL0-DL3 being output for the number of cycles set in the register unit GT1. As a result, the multiply-accumulators 3a-3d execute a multiply-accumulator operation between the weight data sequentially supplied from the group G1a and the input data DL0-DL3 (i.e., data A-D) for two cycles. After two cycles, the multiply-accumulators 3a-3d execute a multiply-accumulator operation between the weight data from the group G2a and the input data DL0-DL3 (i.e., data E-H). It can be considered that the number of cycles or the period (GT1: first period, GT2: second period) during which the same input data is output is set in the register units GT1, GT2.
[0085] <<Operation of parallel arithmetic circuits>> Next, an example of the operation of the parallel arithmetic circuit 3 will be described with reference to the drawings. Figures 12 to 15 are state diagrams showing the operation of the parallel arithmetic circuit according to the first embodiment.
[0086] 12 to 15, reference numeral 14 denotes a compressed local memory. In these drawings, the weight data stored in the local memory 14 is the same as that in the compressed local memory shown in FIG.
[0087] FIG. 12 shows the state of the multiplication and accumulation operation in the first cycle. Since the register section GT1 is 2, data A to D are supplied in parallel to the unit selectors 15a to 15d as input data DL0 to DL3 during the second cycle. The unit selectors 15a to 15d are supplied with the first additional information ADa to ADd described in FIG. 9. As a result, the unit selector 15a selects data B, which is input data DL1 corresponding to position 1, since the additional information ADa is 1. Furthermore, the unit selector 15b selects data A, which is input data DL0 corresponding to position 0, and the unit selector 15c selects data D, which is input data DL3 corresponding to position 3. Furthermore, the unit selector 15d selects data A, which is input data DL0 corresponding to position 0. As a result, the multiplication operator 16 outputs the multiplication operation results B*a1, A*b0, D*c3, and A*d0 in parallel, as shown in FIG.
[0088] When the first cycle ends and the second cycle begins, the state of the parallel arithmetic circuit 3 changes as shown in FIG.
[0089] That is, word W1 is selected. Also, since register section GT1 is 2, data A to D are maintained as input data DL0 to DL3. By selecting word W2, unit selectors 15a to 15d select input data DL3 (data D), input data DL1 (data B), input data DL0 (data A), and input data DL2 (data C) in accordance with the first additional information shown in FIG. 9. As a result, product-sum calculators 3a to 3d output the sum of the product calculation result calculated in the first cycle and the product calculation result calculated in the second cycle, as shown in FIG.
[0090] When the second cycle ends and the third cycle begins, the state of the parallel arithmetic circuit 3 changes as shown in FIG.
[0091] That is, word W2 is selected. Furthermore, since two cycles indicated by GT1 have been completed, the next GT2 is selected, and since its value is 2, the latch circuit 19 outputs data E to H as the input data DL0 to DL3. By selecting word W2, the unit selectors 15a to 15d select input data DL1 (data F), input data DL0 (data E), input data DL0 (data E), and input data DL1 (data F) in accordance with the first additional information shown in FIG. 9. As a result, the multiply-add calculators 3a to 3d output the sum of the multiply-add calculations calculated in the third cycle to the result of the multiply-add calculations calculated in the second cycle, as shown in FIG. 14.
[0092] When the third cycle ends and the fourth cycle begins, the state of the parallel arithmetic circuit 3 changes as shown in FIG.
[0093] That is, word W3 is selected. Also, since the value of register section GT2 is 2, latch circuit 19 maintains output of data E to H as input data DL0 to DL3. By selecting word W3, unit selectors 15a to 15d select input data DL3 (data H), input data DL2 (data G), input data DL2 (data G), and input data DL3 (data H) in accordance with the first additional information shown in FIG. 9. As a result, multiply-add calculators 3a to 3d output the sum of the result of the multiply-add calculation calculated in the third cycle and the product calculation calculated in the fourth cycle, as shown in FIG.
[0094] In this manner, a multiply-and-accumulate operation is performed between the data A to H and the weight data from the local memory 14.
[0095] <Deep learning processing> Next, a deep learning process using the semiconductor device 1 according to the first embodiment will be described. As shown in FIG. 6, the parallel arithmetic circuit 3 in the semiconductor device 1 includes a local memory 14. Weight data used in the calculation of the deep learning process is compressed and stored in this local memory 14. The above-mentioned additional information is also stored in the local memory 14. In the first embodiment, the compression of the weight data and the generation of the additional information are performed in advance on a device (e.g., a computer) different from the semiconductor device 1 or on the cloud.
[0096] FIG. 16 is a diagram showing an overall flow of deep learning processing according to the first embodiment. In FIG. 16, 100 indicates a learning process, and 101 indicates a processing process of generating weight conversion and additional information for compressing weight data. The learning process 100 and the processing process 101 are performed in advance on a computer or a cloud. That is, in the learning process 100 on a computer or a cloud, original data on which weight data is based is generated, and this original data is provided as weight data to the processing process 101 on a computer or a cloud. The compressed weight data and additional information generated in this preliminary processing process (pre-processing process) are transferred in advance to the local memory 14 of the semiconductor device 1. In the semiconductor device 1, an inference process is performed by performing an arithmetic process between the compressed weight data stored in the local memory 14 and input data. Although not particularly limited, in the first embodiment, learning for a sparse model is performed in the learning process 100. Note that, when the capacity of the local memory 14 is small, the compressed weight data and additional information generated in the preliminary processing process may be transferred to an external memory connected to the bus B_W shown in FIG. 1, instead of the local memory 14.
[0097] Note that a part of the processing performed in the pre-processing step may be performed in the semiconductor device 1, not on a computer or on the cloud.
[0098] In Fig. 16, the process flow shows the process performed over time. A pre-processing step is performed at a time before time t3. The compressed weight data and additional information generated in the pre-processing step are transferred to and stored in the local memory 14 by the transfer path unit 11 shown in Fig. 1. After time t3, arithmetic processing is performed between the compressed weight data stored in the local memory 14 of the semiconductor device 1 and the input data, and inference is repeated.
[0099] <Variation 1> In Fig. 8, an example is shown in which weight data is compressed by omitting data corresponding to the numerical value 0, but this may not be effective in reducing processing time. An example in which the effect of reducing processing time is small will be described with reference to the drawings. Fig. 17 is a diagram for explaining the compression considered by the present inventor.
[0100] FIG. 17 is similar to FIG. 8. The difference is that the number of weight data corresponding to the numerical value 0 arranged in the columns (unit local memories) is not uniform before compression. For example, before compression, 0 weight data corresponding to the numerical value 0 is arranged in the unit local memory 14a of the group G1. In contrast, 3 weight data corresponding to the numerical value 0 are arranged in the unit local memories 14b to 14d. If the data corresponding to the numerical value 0 is omitted by compression, significant weight data is arranged only in the part of word W0 for the unit local memories 14b to 14c as shown after compression, but significant weight data is arranged in each of words W0 to W3 for the unit local memory 14a. As a result, in the product-sum operation, it is necessary to select words W0 to W3, and 4 cycles are required to perform the calculation of group G1, and the effect of reducing the processing time is not obtained.
[0101] Fig. 18 is a diagram showing a compression method according to Modification 1. In Fig. 18, the weight data before compression has one piece of data corresponding to the numerical value 0. Therefore, as shown in Fig. 8, even if the data corresponding to the numerical value 0 is omitted, eight cycles are required to perform the calculation.
[0102] In the first modification, a numerical value (threshold) smaller than a predetermined value is regarded as a numerical value 0. This allows data corresponding to a value smaller than the predetermined value to be omitted. In the case of performing a multiplication operation, a small numerical value has little effect on the result of the multiplication and accumulation operation, so the effect of regarding the small numerical value as a numerical value 0 is small.
[0103] In the first modification, as shown in the compression process, the numerical value 3 is set as the predetermined value. As a result, in the matrix in the compression process, the numerical portion filled with dots is regarded as the numerical value 0. Then, by omitting the data corresponding to the numerical value 0, it is possible to compress the number of rows of each of the groups G1a and G2a to 3 rows, as shown after compression. As a result, the processing time required for 8 cycles before compression can be reduced to 6 cycles, and the processing time can be shortened.
[0104] <Variation 2> In the first modified example, as can be seen from the compressed matrix, data corresponding to the numerical value 0 is not uniform across each column. For example, in the compressed group G1a shown in Fig. 18, three significant numerical values are arranged in the data storage column 14bd of the unit local memory 14b, and zero significant numerical values are arranged in the data storage column 14dd. The processing time is determined by the column in which the most significant numerical values are arranged, so that a processing time of three cycles is required.
[0105] In the second modification, a threshold is set for each column in the matrix before compression so that the number of zeros in each column is uniform for each group. Values below a set predetermined value are regarded as zeros and omitted. In other words, the threshold is changed for each column so that the number of significant weight data is the same for multiple columns in the same group. This makes it possible to reduce the number of cycles required for group calculations and shorten the processing time.
[0106] Next, an example of a compression method according to Modification 2 will be described with reference to the drawings. Fig. 19 is a diagram showing the compression method according to Modification 2.
[0107] The matrix of weight data before compression is the same as that shown in FIG. 18. Here, a predetermined value is set so that two weight data remain in each column of each group. For example, in the column corresponding to the unit local memory 14a in group G1, a numerical value 3 is set as a threshold value. Also, in the same group G1, a numerical value 5 is set as a threshold value in the column corresponding to the unit local memory 14b. As a result, as shown in the matrix in the compression process, in the column corresponding to the unit local memory 14a in group G1, the numerical values 1 and 3 are filled with dots, and two weight data, numerical values 4 and 9, remain. Also, in the column corresponding to the unit local memory 14b in group G1, two weight data, numerical values 6 and 8, remain. Similarly, a threshold value is set for each of the remaining columns, and compression is performed so that two weight data remain in each column. As a result, in the matrix after compression, as shown in FIG. 19, each of groups G1a and G2a has two rows, and calculation can be performed in four cycles, which can enhance the effect of reducing the processing time.
[0108] In addition, a new threshold may be set in the compressed matrix, and values smaller than this threshold may be regarded as the numerical value 0, and the data may be omitted, making calculation unnecessary, thereby achieving low power consumption.
[0109] <Variation 3> 20 is a diagram showing a compression method according to Modification 3. Modification 3 is a method in which importance is used for the number of lines to be left for each group. In the diagram, it is shown that, for group G1, the importance when 1 line is left is 12, when the second line is also left is 8, when the third line is also left is 5, and when the fourth line is also left is 1. Also, it is shown that, for group G2, the importance when 1 line is left is 15, when the second line is also left is 10, when the third line is also left is 9, and when the fourth line is also left is 3.
[0110] The importance of keeping this row can be determined based on the effect of keeping the most important values in the column in each group, starting from the most important. For example, in each group, the maximum value of each column can be calculated as the sum of all columns, and this can be used as the importance of keeping one row. Similarly, the sum of all columns of the second largest value in each column is used as the importance of keeping the second row. Looking at each column in the group, the importance of keeping each value can be shown as a representative value of the column, starting from the most important value. However, the method is not limited to the above-mentioned method of using the maximum value or calculating the sum of all columns, as long as it is a method of obtaining a representative value of a row that can show the value of increasing the number of rows to be kept.
[0111] In the matrix groups G1 and G1a shown in FIG. 20, weight data with a downward left diagonal line indicates the weight data with the largest value in the column in which that weight data is placed. Also, in the matrix groups G2 and G2a shown in FIG. 20, weight data with a downward right diagonal line indicates the weight data with the largest value in the column in which that weight data is placed, and weight data with a vertical line indicates the weight data with the second largest value in the column in which that weight data is placed. Also, in the matrix groups G2 and G2a shown in FIG. 20, weight data with a dot indicates the weight data with the third largest value in the column in which that weight data is placed, and blank weight data indicates the weight data with the smallest value in the column in which that weight data is placed. When compressing, you set how many lines to leave for the entire group. Lines with high importance will be left according to the set number of lines. For example, if you set it to leave four lines for the entire group, the candidates for importance are 15, 12, 10, and 9. In this case, one line (importance 12) will be left from group G1, and three lines (importance 15, 10, 9) will be left from group G2.
[0112] In order to leave one row from group G1, the largest weight data value is left from each column (unit local memories 14a to 14d) of group G1. That is, in the same column, the largest weight data value (weight data with a diagonal line rising to the right) is left, and the remaining weight data is regarded as having a value of 0 and omitted. Similarly, in group G2, three large weight data values are left from each column (unit local memories 14a to 14d). That is, in the same column, three large weight data values are left, and the remaining weight data value (blank weight data) is regarded as having a value of 0 and omitted. This makes it possible to perform calculations in four cycles using the most important ones.
[0113] By compressing, it is possible to generate a matrix after compression 1 or after compression 2. The matrix after compression 1 is generated by simply deleting less important parts from the matrix before compression. In other words, in one group, the mutual arrangement of weight data when viewed by row is the same as the arrangement of the matrix before compression.
[0114] In contrast, in the matrix after compression 2, the weight data in each group is rearranged in descending order of importance. Taking group G2 as an example, the weight data with the largest value in the same column is arranged in the lowest row, and the weight data with the smaller value is arranged in the upper rows. By arranging the most important data in the lowest row in this way, it is possible to suppress the impact of the calculation being stopped midway within a group.
[0115] When performing a product-and-sum operation using the compressed matrix 1, the mutual relationship of the arrangement of weight data in the same column is maintained, so the first additional information can use the additional information described in Figures 9 and 10. On the other hand, when performing a product-and-sum operation using the compressed matrix 2, it is necessary to specify the original position of each weight data as the first additional information, so the operation is performed using the additional information shown in Figure 9.
[0116] (Embodiment 2) In the first embodiment, a case has been described in which the weight data (e.g., WDa in FIG. 6) output from the unit local memory has the same bit width (16 bits) as the input data (e.g., DL0), and both the weight data and the input data are INT type or FP type and represent numerical values. In the second embodiment, the input data represents numerical values in INT type or FP type, as in the first embodiment, but the weight data is binary data representing multiple values. In the second embodiment, an example of a parallel arithmetic circuit is shown in which each bit of the weight data output from the unit local memory represents a binary weight.
[0117] FIG. 21 is a block diagram showing a configuration of a parallel arithmetic circuit according to the second embodiment. The parallel arithmetic circuit 3 is composed of a plurality of unit local memories 14a to 14m, a plurality of unit selectors 15a to 15m, and a plurality of sum calculators 31a to 31m. In FIG. 21, only one unit local memory 14a, a corresponding unit selector 15a, and a corresponding sum calculator 31a are depicted as representatives. The remaining unit local memories, unit selectors, and sum calculators are similar to those shown in FIG. 21. There are eight pieces of input data DL0 to DL7. At time t1, data A to H are simultaneously supplied to the unit selector 15a as input data DL0 to DL7. Following time t1, at time t2, data I to P are simultaneously supplied to the unit selector 15a as input data DL0 to DL7. To avoid complicating the drawing, a case will be described here where each of the input data DL0 to DL7 (A to P) is 8 bits wide, but the present invention is not limited to this, and the input data may be, for example, 16 bits wide.
[0118] The unit local memory 14a includes eight memory columns 14a0-14a7. Each of the memory columns 14a0-14a7 outputs 1-bit weight data. For example, when word W0 is selected, the weight data stored in the lowest row of the memory columns 14a0-14a7 are simultaneously read out as weight data WDa0-WDa7 and supplied to the unit selector 15a. Of course, the unit local memory 14a may be regarded as a memory that outputs eight 1-bit data.
[0119] The unit selector 15a functions as a unit calculator (unit multiplier) that performs calculations between values from the memory and values of input data. The unit selector (unit multiplier) 15a multiplies the values 14a0-14a7 from the memory by the values of the input data DL0-DL7. Since each value from the memory means +1 or -1, the unit selector outputs the input data value as is or with its sign inverted according to the value from the memory, which is equivalent to the result of the multiplication calculation. For this purpose, the unit selector is provided with a plurality of selector sections 15a0-15a7 corresponding to the memory rows 14a0-14a7 and a plurality of sign inverters 30a0-30a7. When the weight is 1 bit wide, it means that the input data is multiplied by +1 or -1, so the selector sections 15a0-15a7 input the input data DL0-DL7 as is and the input data value with its sign inverted by the sign inverters 30a0-30a7. Furthermore, weight data WDa0 to WDa7 are input to the selector, and depending on the value, either the input data itself or a sign-inverted value is selected, and a value obtained by multiplying the input data by the weight is output from the selector.
[0120] The sum calculator 31a includes an adder 17 and a register 18. The adder 17 adds the outputs from the selector units 15a0 to 15a7 and the output from the register 18, and supplies the addition result to the register 18. As a result, the product-sum calculation result is output from the register 18.
[0121] That is, eight pieces of weight data WDa0-WDa7 simultaneously output from the unit local memory 14a and eight pieces of data A-H simultaneously output as input data DL0-DL7 at time t1 are simultaneously calculated and added together. The results of the addition are accumulated to become a product-sum operation result.
[0122] According to the parallel arithmetic circuit 3 shown in Fig. 21, eight arithmetic operations can be executed simultaneously, so that the arithmetic speed can be improved by eight times, and the processing time can be shortened. Also, in the parallel arithmetic circuit 3 of Fig. 21, since the multiplication unit 16 as shown in Fig. 6 is not required, it is possible to suppress an increase in area.
[0123] <Variation 1> Fig. 22 is a block diagram showing the configuration of a parallel arithmetic circuit according to Modification 1. In Fig. 22, the number of input data is reduced to four, DL0 to DL3, compared to Fig. 21, and the internal configuration of the product calculation unit 15a is also reduced to four. In contrast, the unit local memory 14a includes eight memory columns 14a0 to 14a7, as in Fig. 21, and eight weight data are read out simultaneously by selecting a word (for example, W0). That is, the configuration is asymmetric in that the number of input data does not match the number of weight data read out from the unit local memory.
[0124] In the first modification, four read selectors 32_01 to 32_67 are added to the parallel arithmetic circuit 3. Each read selector corresponds to two memory columns, selects one weight data from two weight data read from the corresponding two memory columns according to a selection signal S32, and supplies the selected weight data to a corresponding selector unit. Taking the read selector 32_01 as an example, the read selector 32_01 corresponds to the memory columns 14a0 and 14a1. The read selector 32_01 selects read data from the weight data read from the memory columns 14a0 and 14a1 according to the selection signal S32, and supplies the selected weight data to the selector unit 15a0 as weight data WDa0.
[0125] In the first modification, the unit local memory 14a is read once while two addition operations are performed by the adder 17. In other words, if one addition operation is regarded as one cycle, the unit local memory 14a is read once every two cycles.
[0126] The selection signal S32 changes so as to specify a different memory column for each cycle. As a result, the read selector 32_01 selects, for example, the weight data read from the memory column 14a in the first cycle, and selects the weight data read from the memory column 14a in the second cycle. An operation is performed between the selected weight data and the input data DL0 in the same manner as described in FIG. 21.
[0127] In the first modification, eight pieces of input data are divided into four pieces of input data, and the multiply-and-accumulate operation is performed. Therefore, the operation speed is four times faster than that of Fig. 21. However, it is possible to reduce the number of selector sections and adders and reduce the number of sum operation processes simultaneously executed by the summation unit 31, and therefore it is possible to suppress an increase in area.
[0128] (Embodiment 3) In the third embodiment, an example of a parallel arithmetic circuit is shown in which each bit of weight data output from a unit local memory represents a ternary weight as a multi-value, and input data is an INT type or FP type and represents a numerical value. To represent three values, the weight data is two bits wide. Of course, three values is just an example, and it may be four or more values.
[0129] <If the input data is serial> First, an example in which input data A to H, each of which is 8 bits wide, is sequentially supplied to a parallel arithmetic circuit to execute a product-sum operation will be described with reference to the drawings. Fig. 23 is a block diagram showing the configuration of a parallel arithmetic circuit using ternary weights.
[0130] In the first cycle, when data A is supplied to the parallel arithmetic circuit, 2-bit-wide weight data indicating a ternary weight is read from each of the memory columns 14a0 to 14a3 of the unit local memory 14a. The selector 33a selects one weight data from the four weight data read. In FIG. 23, the weight data "a0" read from the memory column 14a0 is selected. A multiplication operation with the data A is performed according to the selected weight data, and a multiplication operation result A*a0 is output. Similarly, 2-bit-wide weight data indicating a ternary weight is read from each of the memory columns 14b0 to 14b3 of the unit local memory 14b. A multiplication operation is performed between the selected weight data "b0" and the data A, and a multiplication operation result A*b0 is output.
[0131] Next, when the data changes from A to B (second cycle), the selectors 33a and 33b select the weight data "a1" and "b1" read from the memory columns 14a1 and 14b1. A multiplication operation is performed between this weight data and data B, and the weight data is added to the previously obtained multiplication operation result. As a result, the multiplication and accumulation operation result corresponding to the unit local memory 14a becomes A*a0+B*a1. When the data changes to C (third cycle), the weight data a2 and b2 read from the memory columns 14a2 and 14b2 are selected. Next, when the data changes to D (third cycle), the weight data a3 and b3 read from the memory columns 14a3 and 14b3 are selected. This causes a multiplication and accumulation operation to be performed between the weight data and data A to D. That is, the weight data simultaneously read from the unit local memory is packed with four 2-bit weight data. These four packs are sequentially operated between the data A to D.
[0132] By storing weight data representing three values as the weight data stored in the unit local memory, it is possible to reduce the capacity of the local memory and reduce power consumption. However, when performing a multiply-and-accumulate operation on data A to D, four cycles are required, making it difficult to improve the operation speed.
[0133] <Parallel Arithmetic Circuit According to Third Embodiment> Next, the configuration of a parallel arithmetic circuit according to the third embodiment will be described with reference to the drawings. Figures 24 and 25 are block diagrams showing the configuration of a parallel arithmetic circuit according to the third embodiment.
[0134] The parallel arithmetic circuit 3 according to the third embodiment includes a local memory, a selector unit, a sign inverter, and an adder. As shown in FIG. 6, the local memory includes a plurality of unit local memories. In FIG. 24, only the unit local memories 14a and 14b are illustrated among the plurality of unit local memories included in the local memory. In addition, in FIG. 24, only the selector units 15a0-15a3 and 15b0-15b3 corresponding to the unit local memories 14a and 14b and the sign inverters 30a0-30a3 and 30b0-30b3 corresponding to the unit local memories 14a and 14b are illustrated as representatives. Furthermore, as for the adder, only the adder units 31a and 31b corresponding to the unit local memories 14a and 14b are illustrated in FIG. 24 as representatives.
[0135] Although the unit local memory 14a and its corresponding parts will be described as an example, the remaining unit local memories and their corresponding parts are similar.
[0136] The unit local memory 14a includes memory columns 14a0 to 14a3. When common words W0 to W2 are selected in the local memory 14, weight data WDa0 to WDa3 are simultaneously read out from portions (2-bit memory cells) connected to the selected words in each of the memory columns 14a0 to 14a3. Here, each of the weight data WDa0 to WDa3 is 2-bit data indicating a ternary weight.
[0137] The sign inverters 30a0 to 30a3 are supplied with the input data DL0 to DL3. The sign inverters 30a0 to 30a3 generate and output values by inverting the signs of the supplied input data DL0 to DL3.
[0138] The selector units 15a0-15a3 are supplied with the corresponding input data DL0-DL3, the input data multiplied by -1 generated by the sign inverters 30a0-30a3, and fixed data corresponding to the numerical value 0. The selector units 15a0-15a3 select data from the input data supplied, the input data multiplied by -1, and the fixed value 0 according to the corresponding weight data WDa0-WDa3, and output the data to the corresponding sum calculator 31a. This means that the input data is multiplied by +1, -1, or 0 according to the weight value. Taking the memory row 14a0 as an example, the corresponding sign inverter 30a0 and selector unit 15a0 are supplied with the input data DL0. The selector unit 15a0 selects from the input data DL0, its sign-inverted value, and the fixed data 0 according to the weight data WDa0 from the memory row 14a0, and outputs the data to the sum calculator 31a.
[0139] Like the adder explained in FIG. 21, the adder 31a includes an adder 17 and a register 18, and adds up the data supplied from the selector units 15a0 to 15a3, and outputs the result as a product-sum operation result.
[0140] In the first cycle, at time t1, data A to D are supplied to the parallel arithmetic circuit 3 as input data DL0 to DL3. In this first cycle, word W0 is selected. As a result, 2-bit weight data "a0, a1, a2, a3" are simultaneously read out from unit local memory 14a as four weight data WDa0 to WDa3. At this time, 2-bit weight data "b0, b1, b2, b3" are simultaneously read out from unit local memory 14b as four weight data WDb0 to WDb3.
[0141] The selector units 15a0-15a3 select data according to the supplied weight data "a0, a1, a2, a3" and output the selected data to the adder 31a. As a result, the adder 31a outputs A*a0+B*a1+C*a2+D*a3, which is the result of a multiply-and-accumulate operation between the input data A-D and the weight data "a0, a1, a2, a3". Similarly, in the first cycle, the adder 31b outputs A*b0+B*b1+C*b2+D*b3, which is the result of a multiply-and-accumulate operation between the input data A-D and the weight data "b0, b1, b2, b3".
[0142] The state of the second cycle is shown in FIG. 25. In the second cycle, data E to H are supplied to the parallel arithmetic circuit 3 as input data DL0 to DL3. Also, in the second cycle, word W1 is selected. As a result, 2-bit weight data "a4, a5, a6, a7" are simultaneously read out from the unit local memory 14a as weight data WDa0 to WDa3. As a result, arithmetic is performed between the data E to H and the weight data "a4, a5, a6, a7", and the sum of the data and the previous arithmetic result is output from the sum calculator 31a as shown in FIG. 25.
[0143] Similarly, in the second cycle, 2-bit weight data "b4, b5, b6, b7" are simultaneously read out from unit local memory 14b as weight data WDb0 to WDb3. As a result, operations are performed between input data E to H and weight data "b4, b5, b6, b7", and sum of the previous operation result and the input data E to H is output from summation unit 31b as shown in FIG.
[0144] According to the third embodiment, it is possible to reduce the number of cycles necessary for the calculation to two cycles, and it is possible to improve the calculation speed.
[0145] <Variation 1> 26 is a block diagram showing the configuration of a parallel arithmetic circuit according to Modification 1. Modification 1 provides a parallel arithmetic circuit for the case where the number of weight data simultaneously read out from the unit local memory is different from the number of input data.
[0146] FIG. 26 is similar to FIG. 25. The difference is that there are eight signal wiring groups (L0 to L7) that transmit input data. In the first modification, only four signal wiring groups L0 to L3 out of the eight signal wiring groups L0 to L7 are used, and the remaining four signal wiring groups L4 to L7 are unused. Data A to D and E to H are supplied to the four signal wiring groups L0 to L3 in a time-division manner. That is, at time t1, data A to D are supplied as input data DL0 to DL3 via the signal wiring groups L0 to L3, and at time t2, data E to H are supplied as input data DL0 to DL3 via the signal wiring groups L0 to L3. This makes it possible to increase the calculation speed even when the number of weight data simultaneously read out from the unit local memory is different from the number of input data and is asymmetric.
[0147] (Embodiment 4) Fig. 27 is a block diagram showing the configuration of a parallel arithmetic circuit according to the fourth embodiment. The parallel arithmetic circuit according to the fourth embodiment includes an arithmetic unit that performs an arithmetic operation with a narrow bit width and a conventional parallel arithmetic unit, and these two arithmetic units are used exclusively. Here, an example will be described in which the parallel arithmetic circuit shown in Fig. 21 is used as the arithmetic unit that performs an arithmetic operation with a narrow bit width, but the present invention is not limited to this.
[0148] In FIG. 27, the unit local memory 14a, the unit selector (unit product calculator) 15a and the sum calculator 31a are the same as those shown in FIG.
[0149] In Fig. 27, mul indicates an 8-bit parallel arithmetic unit (parallel arithmetic unit). In Fig. 27, 1-bit data is bundled and supplied from each of the memory columns 14a0-14a7 of the unit local memory 14a to the parallel arithmetic unit mul. The parallel arithmetic unit mul executes multiplication operations in parallel between 8 bits in the input data DL0 and the 8-bit data from the unit local memory 14a, and supplies the operation result to an adder 31a.
[0150] Although not particularly limited, the summation unit 31a selects the product operation result from the unit selector 15a or the product operation result from the parallel operation unit mul, accumulates it, and outputs it.
[0151] In other words, the parallel arithmetic circuit of embodiment 4 has a mode that uses the conventional parallel arithmetic unit mul and a mode that uses a narrow bit-width arithmetic unit, and it is possible to select these two modes to be used exclusively.
[0152] In Fig. 27, an example is shown in which the 8-bit width output from the unit local memory 14a is divided into 8 (n) pieces of data each having a 1-bit width, but this is not limiting. For example, the 8-bit width may be divided into n / 2 (=4) pieces, and four pieces of data each having a 2-bit width may be used. In Fig. 27, the unit selector 15a and the sum calculator 31a are drawn relatively large, but in reality, they can be configured with a circuit that occupies a relatively small area.
[0153] Additional Notes This specification discloses various inventions in addition to the inventions described in the claims. Representative inventions not described in the claims are listed below. A. A memory that outputs a plurality of first data in parallel, each of which represents a multi-value; a multiplication unit that receives input data corresponding to the plurality of first data and different from each other and performs a multiplication operation between the corresponding first data and the input data; a calculator that adds and accumulates a plurality of data output from the multiplication unit and outputs a product-sum calculation result; A semiconductor device comprising: A1. A semiconductor device in which the product calculation unit described in A above selects, in accordance with first data, any one of input data, predetermined fixed data, and data obtained by processing the input data using a selector. A2. The semiconductor device according to A, The selector unit is further supplied with data generated by executing a predetermined operation on input data. A3. The semiconductor device according to A2, The first data is one-bit data representing a binary value. A4. The semiconductor device according to A2, The first data is two-bit data representing three values. B. A parallel arithmetic circuit comprising a memory capable of supplying an individual value to each of a plurality of arithmetic units every cycle, an external input capable of supplying an individual value different from the individual values to each of the arithmetic units every cycle, and a mechanism for combining outputs from the plurality of arithmetic units into one, wherein a plurality of the above units are arranged in parallel and the data supply from the external input is common to all units. B1. In the parallel arithmetic circuit described in B, A parallel arithmetic circuit, wherein each value supplied from said memory to said arithmetic unit is 1 or 2 bits wide. B2. In the parallel arithmetic circuit described in B, A parallel arithmetic circuit, wherein the arithmetic unit performs an operation equivalent to multiplication, and the mechanism for combining into one is an accumulation mechanism. C. A parallel arithmetic circuit that divides an n-bit wide value from memory into n 1-bit wide parts within one unit and performs n parallel arithmetic operations within the unit using these n parts. C1. A parallel arithmetic circuit that performs 1 parallel operation using an n-bit width value from memory within one unit and can select any of the above Cs. D. A parallel arithmetic circuit that can divide an n-bit value from memory into m 1-bit pieces using m bits within one unit, and perform m parallel intra-unit calculations using these m pieces. D1. In the parallel arithmetic circuit described in D, A parallel arithmetic circuit, where n>m and m is a divisor of n. D2. A parallel arithmetic circuit capable of performing 1 parallel operation using an n-bit width value from memory within one unit, and of selecting any of the above Cs. E. A parallel arithmetic circuit that divides an n-bit value from memory into n / 2 2-bit widths within one unit and performs an intra-unit calculation in n / 2 parallel fashion using these n / 2 pieces. E1. A parallel arithmetic circuit that performs 1 parallel operation using an n-bit width value from memory within one unit and can select any of the above E. F. A parallel arithmetic circuit that, within one unit, can divide an n-bit value from memory into m / 2 2-bit widths using m bits of the value, and perform intra-unit arithmetic in m / 2 parallel using these m / 2 units. F1. In the parallel arithmetic circuit described in F, A parallel arithmetic circuit, where n>m and m is a divisor of n. F2. A parallel arithmetic circuit that performs 1 parallel operation using an n-bit width value from memory within one unit and can select any of the above Fs.
[0154] The invention made by the present inventor has been specifically described above based on the embodiment, but it goes without saying that the present invention is not limited to the above embodiment and can be modified in various ways without departing from the gist of the invention. For example, although an example has been shown in which the weight data is supplied from a local memory to the parallel arithmetic circuit, the present invention is not limited to this. Similarly, the source of the input data is not limited. [Explanation of symbols]
[0155] 1 Semiconductor device 2. Array-type processor 3, 3_0~3_8 parallel calculation circuit 3a~3m Multiply-and-accumulate unit 14 Local Memory 14a~14m Unit local memory 15 Selectors 15a~15m Unit Selector 19 Latch Circuit ADa~ADm Additional information DL0~DL7 Input data WDa~WDm weight data
Claims
1. a memory for outputting, in parallel, a plurality of weight data and a plurality of pieces of additional information indicating a correspondence relationship between the plurality of weight data and a plurality of pieces of input data; a plurality of multiply-add calculators corresponding to the plurality of weight data; a plurality of selectors corresponding to the plurality of multiply-add calculators, the plurality of input data being supplied in parallel, and selecting and outputting input data specified by the additional information from the plurality of input data; Equipped with The semiconductor device, wherein each of the plurality of multiply-accumulate units performs a multiply-accumulate operation between mutually different weight data in the plurality of weight data and input data output from a corresponding selector.
2. 2. The semiconductor device according to claim 1, the memory stores the plurality of weighting data and the plurality of additional information, and the memory continuously outputs the plurality of weighting data and the plurality of additional information, the plurality of weighting data being different from each other, during a first period; The plurality of input data are continuously supplied to the plurality of selectors during the first period.
3. 3. The semiconductor device according to claim 2, The memory stores a plurality of weight data compressed so that data corresponding to the numerical value 0 is reduced.
4. 4. The semiconductor device according to claim 3, In a second period following the first period, the selectors are supplied with a plurality of input data different from the plurality of input data supplied in the first period.
5. A data generation method for generating a plurality of weight data to be used in a parallel arithmetic circuit comprising: a plurality of multiply-accumulators corresponding to a plurality of weight data output in parallel from a memory; and a plurality of selectors corresponding to the plurality of multiply-accumulators, to which a plurality of input data are supplied in parallel, for selecting input data from the plurality of input data that is specified by additional information indicating a correspondence between the plurality of weight data and the plurality of input data, and outputting the input data to a corresponding multiply-accumulator, the method comprising: an omission step of omitting original weight data corresponding to a predetermined value from a plurality of original weight data arranged in columns in the original weight data arranged in a matrix; a compression step of compressing original weight data in a higher row arranged in the same column as the original weight data omitted in the omission step in place of the omitted original weight data; an additional information generating step of generating additional information indicating a correspondence between the original weight data that was not omitted in the omission step and the input data to be calculated in the plurality of input data; Equipped with A data generating method for generating a plurality of weight data from a plurality of original weight data arranged in the same row.
6. 6. The data generating method according to claim 5, A data generating method, wherein the predetermined value is the numerical value 0.
7. 6. The data generating method according to claim 5, A data generation method, in which, among the original weight data arranged in a matrix, a small value among the original weight data arranged in the same column is set as a threshold, and a value smaller than the threshold is set as the specified value.
8. 8. The data generating method according to claim 7, A data generating method, wherein the threshold value in each column is set so that the number of original weight data arranged in each column is equal in the original weight data arranged in a matrix.
9. 6. The data generating method according to claim 5, a plurality of matrices are arranged with the original weight data, and each time the plurality of input data supplied to the selector changes, the plurality of weight data are supplied from different matrices to the plurality of multiply-accumulate operators; an importance setting step of setting an importance for each row in each of the plurality of matrices; a row number setting step of determining the number of rows to be left for each of the plurality of matrices based on the importance set in the importance setting step in accordance with a designated value that designates the number of rows to be left; A data generating method, wherein the predetermined value is set for each column of a matrix such that the number of rows determined in the row number setting step remains.
Citation Information
Patent Citations
Neural network circuit device, neural network, neural network processing method, and neural network execution program
JP2019200553A
Semiconductor device and control method thereof
JP2019219753A
Neural network processing device, and neural network processing method
JP2020060967A
Convolution processing method, device, and storage medium for convolutional neural networks
JP2022502747A
Instruction set architecture for a vector computational unit
US20190235866A1
