Neural network circuit

Abstract

We provide high-performance neural network circuits and the like that can be embedded in embedded devices such as IoT devices. [Solution] The neural network circuit has a plurality of neural network computation cores, each having a convolution operation circuit that performs convolution operations and a quantization operation circuit that performs quantization operations, and the plurality of neural network computation cores are connected in a way that enables data input and output.

Description

Technical Field

[0001] The present invention relates to a neural network circuit and a neural network operation method. This application claims priority based on Japanese Patent Application No. 2022-008692 filed in Japan on January 24, 2022, and incorporates its content herein by reference.

Background Art

[0002] In recent years, a convolutional neural network (CNN) has been used as a model for image recognition and the like. A convolutional neural network has a multi-layer structure including a convolutional layer and a pooling layer, and requires a large number of operations such as convolutional operations. Various operation methods for accelerating the operations by a convolutional neural network have been devised (such as Patent Document 1).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] On the other hand, it is desired to realize image recognition and the like using a convolutional neural network in embedded devices such as IoT devices. In an embedded device, it is difficult to incorporate a large-scale dedicated circuit described in Patent Document 1 or the like. Further, in an embedded device with limited hardware resources such as a CPU and a memory, it is difficult to realize sufficient operation performance of a convolutional neural network only by software.

[0005] Based on the above circumstances, the present invention aims to provide a high-performance neural network circuit and neural network computation method that can be embedded in embedded devices such as IoT devices. [Means for solving the problem]

[0006] To solve the above problems, this invention proposes the following means. A neural network circuit according to a first aspect of the present invention has a plurality of neural network computing cores, each having a convolution operation circuit that performs a convolution operation and a quantization operation circuit that performs a quantization operation, and the plurality of neural network computing cores are connected in a way that enables data input and output.

[0007] A neural network computation method according to a second aspect of the present invention is a neural network computation method using a first neural network computation core and a second neural network computation core, wherein the output data of the first neural network computation core is switched between a loopback data flow that loops back the output data of the first neural network computation core to the first neural network and a bypass data flow that bypasses the output data to the second neural network computation core. [Effects of the Invention]

[0008] The neural network circuit and neural network computation method of the present invention are highly functional and can be incorporated into embedded devices such as IoT devices. [Brief explanation of the drawing]

[0009] [Figure 1] This is a diagram of a convolutional neural network. [Figure 2] This diagram illustrates the convolution operation performed by a convolutional layer. [Figure 3] This is a diagram illustrating the data expansion in a convolution operation. [Figure 4] This figure shows the overall configuration of the neural network circuit according to the first embodiment. [Figure 5] This is a diagram showing the overall configuration of the NN arithmetic core. [Figure 6] This is a timing chart showing an operation example of the same NN arithmetic core. [Figure 7] This is a timing chart showing another operation example of the same NN arithmetic core. [Figure 8] This is a diagram showing the NN arithmetic multi-core. [Figure 9] This is a timing chart showing an operation example of the same NN arithmetic multi-core. [Figure 10] This is a timing chart showing another operation example of the same NN arithmetic multi-core. [Figure 11] [[ID=ID=19]]This is a timing chart showing another operation example of the same NN arithmetic multi-core. [Figure 12] This is an internal block diagram of the DMAC of the same neural network circuit. [Figure 13] This is a state transition diagram of the control circuit of the same DMAC. [Figure 14] This is an internal block diagram of the convolution operation circuit of the same neural network circuit. [Figure 15] This is an internal block diagram of the multiplier of the same convolution operation circuit. [Figure 16] This is an internal block diagram of the product-sum operation unit of the same multiplier. [Figure 17] This is an internal block diagram of the accumulator circuit of the same convolution operation circuit. [Figure 18] This is an internal block diagram of the accumulator unit of the same accumulator circuit. [Figure 19] This is an internal block diagram of the quantization operation circuit of the same neural network circuit. [Figure 20] This is an internal block diagram of the vector operation circuit and quantization circuit of the same quantization operation circuit. [Figure 21] [[ID=ID=49]]This is a block diagram of the operation unit. [Figure 22] This is an internal block diagram of the vector quantization unit of the same quantization circuit. [Figure 23]This is a diagram for explaining the control of the neural network circuit by a semaphore. [Figure 24] This is a timing chart of the first data flow. [Figure 25] This is a timing chart of the second data flow. [Figure 26] This is a timing chart of the first data flow. [Figure 27] This is a timing chart of the second data flow. [Figure 28] This is an internal block diagram of the convolutional operation circuit of the neural network circuit according to the second embodiment. [Figure 29] This is an internal block diagram of the multiplier of the convolutional operation circuit. [Figure 30] This is an internal block diagram of the sum-of-products operation unit array of the multiplier. [Figure 31] This is an internal block diagram of the sum-of-products operation unit of the sum-of-products operation unit array. [Figure 32] This is a diagram showing the overall configuration of the neural network circuit according to the third embodiment. [Figure 33] This is an internal block diagram of the first DMAC of the neural network circuit. [Figure 34] This is a timing chart showing the operation of the clock control unit such as the first DMAC. [Figure 35] This is an internal block diagram of the convolutional operation circuit of the neural network circuit. [Figure 36] This is an internal block diagram of the quantization operation circuit of the neural network circuit. [Figure 37] This is a diagram showing the overall configuration of the neural network circuit according to the fourth embodiment.

Embodiments for Carrying Out the Invention

[0010] (First Embodiment) The first embodiment of the present invention will be described with reference to FIGS. 1 to 27. Figure 1 shows a convolutional neural network 200 (hereinafter referred to as "CNN200"). The operations performed by the neural network circuit 100 (hereinafter referred to as "NN circuit 100") according to the first embodiment are at least a part of the trained CNN200 used during inference.

[0011] [CNN200] CNN200 is a multilayer network that includes a convolutional layer 210 for performing convolution operations, a quantization layer 220 for performing quantization operations, and an output layer 230. In at least a portion of CNN200, the convolutional layer 210 and the quantization layer 220 are alternately connected. CNN200 is a model widely used for image recognition and video recognition. CNN200 may further have layers with other functions, such as fully connected layers.

[0012] Figure 2 illustrates the convolution operation performed by the convolutional layer 210. The convolutional layer 210 performs a convolution operation on the input data a using the weights w. The convolutional layer 210 performs a sum-of-products operation with the input data a and the weights w as inputs.

[0013] The input data a (also called activation data or feature map) to the convolutional layer 210 is multidimensional data such as image data. In this embodiment, the input data a is a 3D tensor consisting of elements (x, y, c). The convolutional layer 210 of the CNN200 performs a convolution operation on the low-bit input data a. In this embodiment, the elements of the input data a are 2-bit unsigned integers (0, 1, 2, 3). The elements of the input data a may also be, for example, 4-bit or 8-bit unsigned integers.

[0014] If the input data to CNN200 is in a different format from the input data a to the convolutional layer 210, such as a 32-bit floating-point type, CNN200 may have an additional input layer before the convolutional layer 210 to perform type conversion and quantization.

[0015] The weights w (also called filters or kernels) of the convolutional layer 210 are multidimensional data having elements that are learnable parameters. In this embodiment, the weights w are a 4-dimensional tensor consisting of elements (i, j, c, d). The weights w have d 3-dimensional tensors (hereinafter referred to as "weights w") consisting of elements (i, j, c). The weights w in a trained CNN200 are trained data. The convolutional layer 210 of the CNN200 performs convolution operations using low-bit weights w. In this embodiment, the elements of the weights w are 1-bit signed integers (0, 1), where the value "0" represents +1 and the value "1" represents -1.

[0016] The convolutional layer 210 performs the convolution operation shown in Equation 1 and outputs output data f. In Equation 1, s represents the stride. In Figure 2, the region shown by the dotted line represents one of the regions ao (hereinafter referred to as "application region ao") to which the weight wo is applied to the input data a. The elements of the application region ao are represented by (x+i, y+j, c).

[0017]

number

[0018] The quantization layer 220 performs quantization on the output of the convolution operation output by the convolution layer 210. The quantization layer 220 includes a pooling layer 221, a batch normalization layer 222, an activation function layer 223, and a quantization layer 224.

[0019] The pooling layer 221 compresses the output data f of the convolution operation output by the convolution layer 210 by performing operations such as average pooling (Equation 2) and MAX pooling (Equation 3). In Equations 2 and 3, u represents the input tensor, v represents the output tensor, and T represents the size of the pooling region. In Equation 3, max is a function that outputs the maximum value of u for the combination of i and j included in T.

[0020]

number

[0021]

number

[0022] The Batch Normalization layer 222 normalizes the data distribution of the output data from the quantization layer 220 and the pooling layer 221 using an operation such as that shown in Equation 4. In Equation 4, u represents the input tensor, v represents the output tensor, α represents the scale, and β represents the bias. In the trained CNN200, α and β are trained constant vectors.

[0023]

number

[0024] The activation function layer 223 performs activation function operations such as ReLU (Equation 5) on the outputs of the quantization operation layer 220, the pooling layer 221, and the Batch Normalization layer 222. In Equation 5, u is the input tensor and v is the output tensor. In Equation 5, max is a function that outputs the largest number among the arguments.

[0025]

number

[0026] The quantization layer 224 performs quantization on the outputs of the pooling layer 221 and the activation function layer 223 based on the quantization parameters, for example, as shown in Equation 6. The quantization shown in Equation 6 reduces the input tensor u to 2 bits. In Equation 6, q(c) is a vector of quantization parameters. In a trained CNN200, q(c) is a trained constant vector. The inequality sign "≦" in Equation 6 may also be "<".

[0027]

number

[0028] The output layer 230 is the layer that outputs the results of CNN200 using an identity function, a softmax function, or the like. The layer preceding the output layer 230 may be a convolutional layer 210 or a quantization layer 220.

[0029] In CNN200, the output data of the quantized quantization layer 224 is input to the convolutional layer 210, resulting in a lower convolutional computation load on the convolutional layer 210 compared to other convolutional neural networks that do not perform quantization.

[0030] [Partitioning of Convolution Operations] The NN circuit 100 performs the convolution operation (Equation 1) of the convolution layer 210 by dividing the input data into subtensors. The method of dividing into subtensors and the number of divisions are not particularly limited. A subtensor is formed, for example, by dividing the input data a(x+i,y+j,c) into a(x+i,y+j,co). The NN circuit 100 can also perform the convolution operation (Equation 1) of the convolution layer 210 without dividing the input data.

[0031] In the input data partitioning for the convolution operation, the variable c in Equation 1 is partitioned into blocks of size Bc, as shown in Equation 7. Similarly, the variable d in Equation 1 is partitioned into blocks of size Bd, as shown in Equation 8. In Equation 7, co is the offset and ci is the index from 0 to (Bc-1). In Equation 8, do is the offset and di is the index from 0 to (Bd-1). Note that sizes Bc and Bd may be the same.

[0032]

number

[0033]

number

[0034] The input data a(x+i, y+j, c) in Equation 1 is divided in the c-axis direction by size Bc and is represented by the divided input data a(x+i, y+j, co). In the following description, the divided input data a is also referred to as "divided input data a".

[0035] The weight w(i, j, c, d) in Equation 1 is divided in the c-axis direction by size Bc and in the d-axis direction by size Bd and is represented by the divided weight w(i, j, co, do). In the following description, the divided weight w is also referred to as "divided weight w".

[0036] The output data f(x, y, do) divided by size Bd is obtained by Equation 9. By combining the divided output data f(x, y, do), the final output data f(x, y, d) can be calculated.

[0037] [Expansion of Data in Convolution Operation]

[0038] The NN circuit 100 expands the input data a and the weight w in the convolution operation of the convolution layer 210 and performs the convolution operation.

[0039] Figure 3 is a diagram for explaining the expansion of data in the convolution operation. The divided input data a(x+i, y+j, co) is expanded into vector data having Bc elements. The elements of the divided input data a are indexed by ci (0 ≤ ci < Bc). In the following description, the divided input data a expanded into vector data for each i and j is also referred to as "input vector A". The input vector A has elements from the divided input data a(x+i, y+j, co×Bc) to the divided input data a(x+i, y+j, co×Bc+(Bc-1)).

[0040] ​The segmentation weight w(i, j, co, do) is expanded into matrix data having Bc × Bd elements. The elements of the segmentation weight w expanded into matrix data are indexed by ci and di (0 ≦ di < Bd). In the following description, the segmentation weight w expanded into matrix data for each i and j is also referred to as a "weight matrix W". The weight matrix W has elements from the segmentation weight w(i, j, co × Bc, do × Bd) to the segmentation weight w(i, j, co × Bc + (Bc - 1), do × Bd + (Bd - 1)).

[0041] By multiplying the input vector A and the weight matrix W, vector data is calculated. By shaping the vector data calculated for each i, j, and co into a three-dimensional tensor, the output data f(x, y, do) can be obtained. By performing such data expansion, the convolution operation of the convolutional layer 210 can be implemented by multiplying vector data and matrix data.

[0042] [NN Circuit 100] FIG. 4 is a diagram showing the overall configuration of the NN circuit 100 according to the present embodiment. The NN circuit 100 includes a first DMA controller 3 (hereinafter also referred to as "first DMAC 3"), a controller 6, an IFU 7, a shared memory 8, a second DMA controller 9 (hereinafter also referred to as "second DMAC 9"), and at least one neural network arithmetic core 10 (hereinafter also referred to as "NN arithmetic core 10").

[0043] The NN circuit 100 can implement a plurality of NN arithmetic cores 10. The NN circuit 100 illustrated in FIG. 4 can implement up to four NN arithmetic cores 10 at most. The plurality of NN arithmetic cores 10 constitute a "neural network arithmetic multi-core 10M (hereinafter also referred to as "NN arithmetic multi-core 10M")" that executes at least some operations of the NN200 in cooperation with each other. The plurality of NN arithmetic cores 10 are daisy-chain connected in the present embodiment. Note that the number of NN arithmetic cores 10 that can be implemented in the NN circuit 100 may be five or more.

[0044] The first DMAC3 is connected to the external bus EB and performs data transfer between the external memory 120, such as DRAM, and the NN computation core 10. The first DMAC3 transfers data read from the external memory 120 to any of the multiple NN computation cores 10. The first DMAC3 may transfer the same data read from the external memory 120 to multiple NN computation cores 10, or it may broadcast the data. The first DMAC3 also performs data transfer between the external memory, such as DRAM, and the shared memory 8.

[0045] Controller 6 is connected to an external bus EB and operates as a slave to an external host CPU 110. Controller 6 has a bus bridge 60 and registers 61.

[0046] The bus bridge 60 relays bus access from the external bus EB to the internal bus IB. The bus bridge 60 also relays write and read requests to register 61 from the external host CPU 110.

[0047] Register 61 contains parameter registers and state registers. The parameter registers control the operation of the NN circuit 100. The state registers include pointers to the instruction sequences of each module, the number of instructions, etc., and indicate the state of the NN circuit 100. The state registers may also include semaphores S. The external host CPU 110 can access register 61 via the bus bridge 60 of the controller 6.

[0048] The controller 6 is connected to each block of the NN circuit 100 (first DMAC3, IFU7, second DMAC9, and NN arithmetic core 10) via the internal bus IB. The external host CPU 110 can access each block of the NN circuit 100 via the controller 6. For example, the external host CPU 110 can issue instructions to the NN arithmetic core 10 via the controller 6. In addition, each block can update the state registers (which may include semaphores S) of the controller 6 via the internal bus IB. The state registers may be configured to be updated via dedicated wiring connected to each block.

[0049] The IFU (Instruction Fetch Unit) 7 reads instruction commands for each block of the NN circuit 100 (first DMAC3, second DMAC9, NN arithmetic core 10) from the external memory 120 via the external bus EB, based on instructions from the external host CPU 110. The IFU 7 also transfers the read instruction commands to the corresponding blocks of the NN circuit 100 (first DMAC3, second DMAC9, NN arithmetic core 10).

[0050] Shared memory 8 is a rewritable memory such as volatile memory composed of SRAM (Static RAM), for example. Shared memory 8 is used to temporarily store data used by the NN computation core 10, or to store data shared by multiple NN computation cores.

[0051] The second DMAC9 connects the shared memory 8 and the NN computation core 10, and performs data transfer between the shared memory 8 and the NN computation core 10. The second DMAC9 may also be capable of broadcasting data read from the shared memory 8 to multiple NN computation cores 10.

[0052] The NN circuit 100 can temporarily save data shared by multiple NN computation cores to the shared memory 8 using the second DMAC 9, without having to save it to the external memory 120 using the first DMAC 3, thereby speeding up data transfer between NN computation cores. Note that the NN circuit 100 does not necessarily have a shared memory 8 and a second DMAC 9.

[0053] [NN Computation Core 10] Figure 5 shows the overall configuration of the NN computing core 10. The NN computation core 10 comprises a first memory 1, a second memory 2, a convolution operation circuit 4, and a quantization operation circuit 5. The NN computation core 10 is characterized in that the convolution operation circuit 4 and the quantization operation circuit 5 are formed in a loop via the first memory 1 and the second memory 2.

[0054] The first memory 1 is a rewritable memory such as a volatile memory composed of SRAM (Static RAM), for example. Data is written to and read from the first memory 1 via the first DMAC 3, the second DMAC 9, and the internal bus IB. The external host CPU 110 can input and output data to the NN computation core 10 by writing to and reading data from the first memory 1.

[0055] The first memory 1 is connected to the input port of the convolution arithmetic circuit 4, and the convolution arithmetic circuit 4 can read data from the first memory 1. The first memory 1 is also connected in a loop (C1) to the output port of the quantization arithmetic circuit 5, and the quantization arithmetic circuit 5 can write data to the first memory 1. Furthermore, data transfer from the first memory 1 is possible via inter-core connections (C2) with other NN arithmetic cores 10, and other NN arithmetic cores 10 connected via inter-core connections (C2) can write data to the first memory 1. In this embodiment, a daisy-chain connection is used as an example of an inter-core connection (C2).

[0056] The second memory 2 is a rewritable memory such as a volatile memory composed of SRAM (Static RAM), for example. Data is written to and read from the second memory 2 via the first DMAC 3, the second DMAC 9, and the internal bus IB. The external host CPU 110 can input and output data to the NN computation core 10 by writing to and reading data from the second memory 2.

[0057] The second memory 2 is connected to the input port of the quantization arithmetic circuit 5, allowing the quantization arithmetic circuit 5 to read data from the second memory 2. The second memory 2 is also connected to the output port of the convolution arithmetic circuit 4, allowing the convolution arithmetic circuit 4 to write data to the second memory 2.

[0058] The convolution operation circuit 4 is a circuit that performs convolution operations in the convolution layer 210 of the trained CNN200. The convolution operation circuit 4 reads the input data a stored in the first memory 1 and performs a convolution operation on the input data a. The convolution operation circuit 4 writes the output data f of the convolution operation (hereinafter also referred to as "convolution operation output data") to the second memory 2.

[0059] The quantization circuit 5 is a circuit that performs at least a portion of the quantization operations in the quantization layer 220 of the trained CNN200. The quantization circuit 5 reads the output data f of the convolution operation stored in the second memory 2 and performs quantization operations (operations including at least quantization, among pooling, batch normalization, activation function, and quantization) on the output data f of the convolution operation.

[0060] The quantization circuit 5 writes the output data of the quantization operation (hereinafter also referred to as "quantization operation output data") to the first memory 1 connected in a loop (C1). The quantization circuit 5 can also transfer data with other NN operation cores 10 via an inter-core connection (C2), and the quantization circuit 5 can output the quantization operation output data to other NN operation cores 10 connected in an inter-core connection (C2).

[0061] Since the NN computation core 10 has a first memory 1 and a second memory 2, etc., it can reduce the number of data transfers of duplicate data when transferring data from an external memory such as DRAM to the first DMAC 3. This can significantly reduce the power consumption or processing load generated by memory access.

[0062] [Example 1 of NN Computation Core 10 Operation] Figure 6 is a timing chart showing an example of the operation of the NN computing core 10. The first DMAC3 stores the input data a from layer 1 into the first memory 1. The first DMAC3 may also divide the input data a from layer 1 and transfer it to the first memory 1 in accordance with the order of the convolution operations performed by the convolution operation circuit 4.

[0063] The convolution circuit 4 reads the input data a of layer 1 stored in the first memory 1. The convolution circuit 4 performs the convolution operation of layer 1 shown in Figure 1 on the input data a of layer 1. The output data f of the convolution operation of layer 1 is stored in the second memory 2.

[0064] The quantization circuit 5 reads the output data f of Layer 1 stored in the second memory 2. The quantization circuit 5 performs the Layer 2 quantization operation on the output data f of Layer 1. The output data of the Layer 2 quantization operation is stored in the first memory 1.

[0065] The convolution circuit 4 reads the output data of the quantization operation of layer 2 stored in the first memory 1. The convolution circuit 4 uses the output data of the quantization operation of layer 2 as input data a and performs the convolution operation of layer 3. The output data f of the convolution operation of layer 3 is stored in the second memory 2.

[0066] The convolution circuit 4 reads the output data of the quantization operation of layer 2M-2 (where M is a natural number) stored in the first memory 1. The convolution circuit 4 uses the output data of the quantization operation of layer 2M-2 as input data a and performs the convolution operation of layer 2M-1. The output data f of the convolution operation of layer 2M-1 is stored in the second memory 2.

[0067] The quantization circuit 5 reads the output data f of layer 2M-1 stored in the second memory 2. The quantization circuit 5 performs a quantization operation on the output data f of layer 2M-1. The output data of the quantization operation of layer 2M is stored in the first memory 1.

[0068] The convolution circuit 4 reads the output data of the quantization operation of layer 2M stored in the first memory 1. The convolution circuit 4 uses the output data of the quantization operation of layer 2M as input data a and performs a convolution operation on layer 2M+1. The output data f of the convolution operation of layer 2M+1 is stored in the second memory 2.

[0069] The convolution circuit 4 and the quantization circuit 5 perform operations alternately, advancing the CNN200 calculations shown in Figure 1. The NN computation core 10 performs the convolution operations of layers 2M-1 and 2M+1 using time-division multiplexing of the convolution circuit 4. The NN computation core 10 also performs the quantization operations of layers 2M-2 and 2M using time-division multiplexing of the quantization circuit 5. Therefore, the NN computation core 10 has a significantly smaller circuit size compared to a case where separate convolution circuits 4 and quantization circuits 5 are implemented for each layer.

[0070] The NN computation core 10 performs calculations for the CNN200, which has a multilayer structure of multiple layers, using a loop-shaped circuit. The NN computation core 10 can efficiently utilize hardware resources through its loop-shaped circuit configuration. In order to form the circuit in a loop, the parameters of the convolutional operation circuit 4 and quantization operation circuit 5, which change in each layer, are updated as needed.

[0071] If the CNN200 calculations include operations that cannot be performed by the NN calculation core 10, the NN calculation core 10 transfers intermediate data to an external computing device such as an external host CPU 110. After the external computing device performs calculations on the intermediate data, the calculation results from the external computing device are input into the first memory 1 and the second memory 2. The NN calculation core 10 then resumes calculations on the calculation results from the external computing device.

[0072] [Example of NN Computation Core 10 Operation 2] Figure 7 is a timing chart showing another example of the operation of the NN computing core 10. The NN computation core 10 may divide the input data a into subtensors and perform operations on the subtensors using time-division multiplexing. The method of dividing into subtensors and the number of divisions are not particularly limited.

[0073] Figure 7 shows an example of operation when input data a is decomposed into two subtensors. The decomposed subtensors are denoted as "first subtensor a1" and "second subtensor a2". For example, the convolution operation of layer 2M-1 is decomposed into the convolution operation corresponding to the first subtensor a1 (denoted as "layer 2M-1(a1)" in Figure 7) and the convolution operation corresponding to the second subtensor a2 (denoted as "layer 2M-1(a2)" in Figure 7).

[0074] The convolution and quantization operations corresponding to the first subtensor a1 and the convolution and quantization operations corresponding to the second subtensor a2 can be performed independently, as shown in Figure 7.

[0075] The convolution circuit 4 performs the convolution operation on layer 2M-1 corresponding to the first subtensor a1 (shown as layer 2M-1(a1) in Figure 7). Subsequently, the convolution circuit 4 performs the convolution operation on layer 2M-1 corresponding to the second subtensor a2 (shown as layer 2M-1(a2) in Figure 7). The quantization circuit 5 also performs the quantization operation on layer 2M corresponding to the first subtensor a1 (shown as layer 2M(a1) in Figure 7). In this way, the NN computation core 10 can perform the convolution operation on layer 2M-1 corresponding to the second subtensor a2 and the quantization operation on layer 2M corresponding to the first subtensor a1 in parallel.

[0076] Next, the convolution circuit 4 performs the convolution operation on layer 2M+1 corresponding to the first subtensor a1 (indicated as layer 2M+1(a1) in Figure 7). The quantization circuit 5 then performs the quantization operation on layer 2M corresponding to the second subtensor a2 (indicated as layer 2M(a2) in Figure 7). In this way, the NN computation core 10 can perform the convolution operation on layer 2M+1 corresponding to the first subtensor a1 and the quantization operation on layer 2M corresponding to the second subtensor a2 in parallel.

[0077] The convolution and quantization operations corresponding to the first subtensor a1 and the convolution and quantization operations corresponding to the second subtensor a2 can be performed independently. Therefore, the NN computation core 10 may, for example, perform the convolution operation of layer 2M-1 corresponding to the first subtensor a1 and the quantization operation of layer 2M+2 corresponding to the second subtensor a2 in parallel. In other words, the convolution and quantization operations performed in parallel by the NN computation core 10 are not limited to operations on consecutive layers.

[0078] By dividing the input data a into subtensors, the NN computation core 10 can operate the convolution circuit 4 and the quantization circuit 5 in parallel. As a result, the waiting time for the convolution circuit 4 and the quantization circuit 5 is reduced, improving the computational efficiency of the NN computation core 10. In the example shown in Figure 7, the number of divisions was 2, but similarly, the NN computation core 10 can operate the convolution circuit 4 and the quantization circuit 5 in parallel even when the number of divisions is greater than 2.

[0079] For example, if the input data a is divided into "first subtensor a1", "second subtensor a2", and "third subtensor a3", the NN computation core 10 may perform the convolution operation of layer 2M-1 corresponding to the second subtensor a2 and the quantization operation of layer 2M corresponding to the third subtensor a3 in parallel. The order of operations is changed as appropriate depending on the storage status of the input data a in the first memory 1 and the second memory 2.

[0080] As an example of how to perform operations on subtensors, we showed an example (Method 1) in which operations on subtensors in the same layer are performed in the convolution circuit 4 or quantization circuit 5, and then operations on subtensors in the next layer are performed. For example, as shown in Figure 7, in the convolution circuit 4, after performing the convolution operation on layer 2M-1 corresponding to the first subtensor a1 and the second subtensor a2 (operations shown as layer 2M-1(a1) and layer 2M-1(a2) in Figure 7), the convolution operation on layer 2M+1 corresponding to the first subtensor a1 and the second subtensor a2 (operations shown as layer 2M+1(a1) and layer 2M+1(a2) in Figure 7) is performed.

[0081] However, the method of operation on subtensors is not limited to this. The method of operation on subtensors may also be to perform operations on some subtensors in multiple layers and then perform operations on the remaining subtensors (Method 2). For example, in the convolution operation circuit 4, after performing a convolution operation on layer 2M-1 corresponding to the first subtensor a1 and layer 2M+1 corresponding to the first subtensor a1, a convolution operation on layer 2M-1 corresponding to the second subtensor a2 and layer 2M+1 corresponding to the second subtensor a2 may be performed.

[0082] Furthermore, the method for performing operations on a subtensor may be a combination of Method 1 and Method 2. However, when using Method 2, it is necessary to perform operations according to the dependencies on the order of operations of the subtensor.

[0083] [NN multi-core 10M] Figure 8 shows a 10M multi-core NN computation. The NN multicore 10M illustrated in Figure 8 comprises two daisy-chained NN cores 10. To distinguish between the two NN cores 10, they are referred to as the "first NN core 10A" and the "second NN core 10B". In Figure 8, the first memory 1 is abbreviated as "A", the convolution circuit 4 as "C", the second memory 2 as "F", and the quantization circuit 5 as "Q".

[0084] Specifically, the quantization circuit 5 of the first NN computation core 10A and the first memory 1 of the second NN computation core 10B are daisy-chained (C2). The quantization circuit 5 of the first NN computation core 10A can write quantization output data to the first memory 1 of the first NN computation core 10A, which is loop-connected (C1), and / or the first memory 1 of the second NN computation core 10B, which is daisy-chained (C2).

[0085] Specifically, the quantization circuit 5 of the second NN computation core 10B and the first memory 1 of the first NN computation core 10A are daisy-chained (C2). The quantization circuit 5 of the second NN computation core 10B can write quantization output data to the first memory 1 of the second NN computation core 10B which is loop-connected (C1) and / or the first memory 1 of the first NN computation core 10A which is daisy-chained (C2).

[0086] Similarly, when the NN multicore 10M has three or more NN cores 10, the multiple NN cores 10 are daisy-chained. The quantization calculation circuits 5 of the NN cores 10 other than the final stage NN core 10 are daisy-chained (C2) with the first memory 1 of the subsequent NN core 10B. The quantization calculation circuit 5 of the final stage NN core 10 is daisy-chained (C2) with the first memory 1 of the first stage NN core 10. The multiple NN cores 10 are characterized by being formed in a daisy-chain loop (beaded) configuration.

[0087] In a single NN computation core 10, the first memory (A)1, the convolution circuit (C)4, the second memory (F)2, and the quantization circuit (Q)5 are connected in a loop. On the other hand, in the NN computation multicore 10M, the first memory (A)1, the convolution circuit (C)4, the second memory (F)2, and the quantization circuit (Q)5 are connected in a daisy-chain loop so that the first memory (A)1, the convolution circuit (C)4, the second memory (F)2, and the quantization circuit (Q)5 are repeatedly arranged in the same order.

[0088] The multiple NN computation cores 10 that make up the NN computation multicore 10M do not have to have the same hardware configuration. For example, the capacity and configuration of the first memory 1 of the first NN computation core 10A may differ from the capacity and configuration of the first memory 1 of the second NN computation core 10B. For example, the configuration of the quantization computation circuit 5 of the first NN computation core 10A may differ from the configuration of the quantization computation circuit 5 of the second NN computation core 10B.

[0089] [Example 1 of operation of NN computation multicore 10M] Figure 9 is a timing chart showing example 1 of the operation of the NN multicore 10M. The convolution and quantization operations corresponding to the first subtensor a1 and the convolution and quantization operations corresponding to the second subtensor a2 are performed independently by different NN computation cores 10, as shown in Figure 9.

[0090] The convolution circuit 4 of the first NN computation core 10A performs a convolution operation on layer 2M-1 corresponding to the first subtensor a1 (indicated as layer 2M-1(a1) in Figure 9). Subsequently, the quantization circuit 5 of the first NN computation core 10A performs a quantization operation on layer 2M corresponding to the first subtensor a1 (indicated as layer 2M(a1) in Figure 9). The quantization circuit 5 of the first NN computation core 10A stores the output data of the quantization operation on layer 2M corresponding to the first subtensor a1 in the first memory 1 of the first NN computation core 10A.

[0091] The convolution circuit 4 of the second NN computation core 10B performs a convolution operation on layer 2M-1 corresponding to the second subtensor a2 (indicated as layer 2M-1(a2) in Figure 9). Subsequently, the quantization circuit 5 of the second NN computation core 10B performs a quantization operation on layer 2M corresponding to the second subtensor a2 (indicated as layer 2M(a2) in Figure 9). The quantization circuit 5 of the second NN computation core 10B stores the output data of the quantization operation on layer 2M corresponding to the second subtensor a2 into the first memory 1 of the second NN computation core 10B.

[0092] The second DMAC9 performs a DMA transfer (shown as DMA1 in Figure 9) of the quantization operation output data of layer 2M corresponding to the first partial tensor a1 stored in the first memory 1 of the first NN computation core 10A to the shared memory 8. Next, the second DMAC9 performs a DMA transfer (shown as DMA2 in Figure 9) of the quantization operation output data of layer 2M corresponding to the second partial tensor a2 stored in the first memory 1 of the second NN computation core 10B to the shared memory 8.

[0093] The first DMAC3 performs a DMA transfer (shown as DMA3 in Figure 9) of the quantization operation output data of layer 2M corresponding to the first subtensor a1 and the second subtensor a2 stored in the shared memory 8 to the external memory 120.

[0094] The NN multi-core 10M can, for example, reduce the time required for calculations by independently performing calculations on the same layer using different NN cores 10. Furthermore, the shared memory 8 and the second DMAC 9 allow for the aggregation of calculation results from each NN core 10.

[0095] [Example 2 of NN computation multi-core 10M operation] Figure 10 is a timing chart showing example 2 of the operation of the NN multicore 10M. The convolution and quantization operations corresponding to the first subtensor a1 are performed in cooperation with different NN computation cores 10.

[0096] The convolution circuit 4 of the first NN computation core 10A performs a convolution operation on layer 2M-1 corresponding to the first subtensor a1 (indicated as layer 2M-1(a1) in Figure 10). Subsequently, the quantization circuit 5 of the first NN computation core 10A performs a quantization operation on layer 2M corresponding to the first subtensor a1 (indicated as layer 2M(a1) in Figure 10). The quantization circuit 5 of the first NN computation core 10A stores the output data of the quantization operation on layer 2M corresponding to the first subtensor a1 in the first memory 1 of the second NN computation core 10B.

[0097] The convolution circuit 4 of the second NN computation core 10B performs a convolution operation on layer 2M+1 corresponding to the first subtensor a1 (indicated as layer 2M+1(a1) in Figure 10). Subsequently, the quantization circuit 5 of the second NN computation core 10B performs a quantization operation on layer 2M+2 corresponding to the first subtensor a1 (indicated as layer 2M+2(a1) in Figure 10). The quantization circuit 5 of the second NN computation core 10B stores the output data of the quantization operation on layer 2M+2 corresponding to the second subtensor a2 into the first memory 1 of the second NN computation core 10B.

[0098] The first DMAC3 performs a DMA transfer (a transfer shown as DMA in Figure 10) of the quantization operation output data of layer 2M+2 corresponding to the first partial tensor a1 stored in the first memory 1 of the second NN computation core 10B to the external memory 120.

[0099] The NN multi-core 10M can, for example, shorten the time required for computation by performing operations corresponding to the same subtensor sequentially using different NN cores 10.

[0100] [Example 3 of NN computation multi-core 10M operation] Figure 11 is a timing chart showing example 3 of the operation of the NN multicore 10M. In Operation Example 3, the configuration of the subtensor is changed in the convolution operations of layer 2M-1 and layer 2M+1.

[0101] The second DMAC9 performs a DMA transfer (shown as DMA1 in Figure 11) of the first NN computation core 10A to the first memory 1. Next, the second DMAC9 performs a DMA transfer (shown as DMA2 in Figure 11) of the second NN computation core 10B to the first memory 1.

[0102] The convolution circuit 4 of the first NN computation core 10A performs a convolution operation on layer 2M-1 corresponding to the first subtensor a1 (shown as layer 2M-1(a1) in Figure 11). Subsequently, the quantization circuit 5 of the first NN computation core 10A performs a quantization operation on layer 2M corresponding to the first subtensor a1 (shown as layer 2M(a1) in Figure 11). The quantization circuit 5 of the first NN computation core 10A stores the output data of the quantization operation on layer 2M corresponding to the first subtensor a1 in the first memory 1 of the first NN computation core 10A.

[0103] The second DMAC9 performs a DMA transfer (shown as DMA3 in Figure 11) of the quantization operation output data of layer 2M corresponding to the first partial tensor a1 stored in the first memory 1 of the first NN computation core 10A to the shared memory 8.

[0104] The convolution circuit 4 of the second NN computation core 10B performs a convolution operation on layer 2M-1 corresponding to the second subtensor a2 (indicated as layer 2M-1(a2) in Figure 11). The start of the convolution operation by the convolution circuit 4 of the second NN computation core 10B is later than the start of the convolution operation by the convolution circuit 4 of the first NN computation core 10A. Subsequently, the quantization circuit 5 of the second NN computation core 10B performs a quantization operation on layer 2M corresponding to the second subtensor a2 (indicated as layer 2M(a2) in Figure 11). The quantization circuit 5 of the second NN computation core 10B stores the output data of the quantization operation on layer 2M corresponding to the second subtensor a2 into the first memory 1 of the second NN computation core 10B.

[0105] The second DMAC9 performs a DMA transfer (shown as DMA4 in Figure 11) of the quantization operation output data of layer 2M, which corresponds to the second subtensor a2 stored in the first memory 1 of the second NN computation core 10B, to the shared memory 8.

[0106] For example, if the input data a has 32 channels in the c-axis direction in the convolution operation of layer 2M-1, and the input data a has 64 channels in the c-axis direction in the convolution operation of layer 2M+1, it is desirable to change the subtensor division method in order to efficiently use the convolution operation circuit 4, etc. in the convolution operation of layer 2M+1. For example, consider a case where each NN operation core 10 is optimized to perform parallel operations on input data a with 32 channels in the c-axis direction, and a convolution operation of layer 2M+1 is performed on input data a with 64 channels in the c-axis direction. In this case, the division method from input data a to subtensors may be changed so that the convolution operation circuit 4 of NN operation core 10A performs operations on input data a from channels 0 to 31, and the convolution operation circuit 4 of NN operation core 10B performs operations on input data a from channels 32 to 63. Two of the subdivided subtensors are called the "third subtensor a3" and the "fourth subtensor a4".

[0107] The second DMAC9 performs a DMA transfer (shown as DMA5 in Figure 11) of the third subtensor a3 to the first memory 1 of the first NN computation core 10A. Next, the second DMAC9 performs a DMA transfer (shown as DMA6 in Figure 11) of the fourth subtensor a4 to the first memory 1 of the second NN computation core 10B.

[0108] The convolution circuit 4 of the first NN computation core 10A performs a convolution operation on layer 2M+1 corresponding to the third subtensor a3 (shown as layer 2M+1(a3) in Figure 11). Subsequently, the quantization circuit 5 of the first NN computation core 10A performs a quantization operation on layer 2M+2 corresponding to the third subtensor a3 (shown as layer 2M+2(a3) in Figure 11). The quantization circuit 5 of the first NN computation core 10A stores the output data of the quantization operation on layer 2M+2 corresponding to the third subtensor a3 into the first memory 1 of the first NN computation core 10A.

[0109] The convolution circuit 4 of the second NN computation core 10B performs a convolution operation on layer 2M+1 corresponding to the fourth subtensor a4 (shown as layer 2M+1(a4) in Figure 11). Subsequently, the quantization circuit 5 of the second NN computation core 10B performs a quantization operation on layer 2M+2 corresponding to the fourth subtensor a4 (shown as layer 2M+2(a4) in Figure 11). The quantization circuit 5 of the second NN computation core 10B stores the output data of the quantization operation on layer 2M+2 corresponding to the fourth subtensor a4 into the first memory 1 of the second NN computation core 10B.

[0110] Even if the characteristics of the input data a change (for example, if the number of channels increases in a convolution operation), the NN circuit 100 can change the partitioning pattern of the subtensors allocated to the NN computation core 10 using the second DMAC 9 and the shared memory 8. Even when changing the partitioning pattern of the subtensors, the NN circuit 100 can reduce the number of times the input data a is saved to the external memory 120 by DMA transfer using the first DMAC 3.

[0111] Next, we will explain in detail each component of the NN circuit 100.

[0112] [First DMAC3] Figure 12 is an internal block diagram of the first DMAC3. The first DMAC3 includes a data transfer circuit 31 and a state controller 32. The first DMAC3 has a dedicated state controller 32 for the data transfer circuit 31, and when an instruction command is input, it can perform DMA data transfer without requiring an external controller.

[0113] The data transfer circuit 31 is connected to the external bus EB and performs DMA data transfer between the external memory 120, such as DRAM, and the NN arithmetic core 10. The data transfer circuit 31 also performs DMA data transfer between the external memory 120, such as DRAM, and the shared memory 8. The number of DMA channels in the data transfer circuit 31 is not limited. For example, the first NN arithmetic core 10A and the second NN arithmetic core 10B may each have their own dedicated DMA channel.

[0114] The state controller 32 controls the state of the data transfer circuit 31. The state controller 32 is also connected to the controller 6 via the internal bus IB. The state controller 32 includes an instruction queue 33 and a control circuit 34.

[0115] The instruction queue 33 is a queue that stores instruction commands C3 for the first DMAC3, and is composed of, for example, a FIFO memory. One or more instruction commands C3 are written to the instruction queue 33 via the IFU7 or the internal bus IB.

[0116] The control circuit 34 is a state machine that decodes instruction command C3 and sequentially controls the data transfer circuit 31 based on instruction command C3. The control circuit 34 may be implemented by logic circuits or by a CPU controlled by software.

[0117] Figure 13 is a state transition diagram of the control circuit 34. When the instruction command C3 is input to the instruction queue 33 (not empty), the control circuit 34 transitions from the idle state ST1 to the decode state ST2.

[0118] In the decode state ST2, the control circuit 34 decodes the instruction command C3 output from the instruction queue 33. The control circuit 34 also reads the semaphore S stored in the controller 6's register 61 and determines whether the operation of the data transfer circuit 31 instructed by the instruction command C3 is possible. If it is not possible (Not ready), the control circuit 34 waits until it becomes possible (Wait). If it is possible (Ready), the control circuit 34 transitions from the decode state ST2 to the execution state ST3.

[0119] In the execution state ST3, the control circuit 34 controls the data transfer circuit 31 to perform the operation instructed by the instruction command C3. When the operation of the data transfer circuit 31 is complete, the control circuit 34 removes the completed instruction command C3 from the instruction queue 33 and updates the semaphore S stored in the controller 6's register 61. If there is an instruction in the instruction queue 33 (Not empty), the control circuit 34 transitions from the execution state ST3 to the decode state ST2. If there is no instruction in the instruction queue 33 (empty), the control circuit 34 transitions from the execution state ST3 to the idle state ST1.

[0120] [Convolution arithmetic circuit 4] Figure 14 is an internal block diagram of the convolution operation circuit 4. The convolution arithmetic circuit 4 includes a weight memory 41, a multiplier 42, an accumulator circuit 43, and a state controller 44. The convolution arithmetic circuit 4 has a dedicated state controller 44 for the multiplier 42 and the accumulator circuit 43, and when an instruction command is input, it can perform a convolution operation without requiring an external controller.

[0121] The weight memory 41 is a memory that stores the weights w used in the convolution operation, and is a rewritable memory such as a volatile memory composed of SRAM (Static RAM), for example. The first DMAC 3 writes the weights w required for the convolution operation to the weight memory 41 by DMA transfer.

[0122] Figure 15 is an internal block diagram of the multiplier 42. The multiplier 42 multiplies the input vector A and the weight matrix W. As described above, the input vector A is a vector data with Bc elements obtained by expanding the partitioned input data a(x+i, y+j, co) for each i and j. The weight matrix W is a matrix data with Bc × Bd elements obtained by expanding the partitioned weights w(i,j,co, do) for each i and j. The multiplier 42 has Bc × Bd multiply-accumulate units 47 and can perform the multiplication of the input vector A and the weight matrix W in parallel.

[0123] The multiplier 42 reads the input vector A and weight matrix W required for multiplication from the first memory 1 and weight memory 41, and performs the multiplication. The multiplier 42 outputs Bd sum-of-accumulate results O(di).

[0124] Figure 16 is an internal block diagram of the sum-of-accumulate unit 47. The multiply-accumulate unit 47 multiplies the input vector A's element A(ci) by the weight matrix W's element W(ci,di). The multiply-accumulate unit 47 then adds the result of this multiplication to the result S(ci,di) of another multiply-accumulate unit 47. The multiply-accumulate unit 47 outputs the summation result S(ci+1,di). Element A(ci) is a 2-bit unsigned integer (0,1,2,3). Element W(ci,di) is a 1-bit signed integer (0,1), where the value "0" represents +1 and the value "1" represents -1.

[0125] The multiply-accumulate unit 47 includes an inverter 47a, a selector 47b, and an adder 47c. The multiply-accumulate unit 47 performs multiplication using only the inverter 47a and selector 47b, without using a multiplier. Selector 47b selects the input of element A(ci) when element W(ci,di) is "0". Selector 47b selects the complement of element A(ci) inverted by the inverter when element W(ci,di) is "1". Element W(ci,di) is also input to the carry-in of adder 47c. Adder 47c outputs the value obtained by adding element A(ci) to S(ci,di) when element W(ci,di) is "0". Adder 47c outputs the value obtained by subtracting element A(ci) from S(ci,di) when W(ci,di) is "1".

[0126] Figure 17 is an internal block diagram of the accumulator circuit 43. The accumulator circuit 43 accumulates the sum-of-accumulate result O(di) of the multiplier 42 in the second memory 2. The accumulator circuit 43 has Bd accumulator units 48 and can accumulate Bd sum-of-accumulate results O(di) in the second memory 2 in parallel.

[0127] Figure 18 is an internal block diagram of the accumulator unit 48. The accumulator unit 48 includes an adder 48a and a mask unit 48b. The adder 48a adds the elements O(di) of the sum-of-accumulate result O to the partial sum, which is an intermediate result of the convolution operation shown in Equation 1 and stored in the second memory 2. The addition result is 16 bits per element. The addition result is not limited to 16 bits per element, but may be, for example, 15 bits or 17 bits per element.

[0128] The adder 48a writes the addition result to the same address in the second memory 2. The mask unit 48b, when the initialization signal clear is asserted, masks the output from the second memory 2 and sets the element O(di) to zero for addition. The initialization signal clear is asserted when the partial sum of the intermediate results is not stored in the second memory 2.

[0129] Once the convolution operation by the multiplier 42 and the accumulator circuit 43 is completed, the output data f(x,y,do) is stored in the second memory 2.

[0130] The state controller 44 controls the states of the multiplier 42 and the accumulator circuit 43. The state controller 44 is also connected to the controller 6 via the internal bus IB. The state controller 44 has an instruction queue 45 and a control circuit 46.

[0131] The instruction queue 45 is a queue that stores instruction commands C4 for the convolution arithmetic circuit 4, and is composed of, for example, a FIFO memory. Instruction commands C4 are written to the instruction queue 45 via the IFU 7 or via the internal bus IB.

[0132] The control circuit 46 is a state machine that decodes the instruction command C4 and controls the multiplier 42 and the accumulator circuit 43 based on the instruction command C4. The control circuit 46 has a similar configuration to the control circuit 34 of the state controller 32 of the first DMAC3.

[0133] [Quantization operation circuit 5] Figure 19 is an internal block diagram of the quantization operation circuit 5. The quantization operation circuit 5 includes a quantization parameter memory 51, a vector operation circuit 52, a quantization circuit 53, and a state controller 54. The quantization operation circuit 5 has a dedicated state controller 54 for the vector operation circuit 52 and the quantization circuit 53, and when an instruction command is input, it can perform quantization operations without requiring an external controller.

[0134] The quantization parameter memory 51 is a memory that stores the quantization parameters q used in the quantization operation, and is a rewritable memory such as a volatile memory composed of SRAM (Static RAM), for example. The first DMAC 3 writes the quantization parameters q required for the quantization operation to the quantization parameter memory 51 by DMA transfer.

[0135] Figure 20 is an internal block diagram of the vector operation circuit 52 and the quantization circuit 53. The vector arithmetic circuit 52 performs operations on the output data f(x,y,do) stored in the second memory 2. The vector arithmetic circuit 52 has Bd arithmetic units 57 and performs SIMD operations in parallel on the output data f(x,y,do).

[0136] Figure 21 is a block diagram of the arithmetic unit 57. The arithmetic unit 57 includes, for example, an ALU 57a, a first selector 57b, a second selector 57c, a register 57d, and a shifter 57e. The arithmetic unit 57 may further include other arithmetic units, etc., that are present in known general-purpose SIMD arithmetic circuits.

[0137] The vector operation circuit 52 combines the arithmetic units and other components of the operation unit 57 to perform at least one operation on the output data f(x,y,do) from among the operations of the pooling layer 221, the batch normalization layer 222, and the activation function layer 223 in the quantization operation layer 220.

[0138] The arithmetic unit 57 can add the data stored in register 57d and the element f(di) of the output data f(x,y,do) read from the second memory 2 using ALU 57a. The arithmetic unit 57 can store the addition result from ALU 57a in register 57d. The arithmetic unit 57 can initialize the addition result by inputting "0" to ALU 57a in place of the data stored in register 57d, depending on the selection of the first selector 57b. For example, if the pooling area is 2x2, the shifter 57e can output the average value of the addition result by right-shifting the output of ALU 57a by 2 bits. The vector arithmetic circuit 52 can perform the average pooling calculation shown in Equation 2 by repeating the above calculations by Bd arithmetic units 57.

[0139] The arithmetic unit 57 can compare the data stored in register 57d with the element f(di) of the output data f(x,y,do) read from the second memory 2 using the ALU 57a. The arithmetic unit 57 controls the second selector 57c according to the comparison result by ALU 57a to select the larger of the data stored in register 57d and element f(di). The arithmetic unit 57 can initialize the comparison target to the minimum value by inputting the minimum possible value of element f(di) to ALU 57a based on the selection of the first selector 57b. In this embodiment, since element f(di) is a 16-bit signed integer, the minimum possible value of element f(di) is "0x8000". The vector arithmetic circuit 52 can perform the MAX pooling calculation of equation 3 by repeating the above calculations by Bd arithmetic units 57. Note that in the MAX pooling calculation, the shifter 57e does not shift the output of the second selector 57c.

[0140] The arithmetic unit 57 can subtract the data stored in register 57d from the element f(di) of the output data f(x,y,do) read from the second memory 2 using ALU 57a. The shifter 57e can left-shift (i.e. multiply) or right-shift (i.e. divide) the output of ALU 57a. The vector arithmetic circuit 52 can perform the Batch Normalization operation of Equation 4 by repeating the above operations by Bd arithmetic units 57.

[0141] The arithmetic unit 57 can compare the element f(di) of the output data f(x,y,do) read from the second memory 2 with "0" selected by the first selector 57b using the ALU 57a. The arithmetic unit 57 can select and output either the element f(di) or the constant value "0" previously stored in register 57d, depending on the comparison result by the ALU 57a. The vector arithmetic circuit 52 can perform the ReLU operation of equation 5 by repeating the above operations by Bd arithmetic units 57.

[0142] The vector operation circuit 52 can perform mean pooling, MAX pooling, batch normalization, activation function calculations, and combinations thereof. Since the vector operation circuit 52 can perform general-purpose SIMD operations, it may also perform other operations necessary for the operations in the quantization operation layer 220. Furthermore, the vector operation circuit 52 may also perform operations other than those in the quantization operation layer 220.

[0143] Note that the quantization circuit 5 does not necessarily have a vector operation circuit 52. If the quantization circuit 5 does not have a vector operation circuit 52, the output data f(x,y,do) is input to the quantization circuit 53.

[0144] The quantization circuit 53 performs quantization on the output data of the vector operation circuit 52. As shown in Figure 20, the quantization circuit 53 has Bd quantization units 58 and performs operations in parallel on the output data of the vector operation circuit 52.

[0145] Figure 22 is an internal block diagram of the quantization unit 58. The quantization unit 58 performs quantization on the elements in(di) of the output data of the vector operation circuit 52. The quantization unit 58 includes a comparator 58a and an encoder 58b. The quantization unit 58 performs the calculation (Equation 6) of the quantization layer 224 in the quantization operation layer 220 on the output data (16 bits / element) of the vector operation circuit 52. The quantization unit 58 reads the necessary quantization parameters q(th0,th1,th2) from the quantization parameter memory 51 and compares the input in(di) with the quantization parameters q using the comparator 58a. The quantization unit 58 quantizes the comparison result from the comparator 58a to 2 bits / element using the encoder 58b. Since α(c) and β(c) in Equation 4 are different parameters for each variable c, the quantization parameters q(th0,th1,th2) that reflect α(c) and β(c) are different parameters for each in(di).

[0146] The quantization unit 58 classifies the input in(di) into four regions (for example, in≦th0, th0<in≦th1, th1<in≦th2, th2<in) by comparing the input in(di) with three thresholds th0, th1, th2, encodes the classification result into 2 bits, and outputs it. The quantization unit 58 can also perform operations of Batch Normalization and activation functions together with quantization by setting quantization parameters q(th0, th1, th2).

[0147] The quantization unit 58 can perform the operation of Batch Normalization shown in Equation 4 together with quantization by setting the threshold th0 as β(c) in Equation 4 and the threshold differences (th1−th0) and (th2−th1) as α(c) in Equation 4. By increasing (th1−th0) and (th2−th1), α(c) can be decreased. By decreasing (th1−th0) and (th2−th1), α(c) can be increased.

[0148] The quantization unit 58 can perform an activation function together with quantization of the input in(di). For example, the quantization unit 58 saturates the output value in the regions where in(di)≦th0 and th2<in(di). The quantization unit 58 can perform the operation of the activation function together with quantization by setting the quantization parameter q so that the output is non-linear.

[0149] The state controller 54 controls the states of the vector operation circuit 52 and the quantization circuit 53. Also, the state controller 54 is connected to the controller 6 via an internal bus IB. The state controller 54 has an instruction queue 55 and a control circuit 56.

[0150] The instruction queue 55 is a queue in which instruction commands C5 for the quantization operation circuit 5 are stored, and is composed of, for example, a FIFO memory. Instruction commands C5 are written into the instruction queue 55 via the IFU7 or the internal bus IB.

[0151] The control circuit 56 is a state machine that decodes the instruction command C5 and controls the vector operation circuit 52 and the quantization circuit 53 based on the instruction command C5. The control circuit 56 has a similar configuration to the control circuit 34 of the state controller 32 of the first DMAC3.

[0152] The quantization circuit 5 writes quantization output data having Bd elements to the first memory 1. A suitable relationship between Bd and Bc is shown in Equation 10, where n is an integer.

[0153]

number

[0154] [Controller 6] The controller 6 transfers instruction commands received from the external host CPU 110 to the instruction queues of the first DMAC 3, second DMAC 9, convolution arithmetic circuit 4, and quantization arithmetic circuit 5 via the internal bus IB. The controller 6 may also have an instruction memory for storing instruction commands for each circuit.

[0155] Controller 6 is connected to the external bus EB and operates as a slave to the external host CPU 110. Controller 6 has registers 61, including parameter registers and state registers. The parameter registers control the operation of the NN circuit 100. The state registers indicate the state of the NN circuit 100, including the semaphore S.

[0156] [Semaphore S] Semaphore S is decremented by the P operation and incremented by the V operation. The P and V operations performed by the first DMAC3, the convolution arithmetic circuit4, and the quantization arithmetic circuit5 update semaphore S in the controller6 via the internal bus IB.

[0157] Figure 23 illustrates the control of the NN circuit 100 by the semaphore S. A semaphore S is provided for each data flow F via the memory (first memory 1, second memory 2) in the NN circuit 100. In Figure 23 and the following description, semaphores related to data flows associated with the second DMAC9 are omitted for simplicity. In this embodiment, the NN circuit 100 includes multiple NN computation cores 10, so multiple data flows exist. Which data flow is used to perform calculations related to the CNN200 is controlled by the corresponding instruction command.

[0158] [Semaphore S related to the first NN computing core 10A] The semaphore S has a first semaphore S11, a second semaphore S12, a third semaphore S13, and a fourth semaphore S14 with respect to the first NN arithmetic core 10A.

[0159] The first semaphore S11 is used to control the first data flow F11 of the first NN arithmetic core 10A. The first data flow F11 is a data flow in which the first DMAC3 (Producer) writes input data a to the first memory 1 of the first NN arithmetic core 10A, and the convolution arithmetic circuit 4 (Consumer) of the first NN arithmetic core 10A reads the input data a. The first semaphore S11 has a first write semaphore S11W and a first read semaphore S11R.

[0160] The second semaphore S12 is used to control the second data flow F12 of the first NN computation core 10A. The second data flow F12 is a data flow in which the convolutional computation circuit 4 (Producer) of the first NN computation core 10A writes output data f to the second memory 2 of the first NN computation core 10A, and the quantization computation circuit 5 (Consumer) of the first NN computation core 10A reads out the output data f. The second semaphore S12 has a second write semaphore S12W and a second read semaphore S12R.

[0161] The third semaphore S13 is used to control the third data flow F13 of the first NN computation core 10A. The third data flow F13 is a data flow in which the quantization computation circuit 5 (Producer) of the first NN computation core 10A writes the quantization computation output data to the first memory 1 of the first NN computation core 10A, and the convolution computation circuit 4 (Consumer) of the first NN computation core 10A reads the quantization computation output data. The third semaphore S13 has a third write semaphore S13W and a third read semaphore S13R.

[0162] The fourth semaphore S14 is used to control the fourth data flow F14 of the first NN arithmetic core 10A. The fourth data flow F14 is a data flow in which the quantization arithmetic circuit 5 (Producer) of the second NN arithmetic core 10B writes the quantization arithmetic output data to the first memory 1 of the first NN arithmetic core 10A, and the convolution arithmetic circuit 4 (Consumer) of the first NN arithmetic core 10A reads the quantization arithmetic output data. The fourth semaphore S14 has a fourth write semaphore S14W and a fourth read semaphore S14R.

[0163] [Semaphore S related to the second NN computing core 10B] The semaphore S has a first semaphore S21, a second semaphore S22, a third semaphore S23, and a fourth semaphore S24 with respect to the second NN arithmetic core 10B.

[0164] The first semaphore S21 is used to control the first data flow F21 of the second NN arithmetic core 10B. The first data flow F21 is a data flow in which the first DMAC3 (Producer) writes input data a to the first memory 1 of the second NN arithmetic core 10B, and the convolution arithmetic circuit 4 (Consumer) of the second NN arithmetic core 10B reads the input data a. The first semaphore S21 has a first write semaphore S21W and a first read semaphore S21R.

[0165] The second semaphore S22 is used to control the second data flow F22 of the second NN arithmetic core 10B. The second data flow F22 is a data flow in which the convolution arithmetic circuit 4 (Producer) of the second NN arithmetic core 10B writes output data f to the second memory 2 of the second NN arithmetic core 10B, and the quantization arithmetic circuit 5 (Consumer) of the second NN arithmetic core 10B reads the output data f. The second semaphore S22 has a second write semaphore S22W and a second read semaphore S22R.

[0166] The third semaphore S23 is used to control the third data flow F23 of the second NN computation core 10B. The third data flow F23 is a data flow in which the quantization computation circuit 5 (Producer) of the second NN computation core 10B writes the quantization computation output data to the first memory 1 of the second NN computation core 10B, and the convolution computation circuit 4 (Consumer) of the second NN computation core 10B reads the quantization computation output data. The third semaphore S23 has a third write semaphore S23W and a third read semaphore S23R.

[0167] The fourth semaphore S24 is used to control the fourth data flow F24 of the second NN arithmetic core 10B. The fourth data flow F24 is a data flow in which the quantization arithmetic circuit 5 (Producer) of the first NN arithmetic core 10A writes the quantization arithmetic output data to the first memory 1 of the second NN arithmetic core 10B, and the convolution arithmetic circuit 4 (Consumer) of the second NN arithmetic core 10B reads the quantization arithmetic output data. The fourth semaphore S24 has a fourth write semaphore S24W and a fourth read semaphore S24R.

[0168] [First data flow F11 of the first NN computing core 10A] Figure 24 is a timing chart for the first data flow F11. The first write semaphore S11W is a semaphore that restricts the first DMAC3 from writing to the first memory 1 of the first NN computation core 10A in the first data flow F11 of the first NN computation core 10A. The first write semaphore S11W indicates the number of memory areas in the first memory 1 that have already had data read and can be written to, out of the memory areas that can store data of a predetermined size, such as input vector A. If the first write semaphore S11W is "0", the first DMAC3 cannot write to the first memory 1 in the first data flow F11 and is made to wait until the first write semaphore S11W becomes "1" or greater.

[0169] The first read semaphore S11R is a semaphore that restricts the reading of data from the first memory 1 of the first NN calculation core 10A by the convolution calculation circuit 4 of the first NN calculation core 10A in the first data flow F1 of the first NN calculation core 10A. The first read semaphore S11R indicates the number of memory areas in the first memory 1 that have already had data written to them and are readable, out of the memory areas that can store data of a predetermined size, such as an input vector A. When the first read semaphore S11R is "0", the convolution calculation circuit 4 cannot read data from the first memory 1 in the first data flow F11 and is made to wait until the first read semaphore S11R becomes "1" or greater.

[0170] The first DMAC3 initiates a DMA transfer when the command C3 is stored in the command queue 33. As shown in Figure 24, the first DMAC3 initiates a DMA transfer (DMA transfer 1) because the first write semaphore S11W is not "0". When initiating the DMA transfer, the first DMAC3 performs a P operation on the first write semaphore S11W. After the DMA transfer is complete, the first DMAC3 performs a V operation on the first read semaphore S11R.

[0171] The convolution arithmetic circuit 4 of the first NN arithmetic core 10A starts the convolution arithmetic operation when the instruction command C4 is stored in the instruction queue 45. As shown in Figure 24, since the first read semaphore S11R is "0", the convolution arithmetic circuit 4 is made to wait until the first read semaphore S11R becomes "1" or greater (Wait in decode state ST2). When the first read semaphore S11R becomes "1" due to a V operation by the first DMAC3, the convolution arithmetic circuit 4 starts the convolution arithmetic operation (convolution arithmetic 1). When starting the convolution arithmetic operation, the convolution arithmetic circuit 4 performs a P operation on the first read semaphore S11R. After the completion of the convolution arithmetic operation, the convolution arithmetic circuit 4 performs a V operation on the first write semaphore S11W.

[0172] In Figure 24, when the first DMAC3 initiates the DMA transfer labeled "DMA transfer 3," the first write semaphore S11W is "0," so the first DMAC3 is made to wait until the first write semaphore S11W becomes "1" or greater (Wait in decode state ST2). When the first write semaphore S11W becomes "1" or greater due to the V operation by the convolutional operation circuit 4, the first DMAC3 starts the DMA transfer.

[0173] The convolutional circuit 4 of the first DMAC3 and the first NN arithmetic core 10A can prevent access contention to the first memory 1 in the first data flow F11 by using the first semaphore S11. Furthermore, the convolutional circuit 4 of the first DMAC3 and the first NN arithmetic core 10A can operate independently in parallel while synchronizing data transfer in the first data flow F11 of the first NN arithmetic core 10A by using the first semaphore S11.

[0174] [Second dataflow F12 of the first NN computing core 10A] Figure 25 is a timing chart for the second data flow F12. The second write semaphore S12W is a semaphore that restricts the writing of the first NN calculation core 10A to the second memory 2 by the convolution calculation circuit 4 of the first NN calculation core 10A in the second data flow F12 of the first NN calculation core 10A. The second write semaphore S12W indicates the number of memory areas in the second memory 2 that have already had data read and can be written to, out of the memory areas that can store data of a predetermined size, such as output data f. When the second write semaphore S12W is "0", the convolution calculation circuit 4 cannot write to the second memory 2 in the second data flow F12 and is made to wait until the second write semaphore S12W becomes "1" or greater.

[0175] The second read semaphore S12R is a semaphore that restricts the reading of the second memory 2 of the first NN arithmetic core 10A by the quantization arithmetic circuit 5 of the first NN arithmetic core 10A in the second data flow F2 of the first NN arithmetic core 10A. The second read semaphore S12R indicates the number of memory areas in the second memory 2 that have already been written to and are readable, out of the memory areas that can store data of a predetermined size, such as output data f. When the second read semaphore S12R is "0", the quantization arithmetic circuit 5 cannot read from the second memory 2 in the second data flow F12 and is made to wait until the second read semaphore S12R becomes "1" or greater.

[0176] As shown in Figure 25, the convolution circuit 4 of the first NN computation core 10A performs a P operation on the second write semaphore S12W when starting the convolution operation. After the completion of the convolution operation, the convolution circuit 4 performs a V operation on the second read semaphore S12R.

[0177] The quantization arithmetic circuit 5 of the first NN arithmetic core 10A starts quantization when the instruction command C5 is stored in the instruction queue 55. As shown in Figure 25, since the second read semaphore S12R is "0", the quantization arithmetic circuit 5 is made to wait until the second read semaphore S12R becomes "1" or greater (Wait in decode state ST2). When the second read semaphore S12R becomes "1" due to the V operation by the convolution arithmetic circuit 4, the quantization arithmetic circuit 5 starts quantization (quantization operation 1). When starting quantization, the quantization arithmetic circuit 5 performs a P operation on the second read semaphore S12R. After the completion of quantization, the quantization arithmetic circuit 5 performs a V operation on the second write semaphore S12W.

[0178] In Figure 25, when the quantization circuit 5 starts the quantization operation labeled "Quantization Operation 2," the second read semaphore S12R is "0," so the quantization circuit 5 is made to wait until the second read semaphore S12R becomes "1" or greater (Wait in decode state ST2). Once the second read semaphore S12R becomes "1" or greater due to the V operation by the convolution circuit 4, the quantization circuit 5 starts the quantization operation.

[0179] The convolution circuit 4 and the quantization circuit 5 of the first NN computation core 10A can prevent access contention to the second memory 2 in the second data flow F12 by using the second semaphore S12. Furthermore, the convolution circuit 4 and the quantization circuit 5 of the first NN computation core 10A can operate independently in parallel while synchronizing data transfer in the second data flow F12 by using the second semaphore S12.

[0180] [Third dataflow F13 of the first NN computing core 10A] Figure 26 is a timing chart for the third data flow F13. The third write semaphore S13W is a semaphore that restricts the writing of the first NN computation core 10A to the first memory 1 by the quantization computation circuit 5 of the first NN computation core 10A in the third data flow F13 of the first NN computation core 10A. The third write semaphore S13W indicates the number of memory areas in the first memory 1 that have already had data read and can be written to, out of the memory areas that can store data of a predetermined size, such as the quantization computation output data of the quantization computation circuit 5. When the third write semaphore S13W is "0", the quantization computation circuit 5 cannot write to the first memory 1 in the third data flow F13 and is made to wait until the third write semaphore S13W becomes "1" or greater.

[0181] The third read semaphore S13R is a semaphore that restricts the reading of data from the first memory 1 of the first NN calculation core 10A by the convolution calculation circuit 4 of the first NN calculation core 10A in the third data flow F13 of the first NN calculation core 10A. The third read semaphore S13R indicates the number of memory areas in the first memory 1 that have already been written to and are readable, out of the memory areas that can store data of a predetermined size, such as the quantization calculation output data of the quantization calculation circuit 5. When the third read semaphore S13R is "0", the convolution calculation circuit 4 cannot read from the first memory 1 in the third data flow F13 and is made to wait until the third read semaphore S13R becomes "1" or greater.

[0182] As shown in Figure 26, the quantization circuit 5 of the first NN computation core 10A performs a P operation on the third write semaphore S13W when starting the quantization operation. After the completion of the convolution operation, the quantization circuit 5 performs a V operation on the third read semaphore S13R.

[0183] The convolution arithmetic circuit 4 of the first NN arithmetic core 10A starts the convolution arithmetic operation when the instruction command C4 is stored in the instruction queue 45. As shown in Figure 26, since the third read semaphore S13 is "0", the convolution arithmetic circuit 4 is made to wait until the third read semaphore S13R becomes "1" or greater (Wait in decode state ST2). When the third read semaphore S13R becomes "1" due to the V operation by the quantization arithmetic circuit 5, the convolution arithmetic circuit 4 starts the convolution arithmetic operation (convolution arithmetic 5). When the convolution arithmetic circuit 4 starts the convolution arithmetic operation, it performs a P operation on the third read semaphore S13R. After the completion of the convolution arithmetic operation, the convolution arithmetic circuit 4 performs a V operation on the third write semaphore S13W.

[0184] In Figure 26, when the convolution circuit 4 starts the convolution operation labeled "Convolution Operation 7," the third read semaphore S13R is "0," so the convolution circuit 4 is made to wait until the third read semaphore S13R becomes "1" or greater (Wait in decode state ST2). Once the third read semaphore S13R becomes "1" or greater due to the V operation by the quantization circuit 5, the convolution circuit 4 starts the convolution operation.

[0185] The quantization circuit 5 and the convolution circuit 4 of the first NN computation core 10A can prevent access contention to the first memory 1 in the third data flow F13 by using the third semaphore S13. Furthermore, the quantization circuit 5 and the convolution circuit 4 of the first NN computation core 10A can operate independently in parallel while synchronizing data transfer in the third data flow F13 by using the third semaphore S13.

[0186] [Fourth dataflow F14 of the first NN computing core 10A] Figure 27 is the timing chart for the fourth data flow F14. The fourth write semaphore S14W is a semaphore that restricts the writing of the first memory 1 of the first NN computation core 10A by the quantization computation circuit 5 of the second NN computation core 10B in the fourth data flow F14 of the first NN computation core 10A. The fourth write semaphore S14W indicates the number of memory areas in the first memory 1 that have already had data read and can be written to, out of the memory areas that can store data of a predetermined size, such as the quantization computation output data of the quantization computation circuit 5. When the fourth write semaphore S14W is "0", the quantization computation circuit 5 cannot write to the first memory 1 in the fourth data flow F14 and is made to wait until the fourth write semaphore S14W becomes "1" or greater.

[0187] The fourth read semaphore S14R is a semaphore that restricts the reading of data from the first memory 1 of the first NN arithmetic core 10A by the convolution arithmetic circuit 4 of the first NN arithmetic core 10A in the fourth data flow F14 of the first NN arithmetic core 10A. The fourth read semaphore S14R indicates the number of memory areas in the first memory 1 that have already been written to and are readable, out of the memory areas that can store data of a predetermined size, such as the quantization arithmetic output data of the quantization arithmetic circuit 5. When the fourth read semaphore S14R is "0", the convolution arithmetic circuit 4 cannot read from the first memory 1 in the fourth data flow F14 and is made to wait until the fourth read semaphore S14R becomes "1" or greater.

[0188] As shown in Figure 27, the quantization circuit 5 of the second NN computation core 10B performs a P operation on the fourth write semaphore S14W when starting the quantization operation. After the completion of the convolution operation, the quantization circuit 5 performs a V operation on the fourth read semaphore S14R.

[0189] The convolution arithmetic circuit 4 of the first NN arithmetic core 10A starts the convolution arithmetic operation when the instruction command C4 is stored in the instruction queue 45. As shown in Figure 27, since the fourth read semaphore S14R is "0", the convolution arithmetic circuit 4 is made to wait until the fourth read semaphore S14R becomes "1" or greater (Wait in decode state ST2). When the fourth read semaphore S14R becomes "1" due to the V operation by the quantization arithmetic circuit 5, the convolution arithmetic circuit 4 starts the convolution arithmetic operation (convolution arithmetic operation 9). When the convolution arithmetic circuit 4 starts the convolution arithmetic operation, it performs a P operation on the fourth read semaphore S14R. After the completion of the convolution arithmetic operation, the convolution arithmetic circuit 4 performs a V operation on the fourth write semaphore S14W.

[0190] In Figure 27, when the convolution circuit 4 starts the convolution operation labeled "Convolution Operation 10," the fourth read semaphore S14R is "0," so the convolution circuit 4 is made to wait until the fourth read semaphore S14R becomes "1" or greater (Wait in decode state ST2). When the fourth read semaphore S14R becomes "1" or greater due to the V operation by the quantization circuit 5, the convolution circuit 4 starts the convolution operation.

[0191] The quantization circuit 5 of the second NN computation core 10B and the convolution circuit 4 of the first NN computation core 10A can prevent access contention to the first memory 1 in the fourth data flow F14 by using the fourth semaphore S14. Furthermore, the quantization circuit 5 of the second NN computation core 10B and the convolution circuit 4 of the first NN computation core 10A can operate independently in parallel while synchronizing data transfer between multiple NN computation cores 10 in the fourth data flow F14 by using the fourth semaphore S14.

[0192] The first memory 1 of the first NN arithmetic core 10A is shared by three data flows (first data flow F11, third data flow F13, and fourth data flow F14). The NN circuit 100 can distinguish between the first data flow F11, the third data flow F13, and the fourth data flow F14 and synchronize data transfer by separately providing the first semaphore S11, the third semaphore S13, and the fourth semaphore S14.

[0193] [First dataflow F21 of the second NN computing core 10B] The first data flow F21 of the second NN arithmetic core 10B is equivalent to the first data flow F11 of the first NN arithmetic core 10A. The first DMAC3 and the convolution arithmetic circuit 4 of the second NN arithmetic core 10B can prevent access contention to the first memory 1 in the first data flow F21 by using the first semaphore S21. In addition, the first DMAC3 and the convolution arithmetic circuit 4 of the second NN arithmetic core 10B can operate independently in parallel while synchronizing data transfer in the first data flow F21 of the second NN arithmetic core 10B by using the first semaphore S21.

[0194] [Second dataflow F22 of the second NN computing core 10B] The second data flow F22 of the second NN computation core 10B is equivalent to the second data flow F12 of the first NN computation core 10A. The convolutional operation circuit 4 and the quantization operation circuit 5 of the second NN computation core 10B can prevent access contention to the second memory 2 in the second data flow F22 by using the second semaphore S22. Furthermore, the convolutional operation circuit 4 and the quantization operation circuit 5 of the second NN computation core 10B can operate independently in parallel while synchronizing data transfer in the second data flow F22 by using the second semaphore S22.

[0195] [Third dataflow F23 of the second NN computing core 10B] The third data flow F23 of the second NN computation core 10B is equivalent to the third data flow F13 of the first NN computation core 10A. The quantization computation circuit 5 and the convolution computation circuit 4 of the second NN computation core 10B can prevent access contention to the first memory 1 in the third data flow F23 by using the third semaphore S23. Furthermore, the quantization computation circuit 5 and the convolution computation circuit 4 of the second NN computation core 10B can operate independently in parallel while synchronizing data transfer in the third data flow F23 by using the third semaphore S23.

[0196] [Fourth dataflow F24 of the second NN computing core 10B] The fourth data flow F24 of the second NN computation core 10B is equivalent to the fourth data flow F14 of the first NN computation core 10A. The quantization computation circuit 5 of the first NN computation core 10A and the convolution computation circuit 4 of the second NN computation core 10B can prevent access contention to the first memory 1 in the fourth data flow F24 by using the fourth semaphore S24. In addition, the quantization computation circuit 5 of the first NN computation core 10A and the convolution computation circuit 4 of the second NN computation core 10B can operate independently in parallel while synchronizing data transfer between multiple NN computation cores 10 in the fourth data flow F24 by using the fourth semaphore S24.

[0197] [Operation of the convolution circuit 4 of the first NN computing core 10A] The convolution circuit 4 of the first NN calculation core 10A reads from the first memory 1 of the first NN calculation core 10A and writes to the second memory 2 of the first NN calculation core 10A when performing a convolution operation. In other words, the convolution circuit 4 is a Consumer for the three data flows (first data flow F11, third data flow F13, and fourth data flow F14) and a Producer for the second data flow F12. Therefore, when the convolution circuit 4 starts a convolution operation, it performs a P operation on the read semaphore corresponding to the data flow (first read semaphore S11R, third read semaphore S13R, or fourth read semaphore S14R) (see Figures 24, 26, and 27) and a P operation on the second write semaphore S12W (see Figure 24). After completing the convolution operation, the convolution circuit 4 performs a V operation on the write semaphores corresponding to the data flow (first write semaphore S11W, third write semaphore S13W, fourth write semaphore S14W) (see Figures 24, 26, and 27), and performs a V operation on the second read semaphore S12R (see Figure 25).

[0198] The convolution circuit 4 of the first NN computation core 10A waits until the read semaphores corresponding to the data flow (first read semaphore S11R, third read semaphore S13R, or fourth read semaphore S14R) are "1" or greater, and the second write semaphore S12W is "1" or greater, before starting the convolution operation (Wait in decode state ST2).

[0199] [Operation of the quantization arithmetic circuit 5 of the first NN arithmetic core 10A] The quantization circuit 5 of the first NN arithmetic core 10A reads from the second memory 2 of the first NN arithmetic core 10A and writes to the first memory 1 of the first NN arithmetic core 10A or the first memory 1 of the second NN arithmetic core 10B when performing quantization operations. In other words, the quantization circuit 5 is a Consumer in the second data flow F12 and a Producer in the two data flows (third data flow F13 and fourth data flow F24). Therefore, when starting quantization operations, the quantization circuit 5 performs a P operation on the second read semaphore S12R (see Figure 25) and a P operation on the write semaphore corresponding to the data flow (third write semaphore S13W or fourth write semaphore S24W) (see Figure 26). After completing the quantization operation, the quantization circuit 5 performs a V operation on the second write semaphore S12W (see Figure 25) and then performs a V operation on the read semaphore corresponding to the data flow (the third read semaphore S13R or the fourth read semaphore S24R) (see Figure 26).

[0200] The quantization arithmetic circuit 5 of the first NN arithmetic core 10A waits until the second read semaphore S12R is "1" or greater, and the write semaphore corresponding to the data flow (third write semaphore S13W or fourth write semaphore S24W) is "1" or greater before starting the quantization arithmetic (Wait in decode state ST2).

[0201] The quantization circuit 5 of the first NN computation core 10A can change the first memory 1 that stores the quantization output data by switching between the third data flow F13 and the fourth data flow F24. Similarly, the quantization circuit 5 of the second NN computation core 10B can change the first memory 1 that stores the quantization output data by switching between the third data flow F23 and the fourth data flow F14.

[0202] According to the neural network circuit 100 of this embodiment, the NN circuit 100, which can be embedded in embedded devices such as IoT devices, can be operated with high performance. By connecting multiple NN computation cores 10, more neural network computations can be performed efficiently and at high speed.

[0203] Although the first embodiment of the present invention has been described in detail above with reference to the drawings, the specific configuration is not limited to this embodiment, and design changes and the like are also included within the scope of the gist of the present invention. Furthermore, the components shown in the above-described embodiment and modified examples can be combined as appropriate.

[0204] (Second embodiment) A second embodiment of the present invention will be described with reference to Figures 28 to 31. In the following description, components that are common to those already described will be denoted by the same reference numerals, and redundant descriptions will be omitted. The neural network circuit 100B according to the second embodiment (hereinafter referred to as "NN circuit 100B") differs from the neural network circuit 100 according to the first embodiment in that the convolutional operation circuit 4 is different.

[0205] [NN circuit 100B] The NN circuit 100B comprises a first DMAC 3, a controller 6, an IFU 7, a shared memory 8, a second DMAC 9, and at least one neural network computation core 10E (hereinafter also referred to as "NN computation core 10E").

[0206] [NN Computation Core 10E] The NN computation core 10E comprises a first memory 1, a second memory 2, a convolution operation circuit 4B, and a quantization operation circuit 5.

[0207] [Convolution arithmetic circuit 4B] The convolution operation circuit 4B is a circuit that performs convolution operations in the convolution layer 210 of the trained CNN200. The convolution operation circuit 4B reads the input data a stored in the first memory 1 and performs a convolution operation on the input data a. The convolution operation circuit 4B writes the convolution operation output data to the second memory 2.

[0208] Figure 28 is an internal block diagram of the convolution arithmetic circuit 4B. The convolution arithmetic circuit 4B includes a weight memory 41, a multiplier 42B, an accumulator circuit 43, and a state controller 44.

[0209] Figure 29 is an internal block diagram of the multiplier 42. The multiplier (arithmetic unit array) 42B multiplies each element a(x+i,y+j,ci) of the partitioned input data a(x+i,y+j,co) with each element w(i,j,ci,di) of the partitioned weight w(i,j,co,do). The multiplier 42 has Bc × Bd multiply-accumulate unit arrays 42A, and can perform the multiplication of elements a(x+i,y+j,ci) of the partitioned input data a(x+i,y+j,co) and elements w(i,j,ci,di) of the partitioned weight w(i,j,co,do) in parallel.

[0210] The multiplier (arithmetic array) 42B reads the elements a and w required for multiplication from the first memory 1 and weight memory 41, and performs the multiplication. The multiplier 42 outputs the result of Bd sum-of-accumulate operations O(x+i, y+j, di).

[0211] Note that the number of multiply-accumulate unit arrays 42A included in the multiplier 42 is not limited to Bc × Bd. For example, the number of multiply-accumulate unit arrays 42A may be (Bc / P) × Bd (where P is Bc or a divisor of Bc). In this case, the multiply-accumulate unit array 42A divides the divided input data a(x+i, y+j, co) into P units along the c-axis.

[0212] The multiplier (arithmetic unit array) 42B reads out the element a(x+i,y+j,ci), the element a(x+i,y+y1+j,ci), and the element a(x+i,y+y2+j,ci) from the first memory 1 (0 < y1 < y2). When y1 = 1 and y2 = 2, the three sets of element a are line data that are continuous in the y-axis direction in the xy-plane. When y1 = 1 + ST and y2 = 2 + 2ST, the three sets of element a are line data that are separated by a predetermined number of lines ST in the y-axis direction (ST is the stride in the y-axis direction). The multiplier (arithmetic unit array) 42B may have a line memory for storing the element a.

[0213] The first memory 1 is preferably a multi-bank memory. In this case, the element a(x+i,y+j,ci), the element a(x+i,y+y1+j,ci), and the element a(x+i,y+y2+j,ci) are stored in different banks, and each element can be read out independently at high speed.

[0214] FIG. 30 is an internal block diagram of the dot product operation unit array 42A. The dot product operation unit array 42A multiplies the element a and the element w. The dot product operation unit array 42A has three dot product operation units 47B. In the following description, the three dot product operation units 47B are referred to as the first dot product operation unit 471, the second dot product operation unit 472, and the third dot product operation unit 473.

[0215] FIG. 31 is an internal block diagram of the dot product operation unit 47B. The dot product operation unit 47B multiplies the element A(ci) of the input vector A and the element W(ci,di) of the weight matrix W. The dot product operation unit 47 outputs the multiplication result s(ci). The element A(ci) is a 2-bit unsigned integer (0, 1, 2, 3). The element W(ci,di) is a 1-bit signed integer (0, 1), where the value "0" represents +1 and the value "1" represents -1.

[0216] The multiply-accumulate unit 47B includes an inverter 47a, a selector 47b, and an adder 47c. The multiply-accumulate unit 47 performs multiplication using only the inverter 47a and selector 47b, without using a multiplier. Selector 47b selects the input of element A(ci) when element W(ci,di) is "0". Selector 47b selects the complement of element A(ci) inverted by the inverter when element W(ci,di) is "1". Element W(ci,di) is also input to the carry-in of adder 47c. Adder 47c outputs the value obtained by adding element A(ci) to m(ci,di) when element W(ci,di) is "0". Adder 47c outputs the value obtained by subtracting element A(ci) from m(ci,di) when W(ci,di) is "1".

[0217] The first sum-of-products unit 471 multiplies element a(X,Y,ci) by element w(i,j,ci,di) (where X is an arbitrary x-coordinate in the input data a, and Y is an arbitrary y-coordinate in the input data a). The first sum-of-products unit 471 outputs output m(i,j,ci,di) to adder 47A.

[0218] The second sum-of-products unit 472 multiplies element a(X,Y+y1,ci) by element w(i,j+1,ci,di). The second sum-of-products unit 472 outputs output m(i,j+1,ci,di) to adder 47A.

[0219] The third sum-of-products unit 473 multiplies element a(X,Y+y2,ci) by element w(i,j+2,ci,di). The third sum-of-products unit 473 outputs output m(i,j+2,ci,di) to adder 47A.

[0220] The adder 47A adds its output m(i,j,ci,di), its output m(i,j+1,ci,di), its output m(i,j+2,ci,di), and the multiplication result S(x+i,y+j,ci,di) from another multiply-accumulate unit 47B, and outputs the summation result S(x+i,y+j,ci+1,di).

[0221] According to the neural network circuit 100B of this embodiment, operations can be parallelized by the multiply-accumulate unit array 42A, thereby speeding up convolution operations. The neural network circuit 100B can suitably perform convolution operations even when the stride ST in the y-axis direction in the convolution operation is 2 or more.

[0222] Although a second embodiment of the present invention has been described in detail above with reference to the drawings, the specific configuration is not limited to this embodiment, and design changes and the like are also included within the scope of the gist of the present invention. Furthermore, the components shown in the above-described embodiments and modifications can be combined as appropriate.

[0223] (Third embodiment) A third embodiment of the present invention will be described with reference to Figures 32 to 36. In the following description, components that are common to those already described will be denoted by the same reference numerals, and redundant descriptions will be omitted. The neural network circuit 100G according to the third embodiment (hereinafter referred to as "NN circuit 100G") further has a clock gating function and a power gating function compared to the neural network circuit 100 according to the first embodiment.

[0224] [NN circuit 100G] Figure 32 shows the overall configuration of the NN circuit 100G according to this embodiment. The NN circuit 100G comprises a first DMAC 3G, a controller 6, an IFU 7, a shared memory 8, a second DMAC 9, and at least one neural network computation core 10G (hereinafter also referred to as "NN computation core 10G"). The NN circuit 100G does not necessarily have a shared memory 8 and a second DMAC 9.

[0225] The NN circuit 100G can implement multiple NN computation cores 10G. The NN circuit 100G illustrated in Figure 32 can implement up to four NN computation cores 10G. Multiple NN computation cores 10G, similar to the NN computation core 10 in the first embodiment, constitute an "NN computation multicore 10M" that coordinately executes at least some of the calculations of the NN200. Multiple NN computation cores 10G are daisy-chained, similar to the first embodiment. Note that the number of NN computation cores 10 that can be implemented in the NN circuit 100G is not limited to four.

[0226] [First DMAC3G] Figure 33 is an internal block diagram of the first DMAC3G. The first DMAC3G, like the first DMAC3 in the first embodiment, is connected to an external bus EB and performs data transfer between an external memory 120 such as DRAM and the NN computing core 10G. The first DMAC3G includes a data transfer circuit 31, a state controller 32, and a clock control unit 39.

[0227] Figure 34 is a timing chart showing the operation of the clock control unit 39. The clock control unit 39 generates a gated clock (third clock) GC3 from the clock CK supplied to the NN circuit 100G based on the clock enable signal CE3. When the clock enable signal CE3 is negated and set to disabled (Low in Figure 34), the toggling of the gated clock GC3 is stopped. When the clock enable signal CE3 is asserted and set to enabled (High in Figure 34), the toggling of the gated clock GC3 is started. The circuit for generating the gated clock GC3 is a circuit appropriately selected from known clock gating circuits.

[0228] The clock enable signal CE3 is controlled by the state controller 32. When the control circuit 34 of the state controller 32 determines in the decode state ST2 that the operation of the data transfer circuit 31 instructed by the instruction command C3 is not executable, it waits (Wait) until it becomes executable. During the period when the control circuit 34 waits until the above operation becomes executable, it negates the clock enable signal CE3 and sets it to invalid (Disable). As a result, the toggling of the gated clock GC3 stops. When the control circuit 34 of the state controller 32 transitions from the decode state ST2 to the execution state ST3 when the above operation becomes executable, it asserts the clock enable signal CE3 and sets it to valid (Enable). As a result, when the control circuit 34 is in the execution state ST3, the toggling of the gated clock GC3 resumes.

[0229] As shown in FIG. 33, the generated gated clock GC3 is output to a part of the state controller 32 and the data transfer circuit 31 and is used as an operation clock.

[0230] In the idle state ST1, the clock control unit 39 may negate the clock enable signal CE'3 and set it to invalid. Further, in the idle state ST1, the control circuit 34 may stop power supply (power gating) to the circuit where the gated clock GC3 is provided and transition to a power saving mode.

[0231] [NN arithmetic core 10G] The NN arithmetic core 10G includes a first memory 1, a second memory 2, a convolution arithmetic circuit 4G, and a quantization arithmetic circuit 5G.

[0232] [Convolution arithmetic circuit 4G] FIG. 35 is an internal block diagram of the convolution arithmetic circuit 4G. The convolution arithmetic circuit 4G has a weight memory 41, a multiplier 42, an accumulator circuit 43, a state controller 44, and a clock control unit 49.

[0233] The clock control unit 49 generates a gated clock (first clock) GC4 from the clock CK supplied to the NN circuit 100G based on the clock enable signal CE4. As shown in Figure 34, when the clock enable signal CE4 is negated and set to disabled, the toggling of the gated clock GC4 is stopped. When the clock enable signal CE4 is asserted and set to enabled, the toggling of the gated clock GC4 is started. The clock control unit 49 has the same configuration as the clock control unit 39 of the first DMAC 3G.

[0234] The clock enable signal CE4 is controlled by the state controller 44. In the decode state ST2, if the control circuit 46 of the state controller 44 determines that the operation of the multiplier 42, accumulator circuit 43, etc., instructed by command C4 is not possible, it waits until it becomes possible. During the waiting period until the above operation becomes possible, the control circuit 46 negates the clock enable signal CE4 and sets it to disabled. As a result, the gated clock GC4 is stopped from toggling. When the above operation becomes possible and the state controller 44 transitions from the decode state ST2 to the execution state ST3, the control circuit 46 of the state controller 44 asserts the clock enable signal CE4 and sets it to enabled. As a result, when the control circuit 46 is in the execution state ST3, the gated clock GC4 is restarted from toggling.

[0235] The generated gated clock GC4 is output to a portion of the state controller 44, the weight memory 41, the multiplier 42, and the accumulator circuit 43, as shown in Figure 35, and is used as the operating clock.

[0236] The clock control unit 49 may negate and disable the clock enable signal CE4 in the idle state ST1. Furthermore, the control circuit 46 may stop supplying power to the circuit to which the gated clock GC4 is provided (power gating) in the idle state ST1 and transition to a power saving mode.

[0237] [Quantization operation circuit 5G] Figure 36 is an internal block diagram of the quantization arithmetic circuit 5G. The quantization operation circuit 5G includes a quantization parameter memory 51, a vector operation circuit 52, a quantization circuit 53, a state controller 54, and a clock control unit 59.

[0238] The clock control unit 59 generates a gated clock (third clock) GC5 from the clock CK supplied to the NN circuit 100G based on the clock enable signal CE5. As shown in Figure 34, when the clock enable signal CE5 is negated and set to disabled, the toggling of the gated clock GC5 is stopped. When the clock enable signal CE5 is asserted and set to enabled, the toggling of the gated clock GC5 is started. The clock control unit 59 has the same configuration as the clock control unit 39 of the first DMAC 3G.

[0239] The clock enable signal CE5 is controlled by the state controller 54. In the decode state ST2, if the control circuit 56 of the state controller 54 determines that the operation of the vector arithmetic circuit 52, quantization circuit 53, etc., instructed by command C5 is not possible, it waits until it becomes possible. During the waiting period until the above operation becomes possible, the control circuit 56 negates the clock enable signal CE5 and sets it to disabled. As a result, the gated clock GC5 is stopped from toggling. When the above operation becomes possible and the state controller 54 transitions from the decode state ST2 to the execution state ST3, the control circuit 56 of the state controller 54 asserts the clock enable signal CE5 and sets it to enabled. As a result, when the control circuit 56 is in the execution state ST3, the gated clock GC5 is restarted from toggling.

[0240] The generated gated clock GC5 is output to a portion of the state controller 54, the quantization parameter memory 51, the vector operation circuit 52, and the quantization circuit 53, as shown in Figure 36, and is used as the operating clock.

[0241] The clock control unit 59 may negate the clock enable signal CE5 and disable it in the idle state ST1. Furthermore, the control circuit 56 may stop supplying power to the circuit to which the gated clock GC5 is provided (power gating) in the idle state ST1 and transition to a power saving mode.

[0242] According to the neural network circuit 100G of this embodiment, power consumption can be reduced by clock gating and power gating. The first DMAC 3G, the convolutional operation circuit 4G, and the quantization operation circuit 5G each independently perform clock gating and power gating.

[0243] Although a third embodiment of the present invention has been described in detail above with reference to the drawings, the specific configuration is not limited to this embodiment, and design changes and the like are also included within the scope of the gist of the present invention. Furthermore, the components shown in the above-described embodiments and modifications can be combined as appropriate.

[0244] (Fourth embodiment) A fourth embodiment of the present invention will be described with reference to Figure 37. In the following description, components that are common to those already described will be denoted by the same reference numerals, and redundant descriptions will be omitted. The neural network circuit 100H according to the fourth embodiment (hereinafter referred to as "NN circuit 100H") further includes a multi-core management unit 11 compared to the neural network circuit 100 according to the first embodiment.

[0245] [NN circuit 100H] Figure 37 shows the overall configuration of the NN circuit 100H according to this embodiment. The NN circuit 100H comprises a first DMAC 3, a controller 6, an IFU 7, a shared memory 8, a second DMAC 9, at least one NN arithmetic core 10, and a multicore management unit 11.

[0246] The multicore management unit 11 monitors the status of the NN computation multicore 10M and manages the clock and power supplied to the NN computation cores 10. It supplies clock and power to the operational NN computation cores 10 of the NN computation multicore 10M, and does not supply clock and power to at least some of the inoperable NN computation cores 10. In other words, the multicore management unit 11 performs at least one of clock gating and power gating for each NN computation core 10 according to the operating status of the NN computation cores 10. The NN circuit 100H can improve computational performance by incorporating multiple NN computation cores 10, and can suitably suppress the increase in power consumption that comes with an increase in circuit size.

[0247] The multi-core management unit 11 may be able to forcibly select an operable NN arithmetic core 10. For example, the multi-core management unit 11 makes some of the NN arithmetic cores 10 operable and sets the other NN arithmetic cores 10 to inoperable. The multi-core management unit 11 stops providing clock and power to the NN arithmetic cores 10 set to inoperable. By limiting the operable NN arithmetic cores 10, the multi-core management unit 11 can reduce power consumption although there is a decrease in arithmetic performance. Also, the multi-core management unit 11 can improve arithmetic performance although there is an increase in power consumption by setting all the NN arithmetic cores 10 to operable.

[0248] According to the neural network circuit 100H according to this embodiment, power consumption can be reduced by clock gating or power gating. The multi-core management unit 11 independently performs clock gating or power gating for each NN arithmetic core 10.

[0249] The NN circuit 100H may also perform the clock gating and power gating performed by the first DMAC 3G, the convolution arithmetic circuit 4G, and the quantization arithmetic circuit 5G in the third embodiment.

[0250] As described above, the fourth embodiment of the present invention has been described in detail with reference to the drawings, but the specific configuration is not limited to this embodiment, and design changes and the like within the scope not departing from the gist of the present invention are also included. Also, the constituent elements shown in the above embodiments and modification examples can be configured by appropriately combining them.

[0251] (Modification Example 1) In the above embodiment, the plurality of NN arithmetic cores 10 were daisy-chain connected, but the connection mode of the plurality of NN arithmetic cores 10 is not limited to this. The NN arithmetic core 10 only needs to be connected so as to be able to input and output data with at least one other NN arithmetic core 10. Even when the connection modes of the plurality of NN arithmetic cores 10 are different, the NN circuit 100 is controlled using the semaphore S provided for each data flow.

[0252] (Modification 2) In the above embodiment, the first memory 1 and the second memory 2 were separate memories, but the configurations of the first memory 1 and the second memory 2 are not limited to this. For example, the first memory 1 and the second memory 2 may be the first memory area and the second memory area of ​​the same memory.

[0253] (Variation 3) In the above embodiment, the semaphore S was provided for the first data flow (F11, F21), the second data flow (F12, F22), the third data flow (F13, F23), and the fourth data flow (F14, F24), but the form of the semaphore S is not limited thereto. For example, the semaphore S may be provided for the data flow in which the first DMAC3 writes weight w to the weight memory 41 and the multiplier 42 reads out weight w. For example, the semaphore S may be provided for the data flow in which the first DMAC3 writes quantization parameter q to the quantization parameter memory 51 and the quantization circuit 53 reads out quantization parameter q.

[0254] (Modification 4) For example, the data input to the NN circuit 100 described in the above embodiment is not limited to a single format, but can consist of still images, moving images, audio, text, numerical values, and combinations thereof. Furthermore, the data input to the NN circuit 100 is not limited to measurement results from physical quantity measuring instruments such as optical sensors, thermometers, Global Positioning System (GPS) measuring instruments, angular velocity measuring instruments, and anemometers that can be mounted on the edge device on which the NN circuit 100 is provided. It may also be a combination of peripheral information such as base station information, vehicle / ship information, weather information, congestion information, financial information, and personal information received from peripheral devices via wired or wireless communication.

[0255] (Variation 5) Edge devices equipped with the NN circuit 100 are envisioned to include mobile devices such as mobile phones powered by batteries, smart devices such as personal computers, digital cameras, game consoles, and robotic products, but are not limited to these. It can also be used in products with a high demand for peak power limiting via Power on Ethernet (PoE), reduced product heat generation, or extended operating time, yielding effects not seen in other prior examples. For example, applying it to in-vehicle cameras mounted on vehicles and ships, or surveillance cameras installed in public facilities and on roads, not only enables long-duration shooting but also contributes to weight reduction and increased durability. Similar effects can also be achieved by applying it to display devices such as televisions and displays, medical equipment such as medical cameras and surgical robots, and work robots used in manufacturing and construction sites.

[0256] (Experimental variation 6) The NN circuit 100 may be implemented using one or more processors, either partially or entirely. For example, the NN circuit 100 may have part or all of its input or output layer implemented by software processing by a processor. The part of the input or output layer implemented by software processing could be, for example, data normalization or transformation. This allows for support of various input or output formats. The software executed by the processor may be configured to be rewritable using communication means or external media.

[0257] (Example 7) The NN circuit 100 may implement part of the processing in CNN200 by combining it with a Graphics Processing Unit (GPU) or the like on the cloud. The NN circuit 100 can perform more complex processing with fewer resources by performing processing on the cloud in addition to processing performed on the edge device where the NN circuit 100 is installed, or by performing processing on the edge device in addition to processing on the cloud. With such a configuration, the NN circuit 100 can reduce the amount of communication between the edge device and the cloud through distributed processing.

[0258] Furthermore, the effects described herein are merely descriptive or illustrative and not limiting. In other words, the technology relating to this disclosure may produce other effects that will be apparent to those skilled in the art from the description herein, in addition to or in lieu of the effects described herein. [Industrial applicability]

[0259] This invention can be applied to neural network computations. [Explanation of Symbols]

[0260] 200 Convolutional Neural Networks 100, 100B, 100G, 100H Neural Network Circuit (NN Circuit) 10,10G Neural Network Computing Core (NN Computing Core) 10A First Neural Network Computing Core (First NN Computing Core) 10B Second Neural Network Computing Core (Second NN Computing Core) 10M Neural Network Computing Multicore (NN Computing Multicore) 11 Multicore Management Unit 1 First Memory 2 Second Memory 3.3G First DMA Controller (First DMAC) 4,4B,4G Convolutional Circuit 42,42B multiplier 43. Accumulator Circuit 49 Clock control unit 5,5G quantization calculation circuit 52 Vector Operation Circuits 53 Quantization circuit 59 Clock Control Unit 6 Controllers 61 registers 7 IFU 8. Shared memory 9. Second DMA Controller (Second DMAC) S semaphore S1 First Semaphore S11R, S21R First Lead Semaphore S11W, S21W First Light Semaphore S2 Second Semaphore S12R, S22R Second Lead Semaphore S12W, S22W Second Light Semaphore S3 Third Semaphore S13R, S23R Third Reed Semaphore S13W, S23W Third Light Semaphore S4 Fourth Semaphore S14R, S24R Fourth Lead Semaphore S14W, S24W Fourth Light Semaphore F11, F21 First Dataflow F12, F22 Second Dataflow F13, F23 Third data flow (loopback data flow) F14, F24 Fourth data flow (bypass data flow)

Claims

1. It has multiple neural network computing cores, each having a convolution circuit that performs convolution operations and a quantization circuit that performs quantization operations. The aforementioned plurality of neural network computing cores are connected in a way that enables data input and output. Neural network circuit.

2. The aforementioned plurality of neural network computing cores are connected in a daisy-chain configuration. The neural network circuit according to claim 1.

3. The neural network processing cores other than the final stage are connected to the neural network processing cores of subsequent stages. The final stage neural network computing core is connected to the first stage neural network computing core. The neural network circuit according to claim 1.

4. The aforementioned neural network computing core is A first memory for storing input data input to the convolution arithmetic circuit, A second memory for storing the convolution operation output data of the aforementioned convolution operation circuit, Equipped with, The quantization output data of the quantization circuit is stored in the first memory. The quantization operation output data stored in the first memory is input to the convolution operation circuit as input data. A neural network circuit according to any one of claims 1 to 3.

5. In the first neural network processing core and the second neural network processing core, which are the aforementioned neural network processing cores, The quantization output data of the quantization circuit of the first neural network computing core can be stored in the first memory of the second neural network computing core. The neural network circuit according to claim 4.

6. In the neural network computation core, the first memory, the convolution operation circuit, the second memory, and the quantization operation circuit are formed in a loop. The neural network circuit according to claim 5.

7. The first memory, the convolution arithmetic circuit, the second memory, and the quantization arithmetic circuit are connected to each other so as to be repeated in the same order. The neural network circuit according to claim 6.

8. A third semaphore controls the data flow in which the quantization operation circuit of the first neural network operation core writes the quantization operation output data to the first memory of the first neural network operation core, and the convolution operation circuit of the first neural network operation core reads the quantization operation output data. A fourth semaphore controls the data flow in which the quantization circuit of the second neural network computing core writes the quantization output data to the first memory of the first neural network computing core, and the convolution circuit of the first neural network computing core reads the quantization output data. Equipped with, The neural network circuit according to claim 6.

9. When the convolution circuit waits to execute the convolution operation, it enables clock gating of the first clock supplied to at least a portion of the convolution circuit. The neural network circuit according to claim 1.

10. When the quantization circuit waits to execute the quantization operation, it enables clock gating of the second clock supplied to at least a portion of the quantization circuit. The neural network circuit according to claim 1.

11. The system further includes a multicore management unit that manages the clock supplied to each of the aforementioned multiple neural network computing cores. The neural network circuit according to claim 1.

12. A neural network computation method using a first neural network computation core and a second neural network computation core, The output data of the first neural network processing core is switched between a loopback data flow, which loops the output data of the first neural network processing core back to the first neural network processing core, and a bypass data flow, which bypasses the output data to the second neural network processing core. Neural network computation methods.

Images