Neural network circuit and neural network computing method

Abstract

This neural network circuit includes a convolution operation circuit that performs a convolution operation on input data and a weight. The input data is a tertiary or higher tensor having elements in an x-axis direction, a y-axis direction, and a c-axis direction. The convolution operation circuit includes: a multiplier that multiplies divided input data obtained by dividing the input data by a predetermined number of elements in the c-axis direction and the corresponding weight; and an accumulator circuit that adds multiplication operation results of the multiplier. The accumulator circuit includes a register that records an addition operation result obtained by cumulatively adding the multiplication operation results from the divided input data contiguous in the c-axis direction, and outputs the addition operation result from the divided input data contiguous in the c-axis direction and recorded in the register.

Description

Neural network circuit and neural network operation method

[0001] This application claims priority to Japanese Patent Application No. 2023-192769, filed November 13, 2023, the contents of which are incorporated herein by reference.

[0002] In recent years, convolutional neural networks (CNNs) have been used as models for image recognition and the like. Convolutional neural networks have a multi-layer structure including convolutional layers and pooling layers, and require a large number of calculations, such as convolutional calculations. Various calculation methods have been devised to speed up calculations by convolutional neural networks (see, for example, Patent Document 1).

[0003] JP 2018-077829 A

[0004] On the other hand, there is a demand for image recognition and the like using convolutional neural networks in embedded devices such as IoT devices. However, it is difficult to incorporate the large-scale dedicated circuits described in Patent Document 1 into embedded devices. Furthermore, in embedded devices with limited hardware resources such as CPUs and memory, it is difficult to achieve sufficient computational performance of convolutional neural networks using software alone.

[0005] In view of the above circumstances, an object of the present invention is to provide a high-performance neural network circuit and a neural network calculation method that can be incorporated into embedded devices such as IoT devices.

[0006] In order to solve the above problems, the present invention proposes the following means: A neural network circuit according to a first aspect of the present invention includes a convolution circuit that performs a convolution operation on input data and weights, the input data being a third- or higher-order tensor having elements in the x-axis, y-axis, and c-axis directions, the convolution circuit including multipliers that multiply the weights by divided input data obtained by dividing the input data by a predetermined number of elements in the c-axis direction, and an accumulator circuit that adds up the multiplication results of the multipliers, the accumulator circuit including a register that records an addition result obtained by accumulating the multiplication results for the divided input data that are continuous in the c-axis direction, and outputs the addition result for the divided input data that are continuous in the c-axis direction that is recorded in the register.

[0007] A neural network calculation method according to a second aspect of the present invention is a calculation method for performing a convolution operation on input data, which is a third- or higher-order tensor having elements in the x-axis, y-axis, and c-axis directions, and a weight, in which the input data is divided into divided input data by a predetermined number of elements in the c-axis direction, and the divided input data is multiplied by the corresponding weight, the results of the multiplication operation on the divided input data that are continuous in the c-axis direction are cumulatively added using a dedicated register, and the results of the addition operation on the divided input data that are continuous in the c-axis direction recorded in the register are output.

[0008] The neural network circuit and neural network operation method of the present invention can be incorporated into embedded devices such as IoT devices and has high performance.

[0009] 1 is a diagram illustrating a convolutional neural network. FIG. 2 is a diagram illustrating the convolution operation performed by a convolution layer. FIG. 3 is a diagram illustrating the expansion of data in a convolution operation. FIG. 4 is a diagram illustrating the overall configuration of a neural network circuit according to a first embodiment. FIG. 5 is a diagram illustrating the overall configuration of an NN operation core. FIG. 6 is a timing chart illustrating an example of the operation of the NN operation core. FIG. 7 is a timing chart illustrating another example of the operation of the NN operation core. FIG. 8 is a diagram illustrating an NN operation multi-core. FIG. 9 is an internal block diagram of a DMAC in the neural network circuit. FIG. 10 is a state transition diagram of a control circuit for the DMAC. FIG. 11 is an internal block diagram of a convolution operation circuit in the neural network circuit. FIG. 12 is an internal block diagram of a multiplier in the convolution operation circuit. FIG. 13 is an internal block diagram of a product-accumulation operation unit of the multiplier. FIG. 14 is an internal block diagram of an accumulator circuit in the convolution operation circuit. FIG. 15 is an internal block diagram of an accumulator unit in the accumulator circuit. FIG. 16 is a flowchart illustrating an example of a convolution operation. FIG. 17 is an internal block diagram of input vectors A[0] to A[n-1]. FIG. 18 is an internal block diagram of a quantization operation circuit in the neural network circuit. FIG. 19 is an internal block diagram of a vector operation circuit and a quantization circuit in the quantization operation circuit. FIG. 19 is a block diagram of an operation unit. FIG. 2 is an internal block diagram of a vector quantization unit of the quantization circuit.

[0010] First Embodiment A first embodiment of the present invention will be described with reference to Fig. 1 to Fig. 21. Fig. 1 is a diagram showing a convolutional neural network 200 (hereinafter referred to as "CNN 200"). The calculations performed by a neural network circuit 100 (hereinafter referred to as "NN circuit 100") according to the first embodiment are at least a part of the trained CNN 200 used during inference.

[0011] [CNN 200] The CNN 200 is a network with a multi-layer structure including a convolution layer 210 that performs a convolution operation, a quantization operation layer 220 that performs a quantization operation, and an output layer 230. In at least a part of the CNN 200, the convolution layer 210 and the quantization operation layer 220 are alternately connected. The CNN 200 is a model widely used for image recognition and video recognition. The CNN 200 may further include layers having other functions, such as a fully connected layer.

[0012] 2 is a diagram illustrating the convolution operation performed by the convolution layer 210. The convolution layer 210 performs a convolution operation on input data a using a weight w. The convolution layer 210 performs a product-sum operation using the input data a and the weight w as input.

[0013] The input data a (also called activation data or feature map) to the convolutional layer 210 is multidimensional data such as image data. In this embodiment, the input data a is a three-dimensional tensor consisting of elements (x, y, c). The convolutional layer 210 of the CNN 200 performs a convolution operation on the low-bit input data a. In this embodiment, the elements of the input data a are 2-bit unsigned integers (0, 1, 2, 3). The elements of the input data a may be, for example, 4-bit or 8-bit unsigned integers.

[0014] If the input data input to the CNN 200 has a different format from the input data a to the convolutional layer 210, such as a 32-bit floating-point format, the CNN 200 may further include an input layer before the convolutional layer 210 that performs type conversion and quantization.

[0015] The weights w (also called filters or kernels) of the convolutional layer 210 are multidimensional data having elements that are learnable parameters. In this embodiment, the weights w are four-dimensional tensors consisting of elements (i, j, c, d). The weights w have d three-dimensional tensors (hereinafter referred to as "weights wo") consisting of elements (i, j, c). The weights w in the trained CNN 200 are trained data. The convolutional layer 210 of the CNN 200 performs convolution operations using low-bit weights w. In this embodiment, the elements of the weights w are 1-bit signed integers (0, 1), where a value "0" represents +1 and a value "1" represents -1.

[0016] The convolution layer 210 performs the convolution operation shown in Equation 1 and outputs output data f. In Equation 1, s represents the stride. The area indicated by the dotted line in FIG. 2 represents one of the areas ao (hereinafter referred to as "application area ao") where weight wo is applied to input data a. The elements of the application area ao are represented by (x+i, y+j, c).

[0017]

[0018] The quantization operation layer 220 performs quantization and the like on the output of the convolution operation output by the convolution layer 210. The quantization operation layer 220 includes a pooling layer 221, a batch normalization layer 222, an activation function layer 223, and a quantization layer 224.

[0019] The pooling layer 221 performs operations such as average pooling (Equation 2) and MAX pooling (Equation 3) on the output data f of the convolution operation output by the convolution layer 210, thereby compressing the output data f of the convolution layer 210. In Equations 2 and 3, u represents the input tensor, v represents the output tensor, and T represents the size of the pooling region. In Equation 3, max is a function that outputs the maximum value of u for the combination of i and j included in T.

[0020]

[0021]

[0022] The batch normalization layer 222 normalizes the data distribution of the output data of the quantization operation layer 220 and the pooling layer 221, for example, by performing an operation such as that shown in Equation 4. In Equation 4, u represents the input tensor, v represents the output tensor, α represents the scale, and β represents the bias. In the trained CNN 200, α and β are trained constant vectors.

[0023]

[0024] The activation function layer 223 performs an activation function calculation such as ReLU (Equation 5) on the outputs of the quantization operation layer 220, the pooling layer 221, and the batch normalization layer 222. In Equation 5, u is an input tensor and v is an output tensor. In Equation 5, max is a function that outputs the largest value among its arguments.

[0025]

[0026] The quantization layer 224 performs quantization on the outputs of the pooling layer 221 and the activation function layer 223 based on the quantization parameter, for example, as shown in Equation 6. The quantization shown in Equation 6 reduces the input tensor u to 2 bits. In Equation 6, q(c) is a vector of quantization parameters. In the trained CNN 200, q(c) is a trained constant vector. The inequality sign "≦" in Equation 6 may be "<".

[0027]

[0028] The output layer 230 is a layer that outputs the results of the CNN 200 using an identity function, a softmax function, etc. The layer preceding the output layer 230 may be the convolution layer 210 or the quantization operation layer 220.

[0029] In the CNN 200, the quantized output data of the quantization layer 224 is input to the convolution layer 210, so the load of the convolution calculation in the convolution layer 210 is smaller than in other convolutional neural networks that do not perform quantization.

[0030] [Dividing the Convolution Operation] The NN circuit 100 divides the input data of the convolution operation (Equation 1) of the convolution layer 210 into partial tensors and performs the operation. The method of division into the partial tensors and the number of divisions are not particularly limited. A partial tensor is formed, for example, by dividing the input data a(x+i, y+j, c) into a(x+i, y+j, co). Note that the NN circuit 100 can also perform the operation without dividing the input data of the convolution operation (Equation 1) of the convolution layer 210.

[0031] In dividing input data for a convolution operation, the variable c in Equation 1 is divided into blocks of size Bc as shown in Equation 7. Furthermore, the variable d in Equation 1 is divided into blocks of size Bd as shown in Equation 8. In Equation 7, co is an offset, and ci is an index from 0 to (Bc-1). In Equation 8, do is an offset, and di is an index from 0 to (Bd-1). Note that the size Bc and the size Bd may be the same.

[0032]

[0033]

[0034] The input data a(x+i, y+j, c) in Equation 1 is divided in the c-axis direction by size Bc and expressed as divided input data a(x+i, y+j, c0). In the following description, the divided input data a will also be referred to as "divided input data a".

[0035] The weight w(i, j, c, d) in Equation 1 is divided by the size Bc in the c-axis direction and the size Bd in the d-axis direction, and is expressed as the divided weight w(i, j, c, d). In the following description, the divided weight w will also be referred to as the "divided weight w."

[0036] The output data f(x, y, do) divided by the size Bd is calculated by Equation 9. The divided output data f(x, y, do) are combined to calculate the final output data f(x, y, d).

[0037]

[0038] [Expansion of Data for Convolutional Operation] The NN circuit 100 expands the input data a and weight w in the convolutional operation of the convolutional layer 210 to perform the convolutional operation.

[0039] 3 is a diagram explaining the expansion of data in a convolution operation. Divided input data a(x+i, y+j, co) is expanded into vector data having Bc elements. The elements of divided input data a are indexed by ci (0≦ci<Bc). In the following explanation, divided input data a expanded into vector data for each i and j is also referred to as "input vector A." Input vector A has elements from divided input data a(x+i, y+j, co×Bc) to divided input data a(x+i, y+j, co×Bc+(Bc-1)).

[0040] The division weights w(i, j, co, do) are expanded into matrix data having Bc x Bd elements. The elements of the division weights w expanded into the matrix data are indexed by ci and di (0≦di<Bd). In the following explanation, the division weights w expanded into matrix data for each i and j will also be referred to as the "weight matrix W." The weight matrix W has elements ranging from division weights w(i, j, co x Bc, do x Bd) to division weights w(i, j, co x Bc + (Bc-1), do x Bd + (Bd-1)).

[0041] Vector data is calculated by multiplying the input vector A by the weight matrix W. The vector data calculated for each i, j, co is shaped into a three-dimensional tensor to obtain output data f(x, y, do). By expanding the data in this way, the convolution operation of the convolution layer 210 can be performed by multiplying the vector data by the matrix data.

[0042] 4 is a diagram showing the overall configuration of the NN circuit 100 according to this embodiment. The NN circuit 100 includes a DMA controller 3 (hereinafter also referred to as "DMAC3"), a controller 6, an IFU 7, and at least one neural network operation core 10 (hereinafter also referred to as "NN operation core 10").

[0043] The NN circuit 100 can implement multiple NN computation cores 10. The NN circuit 100 illustrated in FIG. 4 can implement up to four NN computation cores 10. The multiple NN computation cores 10 form a "neural network computation multi-core 10M (hereinafter also referred to as "NN computation multi-core 10M")" that cooperates to execute at least some of the computations of the NN 200. In this embodiment, the multiple NN computation cores 10 are daisy-chained. Note that the number of NN computation cores 10 that can be implemented in the NN circuit 100 may be five or more.

[0044] The DMAC3 is connected to the external bus EB, and transfers data between an external memory 120 such as a DRAM and the NN calculation core 10. The DMAC3 transfers data read from the external memory 120 to any one of the multiple NN calculation cores 10. The DMAC3 may be capable of transferring the same data read from the external memory 120 to multiple NN calculation cores 10, or may be capable of broadcasting the data.

[0045] The controller 6 is connected to an external bus EB and operates as a slave to the external host CPU 110. The controller 6 includes a bus bridge 60 and a register 61.

[0046] The bus bridge 60 relays bus access from the external bus EB to the internal bus IB, and also relays write and read requests from the external host CPU 110 to the register 61.

[0047] The register 61 has a parameter register and a status register. The parameter register is a register that controls the operation of the NN circuit 100. The status register contains pointers to the instruction strings of each module, the number of instructions, etc., and indicates the status of the NN circuit 100. The status register may also be configured to include a semaphore S. The external host CPU 110 can access the register 61 via the bus bridge 60 of the controller 6.

[0048] The controller 6 is connected to each block of the NN circuit 100 (DMAC 3, IFU 7, NN calculation core 10) via the internal bus IB. The external host CPU 110 can access each block of the NN circuit 100 via the controller 6. For example, the external host CPU 110 can issue commands to the NN calculation core 10 via the controller 6. Furthermore, each block can update a status register (which may include a semaphore S) held by the controller 6 via the internal bus IB. The status register may be configured to be updated via dedicated wiring connected to each block.

[0049] The IFU (Instruction Fetch Unit) 7 reads instruction commands for each block (DMAC3, NN calculation core 10) of the NN circuit 100 from the external memory 120 via the external bus EB based on instructions from the external host CPU 110. The IFU 7 also transfers the read instruction commands to the corresponding blocks (DMAC3, NN calculation core 10) of the NN circuit 100. In this embodiment, the instruction commands are stored in a compressed state (hereinafter also referred to as "compressed instruction commands") in the external memory 120. The IFU 7 reads the compressed instruction commands.

[0050] [NN operation core 10] Fig. 5 is a diagram showing the overall configuration of the NN operation core 10. The NN operation core 10 comprises a first memory 1, a second memory 2, a convolution operation circuit 4, and a quantization operation circuit 5. The NN operation core 10 is characterized in that the convolution operation circuit 4 and the quantization operation circuit 5 are formed in a loop via the first memory 1 and the second memory 2.

[0051] The first memory 1 is a rewritable memory such as a volatile memory configured, for example, by SRAM (Static RAM). Data is written to and read from the first memory 1 via the DMAC 3 and the internal bus IB. The external host CPU 110 can input and output data to and from the NN calculation core 10 by writing and reading data to and from the first memory 1.

[0052] The first memory 1 is connected to an input port of the convolution operation circuit 4, and the convolution operation circuit 4 can read data from the first memory 1. The first memory 1 is also loop-connected (C1) to an output port of the quantization operation circuit 5, and the quantization operation circuit 5 can write data to the first memory 1. The first memory 1 is also capable of data transfer with other NN operation cores 10 via inter-core connections (C2), and other NN operation cores 10 connected via inter-core connections (C2) can write data to the first memory 1. In this embodiment, a daisy-chain connection is used as an example of the inter-core connection (C2).

[0053] The second memory 2 is a rewritable memory such as a volatile memory configured, for example, by SRAM (Static RAM). Data is written to and read from the second memory 2 via the DMAC 3 and the internal bus IB. The external host CPU 110 can input and output data to and from the NN calculation core 10 by writing and reading data to and from the second memory 2.

[0054] The second memory 2 is connected to an input port of the quantization operation circuit 5, and the quantization operation circuit 5 can read data from the second memory 2. In addition, the second memory 2 is connected to an output port of the convolution operation circuit 4, and the convolution operation circuit 4 can write data to the second memory 2.

[0055] The convolution operation circuit 4 is a circuit that performs convolution operations in the convolution layer 210 of the trained CNN 200. The convolution operation circuit 4 reads the input data a stored in the first memory 1 and performs a convolution operation on the input data a. The convolution operation circuit 4 writes output data f of the convolution operation (hereinafter also referred to as "convolution operation output data") to the second memory 2.

[0056] The quantization operation circuit 5 is a circuit that performs at least a part of the quantization operation in the quantization operation layer 220 of the trained CNN 200. The quantization operation circuit 5 reads the output data f of the convolution operation stored in the second memory 2, and performs a quantization operation (an operation including at least quantization among pooling, batch normalization, activation function, and quantization) on the output data f of the convolution operation.

[0057] The quantization operation circuit 5 writes the output data of the quantization operation (hereinafter also referred to as "quantization operation output data") to the loop-connected (C1) first memory 1. The quantization operation circuit 5 is also capable of transferring data to other NN operation cores 10 via the inter-core connection (C2), and the quantization operation circuit 5 can output the quantization operation output data to other NN operation cores 10 connected via the inter-core connection (C2).

[0058] The NN processing core 10 has the first memory 1, the second memory 2, etc., and therefore can reduce the number of times duplicate data is transferred when data is transferred from an external memory such as a DRAM by the DMAC 3. This makes it possible to significantly reduce the power consumption or processing load caused by memory access.

[0059] 6 is a timing chart showing an example of the operation of the NN calculation core 10. The DMAC 3 stores input data a of layer 1 in the first memory 1. The DMAC 3 may divide the input data a of layer 1 and transfer it to the first memory 1 in accordance with the order of the convolution calculation performed by the convolution calculation circuit 4.

[0060] The convolution circuit 4 reads the input data a of layer 1 stored in the first memory 1. The convolution circuit 4 performs the convolution operation of layer 1 shown in FIG. 1 on the input data a of layer 1. The output data f of the convolution operation of layer 1 is stored in the second memory 2.

[0061] The quantization operation circuit 5 reads the output data f of layer 1 stored in the second memory 2. The quantization operation circuit 5 performs a quantization operation for layer 2 on the output data f of layer 1. The output data of the quantization operation for layer 2 is stored in the first memory 1.

[0062] The convolution operation circuit 4 reads the output data of the quantization operation of layer 2 stored in the first memory 1. The convolution operation circuit 4 performs the convolution operation of layer 3 using the output data of the quantization operation of layer 2 as input data a. The output data f of the convolution operation of layer 3 is stored in the second memory 2.

[0063] The convolution operation circuit 4 reads the output data of the quantization operation of layer 2M-2 (M is a natural number) stored in the first memory 1. The convolution operation circuit 4 performs a convolution operation of layer 2M-1 using the output data of the quantization operation of layer 2M-2 as input data a. The output data f of the convolution operation of layer 2M-1 is stored in the second memory 2.

[0064] The quantization operation circuit 5 reads the output data f of layer 2M-1 stored in the second memory 2. The quantization operation circuit 5 performs a quantization operation for layer 2M on the output data f of the 2M-1 layer. The output data of the quantization operation for layer 2M is stored in the first memory 1.

[0065] The convolution operation circuit 4 reads the output data of the quantization operation of layer 2M stored in the first memory 1. The convolution operation circuit 4 performs a convolution operation of layer 2M+1 using the output data of the quantization operation of layer 2M as input data a. The output data f of the convolution operation of layer 2M+1 is stored in the second memory 2.

[0066] The convolution operation circuit 4 and the quantization operation circuit 5 alternately perform operations to advance the operation of the CNN 200 shown in FIG. 1. In the NN operation core 10, the convolution operation circuit 4 performs the convolution operation of layer 2M-1 and layer 2M+1 by time division. In the NN operation core 10, the quantization operation circuit 5 performs the quantization operation of layer 2M-2 and layer 2M by time division. Therefore, the circuit scale of the NN operation core 10 is significantly smaller than when a separate convolution operation circuit 4 and quantization operation circuit 5 are implemented for each layer.

[0067] The NN processing core 10 performs the calculations of the CNN 200, which has a multi-layer structure with multiple layers, using a circuit formed in a loop. The NN processing core 10 can efficiently utilize hardware resources due to the loop circuit configuration. Note that, because the NN processing core 10 forms a circuit in a loop, parameters in the convolution processing circuit 4 and the quantization processing circuit 5, which change in each layer, are updated appropriately.

[0068] If the calculations of the CNN 200 include calculations that cannot be performed by the NN calculation core 10, the NN calculation core 10 transfers intermediate data to an external calculation device such as an external host CPU 110. After the external calculation device performs calculations on the intermediate data, the calculation results by the external calculation device are input to the first memory 1 and the second memory 2. The NN calculation core 10 resumes calculations on the calculation results by the external calculation device.

[0069] 7 is a timing chart showing another example of the operation of the NN processing core 10. The NN processing core 10 may divide the input data a into partial tensors and perform operations on the partial tensors by time division. The method of division into partial tensors and the number of divisions are not particularly limited.

[0070] 7 shows an example of operation when input data a is decomposed into two partial tensors. The decomposed partial tensors are referred to as a "first partial tensor a1" and a "second partial tensor a2." For example, the convolution operation of layer 2M-1 is decomposed into a convolution operation corresponding to the first partial tensor a1 (denoted as "layer 2M-1(a1)" in FIG. 7) and a convolution operation corresponding to the second partial tensor a2 (denoted as "layer 2M-1(a2)" in FIG. 7).

[0071] The convolution and quantization operations corresponding to the first partial tensor a1 and the convolution and quantization operations corresponding to the second partial tensor a2 can be performed independently, as shown in FIG.

[0072] The convolution circuit 4 performs a convolution operation on the layer 2M-1 corresponding to the first partial tensor a1 (the operation shown as layer 2M-1(a1) in FIG. 7). Then, the convolution circuit 4 performs a convolution operation on the layer 2M-1 corresponding to the first partial tensor a1. ２ The convolution operation of layer 2M-1 corresponding to (in FIG. 7, layer 2M-1 (a ２ )) for the layer 2M corresponding to the first partial tensor a1. In addition, the quantization calculation circuit 5 performs the quantization calculation for the layer 2M corresponding to the first partial tensor a1 (the calculation shown as layer 2M(a1) in FIG. 7). In this way, the NN calculation core 10 performs the quantization calculation for the layer 2M corresponding to the first partial tensor a1. ２ The convolution operation of layer 2M−1 corresponding to the first partial tensor a1 and the quantization operation of layer 2M corresponding to the first partial tensor a1 can be performed in parallel.

[0073] Next, the convolution operation circuit 4 performs a convolution operation of the layer 2M+1 corresponding to the first partial tensor a1 (the operation shown as layer 2M+1(a1) in FIG. 7). ２quantization operation of layer 2M corresponding to (in FIG. 7, layer 2M (a ２ In this way, the NN processing core 10 performs the convolution operation of the layer 2M+1 corresponding to the first partial tensor a1 and the operation of the second partial tensor a ２ The layer 2M quantization operations corresponding to can be performed in parallel.

[0074] The convolution operation and quantization operation corresponding to the first partial tensor a1 and the convolution operation and quantization operation corresponding to the second partial tensor a2 can be performed independently. Therefore, the NN processing core 10 can perform, for example, the convolution operation of layer 2M-1 corresponding to the first partial tensor a1 and the convolution operation and quantization operation corresponding to the second partial tensor a2 independently. ２ In other words, the convolution and quantization operations performed in parallel by the NN processing core 10 are not limited to operations on consecutive layers.

[0075] By dividing the input data a into partial tensors, the NN operation core 10 can operate the convolution operation circuit 4 and the quantization operation circuit 5 in parallel. As a result, the standby time of the convolution operation circuit 4 and the quantization operation circuit 5 is reduced, improving the operation processing efficiency of the NN operation core 10. In the operation example shown in FIG. 7 , the number of divisions is 2, but even if the number of divisions is greater than 2, the NN operation core 10 can similarly operate the convolution operation circuit 4 and the quantization operation circuit 5 in parallel.

[0076] For example, if the input data a is divided into a "first partial tensor a1," a "second partial tensor a2," and a "third partial tensor a ３ ", the NN processing core 10 divides the second partial tensor a ２ The convolution operation of layer 2M-1 corresponding to the third partial tensor a ３ The order of the operations may be changed as appropriate depending on the storage status of the input data a in the first memory 1 and the second memory 2.

[0077] As a method of computing partial tensors, an example (Method 1) has been shown in which the partial tensors in the same layer are computed by the convolution computation circuit 4 or the quantization computation circuit 5, and then the partial tensors in the next layer are computed. For example, as shown in FIG. 7, in the convolution computation circuit 4, the first partial tensor a1 and the second partial tensor a2 are computed by the convolution computation circuit 4. ２ The convolution operation of layer 2M-1 corresponding to (in FIG. 7, layer 2M-1(a1) and layer 2M-1(a ２ )), the first partial tensor a and the second partial tensor a ２ The convolution operation of layer 2M+1 corresponding to (in FIG. 7, layer 2M+1(a1) and layer 2M+1(a ２ ) is performed.

[0078] However, the method of operation on the partial tensors is not limited to this. The method of operation on the partial tensors may be a method of performing an operation on some of the partial tensors in multiple layers and then performing an operation on the remaining partial tensors (Method 2). For example, in the convolution operation circuit 4, after performing a convolution operation on the layer 2M−1 corresponding to the first partial tensor a1 and the layer 2M+1 corresponding to the first partial tensor a1, the second partial tensor a ２ Layer 2M-1 and the second partial tensor a ２ , a layer 2M+1 convolution operation corresponding to:

[0079] Furthermore, the method of computing partial tensors may be a method of computing partial tensors by combining Method 1 and Method 2. However, when Method 2 is used, the computation must be performed in accordance with the dependency regarding the computation order of the partial tensors.

[0080] [NN computation multi-core 10M] Fig. 8 is a diagram showing an NN computation multi-core 10M. The NN computation multi-core 10M shown in Fig. 8 includes two daisy-chain connected NN computation cores 10. When distinguishing between the two NN computation cores 10, the two NN computation cores 10 are referred to as a "first NN computation core 10A" and a "second NN computation core 10B." In Fig. 8, the first memory 1 is abbreviated as "A," the convolution computation circuit 4 as "C," the second memory 2 as "F," and the quantization computation circuit 5 as "Q."

[0081] Specifically, the quantization calculation circuit 5 of the first NN calculation core 10A and the first memory 1 of the second NN calculation core 10B are daisy-chain connected (C2). The quantization calculation circuit 5 of the first NN calculation core 10A can write quantization calculation output data to the first memory 1 of the first NN calculation core 10A connected in a loop (C1) and / or the first memory 1 of the second NN calculation core 10B connected in a daisy-chain (C2).

[0082] Specifically, the quantization calculation circuit 5 of the second NN calculation core 10B and the first memory 1 of the first NN calculation core 10A are daisy-chain connected (C2). The quantization calculation circuit 5 of the second NN calculation core 10B can write quantization calculation output data to the first memory 1 of the second NN calculation core 10B, which is loop-connected (C1), and / or the first memory 1 of the first NN calculation core 10A, which is daisy-chain connected (C2).

[0083] Similarly, when the NN computation multi-core 10M has three or more NN computation cores 10, the multiple NN computation cores 10 are daisy-chained. The quantization computation circuits 5 of the NN computation cores 10 other than the final-stage NN computation core 10 are daisy-chained (C2) with the first memory 1 of the subsequent-stage NN computation core 10B. The quantization computation circuit 5 of the final-stage NN computation core 10 is daisy-chained (C2) with the first memory 1 of the initial-stage NN computation core 10. The multiple NN computation cores 10 are characterized by being formed in a daisy-chain loop (linked together).

[0084] In one NN calculation core 10, the first memory (A) 1, the convolution calculation circuit (C) 4, the second memory (F) 2, and the quantization calculation circuit (Q) 5 are connected in a loop. On the other hand, in the NN calculation multi-core 10M, the first memory (A) 1, the convolution calculation circuit (C) 4, the second memory (F) 2, and the quantization calculation circuit (Q) 5 are connected in a daisy chain loop (linked together) so that the first memory (A) 1, the convolution calculation circuit (C) 4, the second memory (F) 2, and the quantization calculation circuit (Q) 5 are repeatedly arranged in the same order.

[0085] The multiple NN computation cores 10 constituting the NN computation multi-core 10M do not need to have the same hardware configuration. For example, the capacity and configuration of the first memory 1 of the first NN computation core 10A may be different from the capacity and configuration of the first memory 1 of the second NN computation core 10B. For example, the configuration of the quantization computation circuit 5 of the first NN computation core 10A may be different from the configuration of the quantization computation circuit 5 of the second NN computation core 10B.

[0086] Next, each component of the NN circuit 100 will be described in detail.

[0087] 9 is an internal block diagram of the DMAC 3. The DMAC 3 has a data transfer circuit 31 and a state controller 32. The DMAC 3 has a state controller 32 dedicated to the data transfer circuit 31, and when an instruction command is input, the DMAC 3 can perform DMA data transfer without requiring an external controller.

[0088] The data transfer circuit 31 is connected to the external bus EB, and performs DMA data transfer between an external memory 120 such as a DRAM and the NN calculation core 10. There is no limitation on the number of DMA channels of the data transfer circuit 31. For example, the first NN calculation core 10A and the second NN calculation core 10B may each have a dedicated DMA channel.

[0089] The state controller 32 controls the state of the data transfer circuit 31. The state controller 32 is also connected to the controller 6 via an internal bus IB. The state controller 32 has an instruction queue 33 and a control circuit .

[0090] The instruction queue 33 is a queue that stores instruction commands C3 for the DMAC 3, and is configured by, for example, a FIFO memory. One or more instruction commands C3 are written to the instruction queue 33 via the IFU 7 or the internal bus IB.

[0091] The control circuit 34 is a state machine that decodes the instruction command C3 and sequentially controls the data transfer circuit 31 based on the instruction command C3. The control circuit 34 may be implemented by a logic circuit or a CPU controlled by software.

[0092] 10 is a state transition diagram of the control circuit 34. When an instruction command C3 is input to the instruction queue 33 (not empty), the control circuit 34 transitions from the idle state ST1 to the decode state ST2.

[0093] In the decode state ST2, the control circuit 34 decodes the instruction command C3 output from the instruction queue 33. The control circuit 34 also reads the semaphore S stored in the register 61 of the controller 6 and determines whether the operation of the data transfer circuit 31 instructed in the instruction command C3 is executable. If the operation is not executable (Not ready), the control circuit 34 waits until the operation is executable (Wait). If the operation is executable (Ready), the control circuit 34 transitions from the decode state ST2 to the execution state ST3.

[0094] In the execution state ST3, the control circuit 34 controls the data transfer circuit 31 to cause the data transfer circuit 31 to perform the operation instructed in the instruction command C3. When the operation of the data transfer circuit 31 is completed, the control circuit 34 removes the executed instruction command C3 from the instruction queue 33 and updates the semaphore S stored in the register 61 of the controller 6. If there are instructions in the instruction queue 33 (Not empty), the control circuit 34 transitions from the execution state ST3 to the decode state ST2. If there are no instructions in the instruction queue 33 (Empty), the control circuit 34 transitions from the execution state ST3 to the idle state ST1.

[0095] 11 is an internal block diagram of the convolution circuit 4. The convolution circuit 4 has a weight memory 41, a multiplier 42, an accumulator circuit 43, and a state controller 44. The convolution circuit 4 has a dedicated state controller 44 for the multiplier 42 and the accumulator circuit 43, and when an instruction command is input, the convolution circuit 4 can perform a convolution operation without requiring an external controller.

[0096] The weight memory 41 is a memory that stores the weight w used in the convolution calculation, and is a rewritable memory such as a volatile memory configured, for example, by SRAM (Static RAM), etc. The DMAC 3 writes the weight w required for the convolution calculation into the weight memory 41 by DMA transfer.

[0097] 12 is an internal block diagram of the multiplier 42. The multiplier 42 multiplies the input vector A by the weight matrix W. As described above, the input vector A is vector data having Bc elements in which the divided input data a(x+i, y+j, co) is expanded for each i and j. The weight matrix W is matrix data having Bc×Bd elements in which the divided weights w(i, j, co, do) are expanded for each i and j. The multiplier 42 has Bc×Bd product-sum operation units 47 and can multiply the input vector A by the weight matrix W in parallel.

[0098] The multiplier 42 performs multiplication by reading the input vector A and weight matrix W required for multiplication from the first memory 1 and the weight memory 41. The multiplier 42 outputs Bd product-sum operation results O(di).

[0099] 13 is an internal block diagram of the product-sum operation unit 47. The product-sum operation unit 47 multiplies an element A(ci) of an input vector A by an element W(ci, di) of a weight matrix W. The product-sum operation unit 47 also adds the multiplication result to the multiplication result S(ci, di) of another product-sum operation unit 47. The product-sum operation unit 47 outputs the addition result S(ci+1, di). The element A(ci) is a 2-bit unsigned integer (0, 1, 2, 3). The element W(ci, di) is a 1-bit signed integer (0, 1), where a value "0" represents +1 and a value "1" represents -1.

[0100] The sum-of-products operation unit 47 has an inverter 47a, a selector 47b, and an adder 47c. The sum-of-products operation unit 47 performs multiplication using only the inverter 47a and the selector 47b, without using a multiplier. When the element W(ci, di) is "0", the selector 47b selects the input of the element A(ci). When the element W(ci, di) is "1", the selector 47b selects the complement of the element A(ci) inverted by the inverter. The element W(ci, di) is also input to the carry-in of the adder 47c. When the element W(ci, di) is "0", the adder 47c outputs the value obtained by adding the element A(ci) to S(ci, di). When W(ci, di) is "1", the adder 47c outputs the value obtained by subtracting the element A(ci) from S(ci, di).

[0101] 14 is an internal block diagram of the accumulator circuit 43. The accumulator circuit 43 accumulates the sum-of-products operation results O(di) of the multiplier 42 in the second memory 2. The accumulator circuit 43 has Bd accumulator units 48 and can accumulate Bd sum-of-products operation results O(di) in parallel in the second memory 2.

[0102] 15 is an internal block diagram of the accumulator unit 48. The accumulator unit 48 has an adder 48a, a mask unit 48b, a register 48c, and a selector 48d. The adder 48a adds an element O(di) of the sum-of-products operation result O to the partial sum, which is an intermediate result of the convolution operation shown in Equation 1 and stored in the register 48c or the second memory 2. The addition result by the accumulator unit 48 is 16 bits per element. The addition result is not limited to 16 bits per element, and may be, for example, 15 bits or 17 bits per element.

[0103] The adder 48a writes the addition result to the register 48c or the second memory 2. When the initialization signal clear is asserted, the masking unit 48b masks the output from the second memory 2 and sets the value to be added to the element O(di) to zero.

[0104] The register 48c is a memory that temporarily stores intermediate partial sums. The register 48c records the result of an addition operation obtained by accumulating the multiplication operation results of successive multipliers 42 in the c-axis direction. The register 48c is a small-scale recording medium formed by a register file or the like.

[0105] The selector 48d is a selector that selects the data to be input to the adder 48a from the data temporarily stored in the register 48c and the data read from the second memory 2. When an intermediate partial sum is stored in the register 48c, the selector 48d selects and outputs the data temporarily stored in the register 48c. When an intermediate partial sum is not stored in the register 48c but is stored in the second memory 2, the selector 48d selects and outputs the data read from the second memory 2.

[0106] The state controller 44 controls the states of the multiplier 42 and the accumulator circuit 43. The state controller 44 is also connected to the controller 6 via an internal bus IB. The state controller 44 has an instruction queue 45 and a control circuit 46.

[0107] The instruction queue 45 is a queue that stores instruction commands C4 for the convolution operation circuit 4, and is configured by, for example, a FIFO memory. The instruction commands C4 are written to the instruction queue 45 via the IFU 7 or the internal bus IB.

[0108] The control circuit 46 is a state machine that decodes the instruction command C4 and controls the multiplier 42 and the accumulator circuit 43 based on the instruction command C4. The control circuit 46 has the same configuration as the control circuit 34 of the state controller 32 of the DMAC3.

[0109] 16 is a flowchart showing an example of a convolution operation performed by the control circuit 46 and the like. In step S110, the DMAC 3 stores input vectors A[0] to A[n-1], which are divided input data divided in the c-axis direction, in the first memory 1. The variable i (0≦i<n) and the register 48c are initialized to zero.

[0110] FIG. 17 is a diagram illustrating input vectors A[0] to A[n-1]. Input vector A[i] is vector data having Bc elements in the c-axis direction. For example, the x-axis and y-axis directions correspond to the screen direction in the image, and the c-axis direction corresponds to the channel direction. From the perspective of computational efficiency, it is desirable that the number n of input vectors A[0] to A[n-1] be C / Bc. However, input vectors A[0] to A[n-1] must be storable in first memory 1. In other words, the number n of divided input data to be stored in first memory 1 is selected within the range that can be stored in first memory 1.

[0111] For example, if the number of elements C in the c-axis direction of input data a is 1024 and Bc is 256, the number of input vectors A[0] to A[n-1] will be 4. In other words, it is desirable that the number of elements C in the c-axis direction of input data a held in first memory 1 be greater than the number of elements in the c-axis direction of input vector A[i] that can be input to multiplier 42.

[0112] The weight matrix W used in the convolution operation with A[0] to A[n-1] contains different elements in the c-axis direction. Therefore, it is desirable that the weight matrix W is also divided to correspond to the input vector A[i]. The product-sum operation unit 47 reads the weight matrix W[i] corresponding to the input vector A[i] from the weight memory 41 and performs the product-sum operation.

[0113] In step S120, the multiplier 42 receives the input vector A[i] and outputs Bd product-sum operation results O(di).

[0114] In step S130, the accumulator circuit 43 adds the result of the sum-of-products operation O(di) to the partial sum of the progress temporarily stored in the register 48c.

[0115] In step S140, the accumulator circuit 43 checks the value of the variable i. If the variable i is not n-1, the accumulator circuit 43 increments the variable i and then executes step S150. If the variable i is n-1, the accumulator circuit 43 then executes step S160.

[0116] In step S150, the accumulator circuit 43 overwrites the partial sum in progress in the register 48c with the addition result, and then executes step S120 again.

[0117] In step S160, the accumulator circuit 43 writes the addition result into the second memory 2.

[0118] When the convolution operation by the multiplier 42 and the accumulator circuit 43 is completed, the output data f(x, y, do) is stored in the second memory 2. By sufficiently increasing the number of elements C in the c-axis direction of the input data a stored in the first memory 1, the output data f(x, y, do) can be calculated without reading intermediate partial sums from the second memory 2, thereby reducing the frequency of access to the second memory 2. The second memory 2 outputs the output data f to the quantization operation circuit 5 while receiving multi-bit data, such as 16 bits, from the accumulator circuit 43. In other words, the second memory 2 inputs and outputs multi-bit data more frequently than the first memory 1. Therefore, by reducing the frequency of access to the second memory 2 as described above, the power consumption and circuit size of the second memory 2 and peripheral circuits can be suitably reduced.

[0119] 18 is an internal block diagram of the quantization operation circuit 5. The quantization operation circuit 5 has a quantization parameter memory 51, a vector operation circuit 52, a quantization circuit 53, and a state controller 54. The quantization operation circuit 5 has a state controller 54 dedicated to the vector operation circuit 52 and the quantization circuit 53, and when an instruction command is input, the quantization operation can be performed without the need for an external controller.

[0120] The quantization parameter memory 51 is a memory that stores the quantization parameter q used in the quantization operation, and is a rewritable memory such as a volatile memory configured, for example, by SRAM (Static RAM), etc. The DMAC 3 writes the quantization parameter q required for the quantization operation into the quantization parameter memory 51 by DMA transfer.

[0121] 19 is an internal block diagram of the vector operation circuit 52 and the quantization circuit 53. The vector operation circuit 52 performs an operation on the output data f(x, y, do) stored in the second memory 2. The vector operation circuit 52 has Bd operation units 57, and performs SIMD operations in parallel on the output data f(x, y, do).

[0122] 20 is a block diagram of the arithmetic unit 57. The arithmetic unit 57 includes, for example, an ALU 57 a, a first selector 57 b, a second selector 57 c, a register 57 d, and a shifter 57 e. The arithmetic unit 57 may further include other arithmetic units included in a known general-purpose SIMD arithmetic circuit.

[0123] The vector operation circuit 52 performs at least one operation on the output data f(x, y, d o) among the operations of the pooling layer 221, the batch normalization layer 222, and the activation function layer 223 in the quantization operation layer 220 by combining the operation units and the like included in the operation unit 57.

[0124] The arithmetic unit 57 can add the data stored in the register 57d and the element f(di) of the output data f(x, y, do) read from the second memory 2 using the ALU 57a. The arithmetic unit 57 can store the addition result by the ALU 57a in the register 57d. The arithmetic unit 57 can initialize the addition result by inputting "0" to the ALU 57a instead of the data stored in the register 57d through the selection of the first selector 57b. For example, if the pooling area is 2x2, the shifter 57e can output the average value of the addition result by shifting the output of the ALU 57a by 2 bits to the right. The vector calculation circuit 52 can perform the average pooling calculation shown in Equation 2 by repeating the above calculations by the Bd arithmetic units 57.

[0125] The arithmetic unit 57 can compare the data stored in the register 57d with the element f(di) of the output data f(x, y, do) read from the second memory 2 using the ALU 57a. The arithmetic unit 57 controls the second selector 57c according to the comparison result by the ALU 57a to select the larger of the data stored in the register 57d and the element f(di). The arithmetic unit 57 can initialize the comparison target to the minimum value by inputting the minimum value of the element f(di) to the ALU 57a using the selection of the first selector 57b. In this embodiment, the element f(di) is a 16-bit signed integer, so the minimum value of the element f(di) is "0x8000." The vector arithmetic circuit 52 can perform the MAX pooling calculation of Equation 3 by repeating the above calculations using the Bd arithmetic units 57. Note that the shifter 57e does not shift the output of the second selector 57c during the MAX pooling calculation.

[0126] The arithmetic unit 57 can subtract, by the ALU 57a, the data stored in the register 57d from the element f(di) of the output data f(x, y, do) read from the second memory 2. The shifter 57e can shift the output of the ALU 57a to the left (i.e., multiplication) or right (i.e., division). The vector arithmetic circuit 52 can perform the Batch Normalization operation of Equation 4 by repeating the above-mentioned operations by the Bd arithmetic units 57.

[0127] The arithmetic unit 57 can compare the element f(di) of the output data f(x, y, do) read from the second memory 2 with "0" selected by the first selector 57b using the ALU 57a. The arithmetic unit 57 can select and output either the element f(di) or a constant value "0" stored in advance in the register 57d depending on the comparison result by the ALU 57a. The vector arithmetic circuit 52 can perform the ReLU operation of Equation 5 by repeating the above-mentioned operation by the Bd arithmetic units 57.

[0128] The vector operation circuit 52 can perform average pooling, MAX pooling, batch normalization, activation function operations, and combinations of these operations. Because the vector operation circuit 52 can perform general-purpose SIMD operations, it may also perform other operations necessary for the operations in the quantization operation layer 220. Furthermore, the vector operation circuit 52 may also perform operations other than those in the quantization operation layer 220.

[0129] The quantization calculation circuit 5 does not necessarily have to have the vector calculation circuit 52. When the quantization calculation circuit 5 does not have the vector calculation circuit 52, the output data f(x, y, do) is input to the quantization circuit 53.

[0130] The quantization circuit 53 quantizes the output data of the vector operation circuit 52. As shown in Fig. 19, the quantization circuit 53 has Bd quantization units 58, and performs calculations on the output data of the vector operation circuit 52 in parallel.

[0131] FIG. 21 is an internal block diagram of the quantization unit 58. The quantization unit 58 quantizes the elements in(di) of the output data of the vector operation circuit 52. The quantization unit 58 has a comparator 58a and an encoder 58b. The quantization unit 58 performs the operation (Equation 6) of the quantization layer 224 in the quantization operation layer 220 on the output data (16 bits / element) of the vector operation circuit 52. The quantization unit 58 reads the necessary quantization parameters q (th0, th1, th2) from the quantization parameter memory 51 and compares the input in(di) with the quantization parameters q using the comparator 58a. The quantization unit 58 quantizes the comparison result from the comparator 58a to 2 bits / element using the encoder 58b. Since α(c) and β(c) in Equation 4 are parameters that differ for each variable c, the quantization parameter q(th0, th1, th2) that reflects α(c) and β(c) is a parameter that differs for each in(di).

[0132] The quantization unit 58 classifies the input in(di) into four regions (e.g., in≦th0, th0<in≦th1, th1<in≦th2, th2<in) by comparing the input in(di) with three thresholds th0, th1, and th2, and encodes and outputs the classification result into two bits. The quantization unit 58 can also perform batch normalization and activation function calculations in addition to quantization by setting a quantization parameter q(th0, th1, th2).

[0133] The quantization unit 58 sets the threshold th0 as β(c) in Equation 4, and the threshold differences (th1-th0) and (th2-th1) as α(c) in Equation 4, and thereby performs the Batch Normalization operation shown in Equation 4 together with the quantization. α(c) can be reduced by increasing (th1-th0) and (th2-th1). α(c) can be increased by reducing (th1-th0) and (th2-th1).

[0134] The quantization unit 58 may perform the activation function in conjunction with the quantization of the input in(d). For example, the quantization unit 58 saturates the output values ​​in the region where in(d)≦th0 and th2<in(d). The quantization unit 58 may perform the operation of the activation function in conjunction with the quantization by setting the quantization parameter q so that the output is nonlinear.

[0135] The state controller 54 controls the states of the vector operation circuit 52 and the quantization circuit 53. The state controller 54 is also connected to the controller 6 via an internal bus IB. The state controller 54 has an instruction queue 55 and a control circuit 56.

[0136] The instruction queue 55 is a queue that stores instruction commands C5 for the quantization calculation circuit 5, and is configured by, for example, a FIFO memory. The instruction commands C5 are written to the instruction queue 55 via the IFU 7 or the internal bus IB.

[0137] The control circuit 56 is a state machine that decodes the instruction command C5 and controls the vector operation circuit 52 and the quantization circuit 53 based on the instruction command C5. The control circuit 56 has the same configuration as the control circuit 34 of the state controller 32 of the DMAC 3.

[0138] The quantization operation circuit 5 writes quantization operation output data having Bd elements into the first memory 1. A preferable relationship between Bd and Bc is shown in Equation 10, where n is an integer.

[0139]

[0140] [Controller 6] The controller 6 transfers instructions and commands transferred from the external host CPU 110 via the internal bus IB to instruction queues included in the DMAC 3, the convolution operation circuit 4, and the quantization operation circuit 5. The controller 6 may have an instruction memory that stores instructions and commands for each circuit.

[0141] The controller 6 is connected to an external bus EB and operates as a slave to the external host CPU 110. The controller 6 has registers 61 including a parameter register and a status register. The parameter register is a register that controls the operation of the NN circuit 100. The status register is a register that indicates the status of the NN circuit 100, including the semaphore S.

[0142] The neural network circuit 100 according to this embodiment can be incorporated into embedded devices such as IoT devices, and can operate with high performance. By connecting multiple NN processing cores 10, it is possible to efficiently and quickly perform more neural network operations.

[0143] Although the first embodiment of the present invention has been described above in detail with reference to the drawings, the specific configuration is not limited to this embodiment and includes design modifications within the scope of the present invention. Furthermore, the components shown in the above embodiment and modifications can be configured by appropriately combining them.

[0144] (Variation 1) In the above embodiment, the input vector A[i] is 1x1 vector data in the xy plane. However, the input vector A[i] is not limited to this. The input vector A[i] may be 3x3 vector data in the xy plane.

[0145] (Variation 2) In the above embodiment, the first memory 1 and the second memory 2 are separate memories, but the configuration of the first memory 1 and the second memory 2 is not limited to this. The first memory 1 and the second memory 2 may be, for example, a first memory area and a second memory area in the same memory.

[0146] (Variation 3) For example, the data input to the NN circuit 100 described in the above embodiment is not limited to a single format, and can be composed of still images, moving images, audio, text, numeric values, or a combination of these. The data input to the NN circuit 100 is not limited to the measurement results of physical quantity measuring devices such as optical sensors, thermometers, Global Positioning System (GPS) measuring devices, angular velocity measuring devices, and anemometers that may be mounted on the edge device in which the NN circuit 100 is installed. Different information such as base station information received from peripheral devices via wired or wireless communication, vehicle / ship information, weather information, congestion information, and other peripheral information, as well as financial information and personal information may also be combined.

[0147] (Variation 4) Edge devices in which the NN circuit 100 is installed are envisioned to include, but are not limited to, communication devices such as battery-powered mobile phones, smart devices such as personal computers, digital cameras, game consoles, and robotic products. Unprecedented benefits can be achieved by using the NN circuit 100 in products that require peak power limits, reduced heat generation, or extended operation times via Power over Ethernet (PoE). For example, application to on-board cameras installed in vehicles and ships, surveillance cameras installed in public facilities and on roads, etc., not only enables long-term recording but also contributes to weight reduction and durability. Similar benefits can also be achieved by applying the NN circuit 100 to display devices such as televisions and displays, medical devices such as medical cameras and surgical robots, and work robots used in manufacturing and construction sites.

[0148] (Variation 5) The NN circuit 100 may be implemented in part or in whole using one or more processors. For example, the NN circuit 100 may implement in part or in whole the input layer or output layer by software processing by a processor. The part of the input layer or output layer implemented by software processing may be, for example, data normalization or conversion. This makes it possible to accommodate various input or output formats. Note that the software executed by the processor may be configured to be rewritable using communication means or external media.

[0149] (Variation 6) The NN circuit 100 may implement part of the processing in the CNN 200 by combining a graphics processing unit (GPU) or the like on the cloud. The NN circuit 100 can implement more complex processing with fewer resources by performing processing on the cloud in addition to processing performed on the edge device on which the NN circuit 100 is installed, or by performing processing on the edge device in addition to processing on the cloud. With this configuration, the NN circuit 100 can reduce the amount of communication between the edge device and the cloud by distributing processing.

[0150] Furthermore, the effects described herein are merely descriptive or exemplary and are not limiting. In other words, the technology according to the present disclosure may achieve other effects that will be apparent to those skilled in the art from the description of this specification, in addition to or in place of the above-described effects.

[0151] The present invention can be applied to neural network operations.

[0152] 200 Convolutional neural network 100 Neural network circuit (NN circuit) 10 Neural network operation core (NN operation core) 10A First neural network operation core (first NN operation core) 10B Second neural network operation core (second NN operation core) 10M Neural network operation multi-core (NN operation multi-core) 1 First memory 2 Second memory 3 DMA controller (DMAC) 4 Convolution operation circuit 42 Multiplier 43 Accumulator circuit 5 Quantization operation circuit 52 Vector operation circuit 53 Quantization circuit 6 Controller 61 Register 7 IFU

Claims

1. A neural network circuit comprising: a convolution operation circuit which performs a convolution operation on input data and weights, the input data being a third- or higher-order tensor having elements in the x-axis, y-axis, and c-axis directions; the convolution operation circuit comprising: a multiplier which multiplies the weights by divided input data obtained by dividing the input data by a predetermined number of elements in the c-axis direction; and an accumulator circuit which adds up the multiplication operation results of the multipliers; the accumulator circuit having a register which records an addition operation result obtained by accumulating the multiplication operation results for the divided input data that are continuous in the c-axis direction, and which outputs the addition operation result for the divided input data that are continuous in the c-axis direction recorded in the register.

2. The neural network circuit according to claim 1, further comprising a first memory for storing the input data, the first memory storing two or more of the divided input data that are consecutive in the c-axis direction.

3. The neural network circuit according to claim 2, wherein the number of divided input data consecutive in the c-axis direction stored in the first memory is a number obtained by dividing the number of elements in the c-axis direction of the input data by the specified number of elements.

4. The neural network circuit according to claim 1, further comprising: a second memory that stores output data output by the accumulator circuit; and a quantization operation circuit that performs a quantization operation on the output data stored in the second memory.

5. The neural network circuit according to claim 1, wherein at least one of the input data or the weights has a bit precision of 8 bits or less per element.

6. The neural network circuit according to claim 2 or 3, wherein the number of elements in the c-axis direction of the input data held in the first memory is greater than the number of elements in the c-axis direction of the divided input data that can be input to the multiplier.

7. A neural network calculation method for performing a convolution operation between input data, which is a third- or higher-order tensor having elements in the x-axis, y-axis, and c-axis directions, and a weight, comprising: multiplying the input data by divided input data obtained by dividing the input data into a predetermined number of elements in the c-axis direction and the corresponding weight; accumulating the results of the multiplication operation for the divided input data that are continuous in the c-axis direction using a dedicated register; and outputting the results of the addition operation for the divided input data that are continuous in the c-axis direction recorded in the register.

8. The neural network operation method according to claim 7, wherein the divided input data continuous in the c-axis direction is stored in a first memory connected to a convolution operation circuit that performs the convolution operation.

9. The neural network computing method according to claim 8, wherein the number of divided input data consecutive in the c-axis direction stored in the first memory is a number obtained by dividing the number of elements in the c-axis direction of the input data by the specified number of elements.

Images