Neural network circuit device

Abstract

To provide a neural network circuit device which can efficiently perform processing.SOLUTION: A neural network circuit device 100 realizes a neural network model constituted by using weight set by random numbers fixed at a learning stage and a super mask generated at the learning stage. The neural network circuit device 100 comprises a weight generation circuit 200 and an arithmetic circuit 400. The weight generation circuit 200 generates the weight by a random number generator configured to generate the same random numbers as the ones generated at the learning stage by using the same seed as the one used at the learning stage when inference is performed. The arithmetic circuit 400 performs a product-sum operation by using inference object data or input data as active value data corresponding to the inference object data, the weight, and the super mask.SELECTED DRAWING: Figure 5

Description

[Technical Field]

[0001] The present invention relates to a neural network circuit device. [Background technology]

[0002] Conventionally, neural networks such as convolutional neural networks (CNNs) are trained by updating (adjusting) the weights of connections (edges) between nodes (neurons) during the training phase. In contrast, Non-Patent Document 1 discloses a neural network (HNN: Hidden Network) algorithm that does not update weights even during the training phase. The algorithm disclosed in Non-Patent Document 1 extracts subnetworks with high inference accuracy that are hidden in the neural network during the training phase. At this time, the algorithm does not update the randomly set weights. In other words, in the technology disclosed in Non-Patent Document 1, the randomly set weights are fixed. Furthermore, in the technology disclosed in Non-Patent Document 1, a supermask (sometimes simply referred to as a "mask") that defines whether connections between nodes are valid or invalid is determined during the training phase. In the technology disclosed in Non-Patent Document 1, a model (subnetwork) used in inference is constructed using the supermask and fixed weights. [Prior art documents] [Non-patent literature]

[0003] [Non-Patent Document 1] Vivek Ramanujan et al., "What's Hidden in a Randomly Weighted Neural Network?", CVPR 2020, https: / / arxiv.org / abs / 1911.13299 Summary of the Invention [Problem to be solved by the invention]

[0004] Non-Patent Document 1 does not disclose a method for implementing the HNN model used in inference in hardware. Therefore, if an attempt is made to simply implement the algorithm described in Non-Patent Document 1 in hardware, when the amount of model information data becomes enormous, there is a risk that hardware with poor processing efficiency will be configured, requiring enormous amounts of memory capacity and memory access.

[0005] An object of the present invention is to provide a neural network circuit device that can perform processing efficiently. [Means for solving the problem]

[0006] The neural network circuit device of the present invention is a neural network circuit device that realizes a neural network model in which, in a learning stage, the weights of each connection between nodes are set by fixed random numbers, a mask is generated by determining whether each of the connections is valid or invalid, and the neural network circuit device is configured using the weights and the mask, and the neural network circuit device has: a weight generation circuit that generates the weights using a random number generator configured to generate the same random numbers as those generated in the learning stage using the same seed as that used in the learning stage when performing inference; and an arithmetic circuit that performs a product-sum operation using input data that is the data to be inferred or activation value data corresponding to the data to be inferred, the weights, and the mask. [Effects of the Invention]

[0007] An object of the present invention is to provide a neural network circuit device that can perform processing efficiently. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 1 is a diagram for explaining an HNN algorithm. [Figure 2] FIG. 1 is a diagram for explaining an HNN algorithm. [Figure 3]1 is a graph showing the relationship between the value of k in the supermask and the accuracy of inference in the HNN algorithm. [Figure 4] FIG. 10 is a diagram for explaining selection of a supermask according to the present embodiment. [Figure 5] FIG. 1 is a diagram illustrating a configuration of a neural network circuit device according to a first embodiment. [Figure 6] 4 is a diagram for explaining the processing of a weight generation circuit and a mask development circuit according to the first embodiment; FIG. [Figure 7] FIG. 2 is a diagram illustrating a configuration of a weight generation circuit according to the first embodiment. [Figure 8] FIG. 2 is a diagram illustrating a configuration of a mask developing circuit according to the first embodiment. [Figure 9] FIG. 2 is a diagram for explaining input data according to the first embodiment. [Figure 10] FIG. 2 is a diagram for explaining the operation of the arithmetic circuit according to the first embodiment. [Figure 11] FIG. 2 is a diagram for explaining processing of a PE vector according to the first embodiment; [Figure 12] FIG. 2 is a timing chart showing the components of the neural network circuit device according to the first embodiment; [Figure 13] FIG. 2 is a diagram illustrating a configuration of a post-processing circuit according to the first embodiment. [Figure 14] FIG. 4 is a diagram showing the results of calculations performed by the neural network circuit device according to the first embodiment. [Figure 15] FIG. 4 is a diagram showing the results of calculations performed by the neural network circuit device according to the first embodiment. [Figure 16] FIG. 4 is a diagram showing the results of calculations performed by the neural network circuit device according to the first embodiment. [Figure 17] FIG. 4 is a diagram showing the results of calculations performed by the neural network circuit device according to the first embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0009] (Outline of the embodiment) Before describing the embodiments, an overview of the embodiments of the present invention will be described. Note that, although the embodiments of the present invention will be described below, the following embodiments do not limit the scope of the invention according to the claims. Furthermore, not all of the combinations of features described in the embodiments are necessarily essential to the solution of the invention.

[0010] <HNNについて> 1 and 2 are diagrams for explaining the HNN algorithm. The HNN algorithm corresponds to the technology disclosed in Non-Patent Document 1. On the left side of Fig. 1, a diagram for explaining the learning of a model (neural network) according to the HNN algorithm is shown.

[0011] In the HNN algorithm, an initial model 10 and a supermask 20 are provided in the learning stage. The initial model 10 is composed of nodes 12 (neurons) and connections 13 between the nodes 12. Here, a weight 14 is set for each connection 13. The weight 14 for each connection is set by, for example, a random number. As described above, in the HNN algorithm, the weights 14 are not updated by learning but are fixed.

[0012] The supermask 20 defines whether each connection 13 is valid or invalid. The supermask 20 is generated using a score 30 indicating the importance of each connection 13. The score 30 is determined in the training phase. In other words, the HNN algorithm trains a score 30 for each of the multiple connections 13. The score 30 is trained, for example, by backpropagation.

[0013] FIG. 2 is a diagram illustrating a method for determining a supermask 20. FIG. 2 is a diagram illustrating a distribution of scores 30. The horizontal axis of FIG. 2 indicates the score 30 value, and the vertical axis indicates the number of connections 13 having the corresponding score 30 value. Here, the supermask 20 is generated by determining that connections 13 with high score 30 values ​​are valid and connections 13 with low score 30 values ​​are invalid. Specifically, the supermask 20 is generated by determining that connections 13 corresponding to the top k% of scores 30 (connections corresponding to the hatched portions in FIG. 2) are valid and the remaining connections 13 are invalid. Here, the value of k is preset for each supermask 20 to be generated. Then, a supermask 20 can be provided for each of multiple k's.

[0014] The thick solid lines in the supermask 20 shown in FIG. 1 correspond to the connections 13 corresponding to the top k% of scores 30. Therefore, these thick solid lines correspond to the connections 13 determined to be valid. On the other hand, the dashed lines in the supermask 20 shown in FIG. 1 correspond to the connections 13 that do not correspond to the top k% of scores 30 (i.e., the connections 13 that correspond to the bottom (100-k)% of scores 30). Therefore, these dashed lines correspond to the connections 13 determined to be invalid. Here, in the supermask 20 according to this embodiment, a "1" is assigned to the connections 13 determined to be valid (i.e., the connections 13 that correspond to the top k% of scores 30). On the other hand, a "0" is assigned to the connections 13 determined to be invalid (i.e., the connections 13 that do not correspond to the top k% of scores 30). Note that the score 30 (and the supermask 20) ​​is learned based on the value of k corresponding to the supermask 20. Specifically, the score 30 is learned for each value of k corresponding to the supermask 20 to be generated. For example, when generating a supermask 20 with k=20%, the top 20% of the scores 30 are used to learn (update) all the scores 30. Note that, as described above, the weights 14 are not updated at this time.

[0015] Then, by combining the initial model 10 and the supermask 20, a sub-network 40 shown on the right side of FIG. 1 is constructed. That is, a model of a neural network (sub-network 40) is constructed by using each connection 13 (weight 14) of the initial model 10 and the supermask 20. Specifically, the sub-network 40 is constructed by logically ANDing each connection 13 (weight 14) of the initial model 10 with the valid or invalid information (e.g., "1" or "0") of each connection 13 shown in the supermask 20. In the HNN algorithm, inference is performed using this sub-network 40 (neural network model).

[0016] Figure 3 is a graph showing the relationship between the k value of SuperMask20 in the HNN algorithm and inference accuracy. The top graph in Figure 3 shows the results when the HNN algorithm is applied to ResNet50 and ImageNet. The bottom graph in Figure 3 shows the results when the HNN algorithm is applied to VGG16 and CIFAR-100. The horizontal axis of each graph corresponds to the k value, and the vertical axis corresponds to the inference accuracy. As shown in Figure 3, when the k value is between 0 and 30%, the larger the k value, the higher the inference accuracy can be.

[0017] As described above, the supermask 20 according to this embodiment is binary information of "0" or "1." Therefore, the supermask 20 generated in the learning stage can be compressed and stored, and the compressed supermask 20 can be read and expanded in the inference stage. By compressing the supermask 20, the size of the supermask 20 corresponding to the model (trained model) read in the inference stage can be reduced. In other words, the amount of data transmitted when reading the supermask 20 can be reduced. This can save power consumption. Therefore, the inference process can be performed more efficiently.

[0018] The smaller the value of k, the fewer the number of "1"s in the supermask 20 (i.e., the number of connections 13 determined to be "valid") and the more the number of "0"s (i.e., the number of connections 13 determined to be "invalid"). Therefore, by compressing the supermask 20 using the Zero-Run-Length (ZRL) compression method, the smaller the value of k for the supermask 20, the higher the compression rate in the compressed state. Note that a "high compression rate" means that the ratio of the data size after compression to the data size before compression is small, that is, good compression. Here, a supermask 20 with a small value of k has fewer "1"s, and can therefore be said to be sparse. Therefore, the sparser the supermask 20, the higher the compression rate. In other words, the sparser the supermask 20 in the compressed state, the higher the compression rate at which it is compressed.

[0019] As described above, in the example of FIG. 3, when the value of k is approximately 0 to 30%, the larger the value of k, the higher the inference accuracy, and the smaller the value of k, the lower the inference accuracy. As described above, the smaller the value of k, the sparser the supermask 20, and therefore the higher the compression rate. Therefore, when the value of k is approximately 0 to 30%, if you want to increase the inference accuracy, you can select a supermask 20 with a large value of k. On the other hand, if you want to perform the inference process efficiently (reduce communication volume and power consumption), you can select a supermask 20 with a small value of k (i.e., a high compression rate). This easily achieves a trade-off between the accuracy of inference and the efficiency of the inference process. Therefore, it is possible to separately generate a supermask 20 that prioritizes the accuracy of inference and a supermask 20 that prioritizes the efficiency of the inference process (reduce communication volume and power consumption).

[0020] Furthermore, a supermask 20 may be generated for each inference target. For example, in an inference to detect animals from image data, different supermasks 20 may be generated when the inference target is a "dog" and when the inference target is a "cat." In this case, for example, a supermask 20 with a relatively large k value (e.g., about 20 to 30%) and a supermask 20 with a relatively small k value (e.g., about 1 to 10%) can be generated when the inference target is learned as a "dog." The same applies when the inference target is learned as a "cat." Therefore, a supermask 20 may be generated for at least one of the inference accuracy and the inference target. This allows inference to be easily performed for different inference targets and with different accuracy simply by switching the supermasks 20 without changing the weights 14.

[0021] FIG. 4 is a diagram illustrating the selection of a supermask 20 according to the present embodiment. As described above, in the initial model 10, a weight 14 is assigned to each connection 13. These weights 14 are fixed. Inference can be performed by superimposing a supermask 20 on the initial model 10. Here, assume that a supermask 20A with k=30%, a supermask 20B with k=20%, and a supermask 20C with k=10% are generated. When the supermask 20A is selected, power consumption during inference may be relatively high, but the inference accuracy may be relatively good. On the other hand, when the supermask 20B is selected, power consumption during inference may be relatively reduced compared to the case of the supermask 20A, but the inference accuracy may be degraded. Furthermore, when the supermask 20C is selected, power consumption during inference may be further reduced compared to the case of the supermask 20B, but the inference accuracy may be further degraded. In this way, the supermask 20 to be used can be selected depending on whether emphasis is placed on inference accuracy or on inference processing efficiency (power consumption reduction).

[0022] As described above, in the HNN algorithm, the weights 14 are set by random numbers in the initial model 10 and are fixed. Here, the weights used in the inference stage must be the same as the weights set in the learning stage. In this case, one method is to store the weights set by random numbers in the learning stage and use the stored weights in the inference stage. However, in this case, a storage area for storing the weights is required.

[0023] On the other hand, as described above, since weights are generated by random numbers, weights 14 can be generated using a random number generator. Here, if the same random number generator is used and the same seed is input to the random number generator, the same random numbers will be output. In this embodiment, the seed and random number generator used in the learning stage are used in the inference stage. As a result, the same random numbers generated in the learning stage are also generated in the inference stage. Therefore, the same weights set in the learning stage can be used in the inference stage. In this case, since there is no need to store weights 14 in the device, it is possible to reduce the size of the device's storage area. Therefore, it is possible to reduce memory capacity. Furthermore, since there is no need to access memory when setting weights in the inference stage, it is possible to suppress memory access. Therefore, it is possible to achieve good processing efficiency.

[0024] Furthermore, the HNN algorithm provides good inference accuracy even when the weight 14 is set to "+1" or "-1." Therefore, the weight 14 can be binary information. That is, for example, the weight 14 can be binary information by interpreting the binary information "1" as "+1" and "0" as "-1." Furthermore, as described above, the supermask 20 is also binary information. Therefore, in this embodiment, when the HNN algorithm is implemented in hardware, it is possible to reduce the size of the device.

[0025] (Embodiment 1) Hereinafter, embodiments will be described with reference to the drawings. For clarity of explanation, the following description and drawings have been omitted and simplified as appropriate. In addition, the same elements in each drawing are designated by the same reference numerals, and duplicate explanations have been omitted as necessary.

[0026] <Neural network circuit device> FIG. 5 is a diagram illustrating a configuration of a neural network circuit device 100 according to the first embodiment. The neural network circuit device 100 according to the first embodiment includes a control circuit 110, a weight generation circuit 200, a mask expansion circuit 300, an arithmetic circuit 400, an activation value data memory 500, a barrel shifter 130, and a post-processing circuit 140. The neural network circuit device 100 is hardware configured to perform inference using the above-described HNN algorithm during inference. The neural network circuit device 100 may include the above circuits on a single chip. The neural network circuit device 100 realizes a neural network model configured using weights set in the learning stage and a supermask generated in the learning stage.

[0027] When performing inference, the weight generation circuit 200 (WGU: Weight Generation Unit) generates weights 14 corresponding to each connection 13 using a random number generator configured to generate the same random numbers as those generated in the learning stage using the same seed as that used in the learning stage. The weight generation circuit 200 generates weights 14 for each kernel (filter, output channel). Details will be described later.

[0028] When performing inference, the mask expansion circuit 300 (SEU: Supermask Expansion Unit) expands the compressed supermask 20. The mask expansion circuit 300 expands the supermask 20 for each kernel (filter, output channel). This will be described in detail later.

[0029] 6 is a diagram for explaining the processing of the weight generation circuit 200 and the mask expansion circuit 300 according to the first embodiment. The weight generation circuit 200 generates weights 14 corresponding to each connection 13, thereby forming an initial model 10 set in the learning stage. The mask expansion circuit 300 expands the supermask 20, thereby obtaining the original, uncompressed supermask 20. Then, the supermask 20 is superimposed on the initial model 10, thereby enabling inference to be performed.

[0030] When performing inference, the control circuit 110 (MCC: Model Construction Controller) controls the construction of an inference model (a combination of the initial model 10 and the supermask 20) ​​constructed in the learning stage. In particular, the control circuit 110 controls the weight generation circuit 200 and the mask expansion circuit 300. The control circuit 110 generates a seed to be input to a random number generator used in the weight generation circuit 200. The control circuit 110 also outputs (provides) information on the compression type (code length) of the compressed supermask 20 to the mask expansion circuit 300. The control circuit 110 also controls the operation of the weight generation circuit 200 so that the operation of the mask expansion circuit 300 is synchronized. This will be described in detail later. The control circuit 110 may include a processor and a storage device. The functions of the control circuit 110 may be implemented by a program stored in the storage device and executed by the processor.

[0031] The activation data memory 500 (AMEM: Activation Memory) stores input data (iAct). Here, "input data" refers to inference target data such as image data, or activation data corresponding to the inference target data. The activation data corresponds to data generated by processing in the first convolutional layer using the inference target data, or data generated by processing in the second or subsequent convolutional layers. In other words, the activation data corresponds to output data from processing in the previous convolutional layer. When the inference target data is image data, the input data can be configured three-dimensionally by the horizontal and vertical directions of the image and input channels. This will be described in detail later. Note that when the input data is inference target data, the input channels correspond to the channels of the image (e.g., each of the RGB values). When the input data is activation data, the input channels correspond to the type of kernel from processing in the previous convolutional layer (i.e., the output channels of the output data from processing in the previous convolutional layer).

[0032] The barrel shifter 130 adjusts the position of the input data. Specifically, the barrel shifter 130 adjusts the position of the input data so that each piece of input data is placed at an appropriate processing target position when the input data is processed by the arithmetic circuit 400. The barrel shifter 130 shifts the three-dimensionally configured input data in the vertical or horizontal direction.

[0033] The arithmetic circuit 400 performs a product-sum operation using the input data, the weights 14 generated by the weight generation circuit 200, and the supermask 20 expanded by the mask expansion circuit 300. The arithmetic circuit 400 performs an operation such that the product of the input data and the weights 14 is validated or invalidated by the supermask 20. The arithmetic circuit 400 performs a product-sum operation for each input channel and for each kernel (output channel). The arithmetic circuit 400 outputs partial sums (PSUM) obtained by each product-sum operation to the post-processing circuit 140. Details will be described later.

[0034] The post-processing circuit 140 (PPU: Post-Processing Unit) performs necessary processing for the convolutional layer in the CNN. The post-processing circuit 140 performs processing subsequent to the processing executed by the arithmetic circuit 400. The post-processing circuit 140 stores the obtained output data (oAct) in the activity value data memory 500. This will be described in detail later.

[0035] 7 is a diagram illustrating a configuration of a weight generation circuit 200 according to the first embodiment. The weight generation circuit 200 is configured to generate weights 14 using a random number generator configured to generate the same random numbers as those generated in the learning stage using the same seed as that used in the learning stage when performing inference. The weight generation circuit 200 can perform processing in parallel for each output channel (i.e., for each kernel). In other words, the weight generation circuit 200 can generate weights 14 for multiple kernels (i.e., output channels) in parallel.

[0036] The weight generation circuit 200 has weight generation units 210 that perform processing for each output channel (i.e., for each kernel) equal to the number of output channels that can be processed in parallel. In the example of FIG. 7, the weight generation circuit 200 performs processing for each of 16 output channels (16oChs). That is, the weight generation circuit 200 has 16 weight generation units 210. The weight generation circuit 200 has a random number generator 202 and a selector 204 for each output channel (i.e., for each weight generation unit 210). The random number generator 202 can be realized by a pseudo-random number generation algorithm such as Xorshift16.

[0037] The number of output channels of the neural network model may be greater than the number of output channels to be processed (i.e., the number of weight generation units 210). That is, the number of output channels does not have to be 16, and may be, for example, 256 or 512. When there are 256 output channels, the weight generation circuit 200 may perform processing for each of the 16 output channels 16 times. This also applies to the mask development circuit 300 and the arithmetic circuit 400 described later.

[0038] As described above, the weight generation unit 210 generates a weight for each kernel. That is, one weight generation unit 210 generates a kernel weight group 220, which is a collection of weights corresponding to one kernel. Here, the kernel weight group 220 is configured three-dimensionally corresponding to the three-dimensional configuration of the input data. The kernel weight group 220 has a width kw corresponding to the horizontal direction of the input data, a height kh corresponding to the vertical direction of the input data, and a depth of 16 channels (16iChs) corresponding to the input channels to be processed in the input data. One weight generation unit 210 generates a kernel weight portion 222 having a size of 1×1×16 in one cycle. Note that the kernel weight portion 222 is a collection of 1-bit weights 14, the number of which is the same as the number of input channels to be processed (16). Therefore, by the weight generation unit 210 repeating the process in kw×kh cycles, a kernel weight group 220 is generated, which is a collection of kw×kh kernel weight portions 222.

[0039] The seed input to the random number generator 202 can be generated by the control circuit 110 using a method described below. In the example of Fig. 7, the control circuit 110 generates a different 16-bit seed for each of the 16 output channels. Therefore, the weight generation circuit 200 (random number generator 202) generates random numbers using a different seed for each output channel. This allows the weight generation circuit 200 to generate different weights (different kernel weight portions 222 and kernel weight groups 220) for each output channel.

[0040] In the first cycle, the random number generator 202 generates a random number using the seed output from the control circuit 110. This generates the first kernel weight portion 222. Then, in the nth cycle (n is an integer greater than or equal to 2 and less than or equal to (kw×kh)), the random number generator 202 generates a random number using the random number (weight) generated in the (n-1)th cycle. This generates the nth kernel weight portion 222. Therefore, in the first cycle, the selector 204 selects and outputs the seed (initial value) of the random number acquired from the control circuit 110 to the random number generator 202, and in the second cycle and thereafter, selects the previous output (generated random number) of the random number generator 202. Each weight generation unit 210 outputs the generated kernel weight portion 222 to the arithmetic circuit 400. Therefore, the weight generation circuit 200 outputs 16 kernel weight portions 222, corresponding to the number of output channels to be processed, to the arithmetic circuit 400 in one cycle. As a result, the weight generation circuit 200 outputs 16(iCh)×16(oCh)=256 (that is, 256 bits) weights 14 to the arithmetic circuit 400 in one cycle.

[0041] In this way, the weight generation circuit 200 is configured to generate the same weights in the inference stage as those set in the learning stage. Therefore, it is not necessary to store weights. This makes it possible to reduce memory capacity. Furthermore, since it is not necessary to access memory when setting weights in the inference stage, it is possible to reduce memory access. This makes it possible to reduce power consumption. This improves processing efficiency.

[0042] The control circuit 110 generates a seed using an output channel number (#oCh) and a layer number (#Layer). Here, the output channel number corresponds to identification information (index number) of the output channel to be processed. The layer number corresponds to identification information (index number) of a layer (convolutional layer) that constitutes the neural network model. For example, in processing the first layer, the control circuit 110 may use "#oCh=0" and "#Layer=0" when generating a seed to be supplied to the front-most weight generation unit 210 shown in FIG. 7.

[0043] In this way, by generating seeds using the output channel number (#oCh) and layer number (#Layer), different seeds can be generated for each process and output channel for each layer. This breaks the periodicity of random number generation for each process and output channel for each layer. In other words, random numbers can be generated with different periods for each process and output channel for each layer. Therefore, even if random numbers generated using a simple pseudorandom number generation algorithm such as Xorshift16 are used as weights, deterioration in inference accuracy can be suppressed. Therefore, it is possible to use a pseudorandom number generation algorithm with a short period for pseudorandom numbers. This allows the random number generator 202 to be made smaller.

[0044] Alternatively, the control circuit 110 may generate a seed using an arbitrary hash function that takes the output channel number (#oCh) and the layer number (#Layer) as input. By generating a seed in this manner, it is possible to further reduce the amount of data stored for generating weights while suppressing deterioration in inference accuracy.

[0045] 8 is a diagram showing the configuration of a mask expansion circuit 300 according to the first embodiment. The mask expansion circuit 300 is configured to expand the supermask 20 in a compressed state when performing inference. As described above, in the pre-stage of the inference process, the supermask 20 is compressed by the zero-run length compression method. The supermask 20 is compressed to a predetermined code length. In this embodiment, the code length may be determined according to the value of k corresponding to the supermask 20 to be processed.

[0046] Specifically, as described above, the smaller the value of k, the sparser the supermask 20. In other words, the smaller the value of k, the fewer the number of "1"s in the supermask 20 (i.e., the number of connections 13 determined to be "valid") and the more the number of "0"s (i.e., the number of connections 13 determined to be "invalid"). Therefore, in this case, performing zero run length compression with a long code length can increase the compression rate. On the other hand, the larger the value of k, the denser the supermask 20. In other words, the larger the value of k, the more "1"s there are and the fewer "0"s there are in the supermask 20. Therefore, in this case, performing zero run length compression with a long code length can increase the data length after compression compared to before compression, so performing zero run length compression with a short code length can increase the compression rate.

[0047] For example, the code length of zero run length compression is {2, 3, 4} bits. In this case, a supermask 20 with a k value of approximately 1 to 15% may be compressed using a compression method with a code length of "4 bits." A supermask 20 with a k value of approximately 15 to 25% may be compressed using a compression method with a code length of "3 bits." A supermask 20 with a k value of approximately 25 to 35% may be compressed using a compression method with a code length of "2 bits."

[0048] The mask expansion circuit 300 includes a compressed data memory 302, a FIFO buffer 304, a decoder 306, and a mask memory 310. A FIFO buffer 304 and a decoder 306 are provided for each output channel to be processed (i.e., for each kernel to be processed). In the example of Fig. 8, a FIFO buffer 304 and a decoder 306 are provided for each of the 16 output channels. As a result, the FIFO buffer 304 and the decoder 306 perform processing in parallel for each output channel (i.e., for each kernel).

[0049] The compressed data memory 302 (ZMEM: ZRL Memory) stores compressed supermask 20 (compressed supermask) data. The compressed data memory 302 acquires compressed supermask (ZRL Encoded Supermask) data. The compressed data memory 302 may acquire the compressed supermask data from the control circuit 110. Here, as shown in FIG. 8, the compressed supermask data may be acquired in a state where it is divided into a predetermined data amount for each output channel and stored in the compressed data memory 302. For example, in FIG. 8, each piece of data "oCh0" stored in the compressed data memory 302 corresponds to the divided compressed supermask data corresponding to the kernel of output channel #0. Note that the supermask 20 acquired by the mask expansion circuit 300 does not have to be compressed. In this case, the acquired supermask 20 may be input directly to the mask memory 310 (described later) rather than to the compressed data memory 302 (pass-through mode).

[0050] The FIFO buffer 304 (FIFO: First-In First-Out) is configured to combine the divided compressed supermasks. Specifically, the FIFO buffer 304 combines the divided compressed supermasks in parallel for each output channel (i.e., for each kernel). For example, the frontmost FIFO buffer 304 shown in FIG. 8 combines multiple pieces of “oCh0” data stored in the compressed data memory 302. Note that the FIFO buffer 304 for each output channel may combine the divided compressed supermasks so as to generate a compressed supermask corresponding to one kernel to be processed. The FIFO buffer 304 for each output channel receives 32 bits of data from the compressed data memory 302 at a time. The FIFO buffer 304 may then output up to 4 bits (corresponding to the maximum code length) of data to the decoder 306 at a time. Here, the amount of output data corresponds to the code length of the compressed supermask. Information on the code length for the acquired compressed supermask may be received from the control circuit 110.

[0051] The decoder 306 (ZRL Decoder) is configured to decode the compressed supermask. The decoder 306 decodes the compressed supermask in parallel for each output channel (i.e., for each kernel). The decoder 306 may receive code length information from the control circuit 110. The decoder 306 decompresses the compressed data output from the FIFO buffer 304 using the received code length information. The decoder 306 may output the decompressed supermask data to the mask memory 310, for example, in 8-bit increments for each output channel.

[0052] The mask memory 310 (SMEM: Supermask Memory) temporarily stores the expanded supermask data. The mask memory 310 stores the kernel mask group 320, which is a collection of supermask data corresponding to one kernel to be processed, as many times as the number of kernels to be processed (the number of output channels: 16oChs). Here, the size of the kernel mask group 320 is substantially the same as the size of the kernel weight group 220 described above. That is, the kernel mask group 320 has a width kw corresponding to the horizontal direction of the input data, a height kh corresponding to the vertical direction of the input data, and a depth of 16 bits (16iChs) corresponding to the input channel of the input data to be processed.

[0053] The mask memory 310 outputs a kernel mask portion 322 of size 1×1×16 to the arithmetic circuit 400 in one cycle in parallel for each output channel. The size of the kernel mask portion 322 is substantially the same as the size of the kernel weight portion 222 described above. Therefore, the kernel mask portion 322 is a collection of 1-bit mask data, the number of which is equal to the number of input channels to be processed (16). Then, the kernel mask group 320 is configured by the kw×kh kernel mask portions 322. Then, for each output channel (kernel), the kernel mask portion 322 is output to the arithmetic circuit 400 in kw×kh cycles.

[0054] The mask memory 310 has two banks 312 (bank 312A (Bank 1) and bank 312B (Bank 2)). Double buffering is achieved by the banks 312A and 312B. Specifically, for example, first, data output from the decoder 306 is stored in the bank 312A. Then, after data of the kernel mask group 320 corresponding to the kernel to be processed is stored in the bank 312A, the data stored in the bank 312A is read out and output to the arithmetic circuit 400, and is used for the calculation. While the data stored in the bank 312A is read out and output to the arithmetic circuit 400, data of the kernel mask group 320 corresponding to the kernel to be next processed is stored in the bank 312B.

[0055] Then, when all the data stored in bank 312A is read out and output to the arithmetic circuit 400, the data stored in bank 312B is read out and output to the arithmetic circuit 400, and is used in the arithmetic operation. While the data stored in bank 312B is being read out and output to the arithmetic circuit 400, the data of the kernel mask group 320 corresponding to the kernel to be processed next is stored in bank 312A. This process is repeated thereafter. This allows the processing corresponding to the arithmetic circuit 400 to be performed efficiently. In other words, since the mask memory 310 reads and writes (stores) data in parallel, the arithmetic circuit 400 can perform the processing efficiently.

[0056] Here, the control circuit 110 synchronizes the timing at which the kernel weight portion 222 is output from the weight generation circuit 200 (random number generator 202) with the timing at which the kernel mask portion 322 is output from the mask development circuit 300 (mask memory 310). Specifically, the control circuit 110 outputs a read signal (read_addr) to the mask memory 310 at a timing corresponding to the timing at which the seed is output to the weight generation circuit 200. As a result, the kernel weight portion 222 and the kernel mask portion 322 corresponding to the same position (position on kw×kh) in the same kernel can be output to the arithmetic circuit 400 at substantially the same timing.

[0057] 9 is a diagram for explaining input data according to the first embodiment. As described above, the input data 510 has a three-dimensional structure consisting of the horizontal direction (width: Width) of the image, the vertical direction (height: Height) of the image, and the input channel (depth: iChs). In the first embodiment, chunks 520 are considered as units of calculation of the input data 510 in the arithmetic circuit 400. The chunks 520 correspond to part of the data of the three-dimensionally configured input data 510. Data corresponding to one chunk 520 can be input to the arithmetic circuit 400 at a time.

[0058] The chunk 520 has a size of, for example, width cw=4, height ch=4, and depth (input channel) iCh=16. In this case, if the size of the input data 510 is Width=12, Height=12, and iChs=48, the input data 510 has 3×3×3=27 chunks 520. Here, the input data 510 is, for example, a collection of 8-bit data. In this case, the data size of the input data 510 is 12×12×48×8 [bits]. Furthermore, the data size of one chunk 520 is 4×4×16×8 [bits].

[0059] 10 is a diagram for explaining the operation of the arithmetic circuit 400 according to the first embodiment. As described above, the kernel weight portions 222 corresponding to each output channel (kernel) are input from the weight generation circuit 200 to the arithmetic circuit 400. In the example of FIG. 10, as indicated by the upward arrows, the kernel weight portions 222 corresponding to each of the 16 output channels (16oChs) are input to the arithmetic circuit 400. Similarly, the mask development circuit 300 inputs the kernel mask portions 322 corresponding to each of the output channels (kernels) to the arithmetic circuit 400. In the example of FIG. 10, as indicated by the downward arrows, the kernel mask portions 322 corresponding to each of the 16 output channels (16oChs) are input to the arithmetic circuit 400.

[0060] Furthermore, data of a chunk 520 is input to the arithmetic circuit 400. Here, the chunk 520 is composed of cw×ch chunk portions 522. In the above example, the chunk 520 has 4×4=16 chunk portions 522. The size of the chunk portion 522 is 1×1×16. In other words, the size of the chunk portion 522 corresponds to the size of the kernel weight portion 222 and the kernel mask portion 322. Then, the 16 (=4×4) chunk portions 522 are input to the arithmetic circuit 400 in parallel.

[0061] The arithmetic circuit 400 has a large number of processing elements (PEs). The arithmetic circuit 400 also has a plurality of PE matrices 410. Here, the number of PE matrices 410 provided corresponds to the number of output channels to be processed (i.e., the number of kernels to be processed). In the example of FIG. 10 , the arithmetic circuit 400 has 16 PE matrices 410 (16 PE Matrices). In addition, one PE matrix 410 has a plurality of PE vectors 420. Each PE matrix 410 has PE vectors 420, the number of which corresponds to the number (cw×ch) of input chunk portions 522. In the example of FIG. 10 , each PE matrix 410 has 16 (=4×4) PE vectors 420 (16 PE Vectors). That is, the PE vectors 420 in the PE matrix 410 correspond to the positions (positional deviations) of the chunk portions 522 in the chunks 520. As will be described later, the PE vector 420 has processing elements whose number corresponds to the number of input channels to be processed.

[0062] Here, the kernel weight portion 222 and the kernel mask portion 322 corresponding to the same position (position on kw×kh) in the same kernel are input to the same PE matrix 410. That is, it is assumed that the kernel weight group 220 and the kernel mask group 320 shown in FIG. 10 correspond to the same kernel. In this case, the kernel weight portion 222 at the position (kw,kh)=(x1,y1) in the kernel weight group 220 and the kernel mask portion 322 at the position (kw,kh)=(x1,y1) in the kernel mask group 320 are input to the same PE matrix 410. In this case, in parallel with the above, the kernel weight portion 222 and the kernel mask portion 322 at the position (kw,kh)=(x1,y1) for another kernel are input to another PE matrix 410.

[0063] These kernel weight portions 222 and kernel mask portions 322 are input to multiple PE vectors 420 of the same PE matrix 410. In other words, the same kernel weight portion 222 and the same kernel mask portion 322 are input to multiple PE vectors 420 included in a certain PE matrix 410. In other words, each PE matrix 410 is associated with each kernel (output channel).

[0064] Furthermore, chunk portions 522 at the same position are input to PE vectors 420 at the same position in each PE matrix 410, as indicated by right-pointing arrows. That is, each of the multiple chunk portions 522 is input to a PE vector 420 corresponding to its position in that chunk 520 in each PE matrix 410. For example, the topmost chunk portion 522 in FIG. 10 is input to the topmost PE vector 420 in each PE matrix 410. The same applies to chunk portions 522 at other positions. In this way, the PE vectors 420 in each PE matrix 410 are associated with positions (positional deviations) within the chunk 520.

[0065] Therefore, a chunk portion 522 at a position in the chunk 520 corresponding to the PE vector 420 is input to each PE vector 420. Furthermore, a kernel weight portion 222 and a kernel mask portion 322 related to the kernel corresponding to the PE vector 420 (PE matrix 410) are input to each PE vector 420. Then, as will be described later, each PE vector 420 performs a product-sum operation using the chunk portion 522, kernel weight portion 222, and kernel mask portion 322.

[0066] Note that by inputting another chunk 520 in the input data 510 to the arithmetic circuit 400, it is possible to perform similar product-sum operations on the data of the other chunk 520. Furthermore, by inputting the kernel weight portion 222 and the kernel mask portion 322 at other positions (other positions in kw × kh) in the kernel weight set 220 and the kernel mask set 320, respectively, to the arithmetic circuit 400, it is possible to perform similar product-sum operations on positions on other kernels. For example, it is assumed that the kernel weight portion 222 and the kernel mask portion 322 corresponding to a certain position of each kernel (e.g., (kw, kh) = (x1, y1)) are input to each PE vector 420 in the corresponding PE matrix 410. Meanwhile, by changing the chunk 520 to be input, the arithmetic circuit 400 can perform product-sum operations on the above positions for all data in the input data 510. Then, the kernel weight portion 222 and the kernel mask portion 322 corresponding to a different position of each kernel (for example, (kw, kh) = (x1, y2)) are input to each PE vector 420 in the corresponding PE matrix 410. Meanwhile, by changing the input chunk 520, the arithmetic circuit 400 can similarly perform a multiply-and-accumulate operation for the above positions for all data in the input data 510. In this way, it is possible to perform a multiply-and-accumulate operation for all positions of the input data 510 for all positions in each kernel. Note that the input timing of these data may be adjusted by, for example, the control circuit 110 using a clock input to the neural network circuit device 100.

[0067] 11 is a diagram for explaining the processing of the PE vector 420 according to the first embodiment. The PE vector 420 has a number of processing elements 430 (PEs) corresponding to the number of input channels to be processed. In the above example, the number of input channels to be processed is 16, so the PE vector 420 has 16 processing elements 430 (16 PEs). Note that, due to space limitations, only eight processing elements 430 are shown in FIG. 11.

[0068] As described above, one kernel weight portion 222, one kernel mask portion 322, and one chunk portion 522 are input to one PE vector 420. Then, data corresponding to each input channel of the kernel weight portion 222, the kernel mask portion 322, and the chunk portion 522 is input to one arithmetic element 430. That is, the PE vector 420 has arithmetic elements 430 corresponding to each input channel. For example, data of input channel #0 of each of the kernel weight portion 222, the kernel mask portion 322, and the chunk portion 522 is input to the arithmetic element 430 corresponding to input channel #0 (the arithmetic element 430 at the bottom right and the front in FIG. 11 ). Furthermore, data of input channel #15 of each of the kernel weight portion 222, the kernel mask portion 322, and the chunk portion 522 is input to the arithmetic element 430 corresponding to input channel #15 (the arithmetic element 430 at the top left and the back in FIG. 11 ).

[0069] The arithmetic element 430 performs a logical product operation using the data of each input channel of the input kernel weight portion 222, kernel mask portion 322, and chunk portion 522. Then, the PE vector 420 calculates the sum of the logical products obtained by each of the 16 arithmetic elements 430 as a partial sum (PSUM). Then, the arithmetic circuit 400 outputs the partial sums obtained by each PE vector 420 (a total of 16 × 16 partial sums) to the post-processing circuit 140.

[0070] Here, the partial sums are PSUM=Σif(SMask){iAct×Weight} ···(1) Here, Equation 1 means that for each input channel, when the value of the supermask (SMask) is "1" (i.e., not "0"), the product of the input data value (iAct) and the weight value (Weight) is calculated, and the sum of the products obtained for each input channel is calculated. In other words, Equation 1 means that an operation is performed so that the product of the input data and the weight value is enabled or disabled depending on the value of the supermask (1 or 0).

[0071] Here, formula 1 is equivalent to formula 2 below. PSUM=Σ{(iAct and SMask)×Weight} ···(2) Equation 2 means that for each input channel, the product of the logical product of the input data value (iAct) and the supermask value (SMask) and the weight value (Weight) is calculated, and the sum of the products obtained for each input channel is calculated.

[0072] The arithmetic element 430 according to the first embodiment performs an operation corresponding to Equation 2. That is, the arithmetic element 430 (arithmetic circuit 400) multiplies the logical product of input data input to a certain connection and a value (1 or 0) in the supermask indicating validity or invalidity corresponding to that connection by the weight corresponding to that connection. Then, the PE vector 420 (arithmetic circuit 400) calculates the sum of the multiplied values ​​obtained for each input channel as a partial sum (PSUM). Note that the weight value is "+1" or "-1", so multiplying the logical product of the input data value (iAct) and the supermask value (SMask) by the weight value (Weight) corresponds to simple sign inversion.

[0073] For simplicity of explanation, a specific example of the above calculation will be described assuming that the number of input channels is four. It is assumed that the data input to the PE vector 420 is as follows. "iAct" = {5,3,7,1} "SMask"={1,0,1,1} "Weight" = {+1,-1,-1,+1}

[0074] At this time, for the first input channel, "SMask" is "1", so "iAct" is valid. The calculation element 430 corresponding to the first input channel calculates the product of "iAct" = "5" and "Weight" = "+1" as "+5". For the second input channel, "SMask" is "0", so the input data is invalid. Therefore, the calculation element 430 corresponding to the second input channel does not perform a product operation.

[0075] Furthermore, for the third input channel, "SMask" is "1", so "iAct" is valid. The calculation element 430 corresponding to the third input channel calculates the product of "iAct" = "7" and "Weight" = "-1" as "-7". For the fourth input channel, "SMask" is "1", so "iAct" is valid. The calculation element 430 corresponding to the fourth input channel calculates the product of "iAct" = "1" and "Weight" = "+1" as "+1". The PE vector 420 then calculates the sum (PSUM) of the obtained products "+5", "-7", and "+1" as "-1".

[0076] 12 is a timing chart illustrating the components of the neural network circuit device 100 according to the first embodiment. The numerical values ​​in the timing chart illustrated in FIG. 12 indicate the indexes of the output channels (kernels). At timing T0, the control circuit 110 outputs information on the code lengths of the compressed supermasks for the 16 kernels #0 to #15 (output channels #0 to #15) to the decoder 306 of the mask expansion circuit 300. Between timings T0 and T1, the decoder 306 expands the compressed supermasks for the kernels #0 to #15 and stores the expanded supermasks 20 in the bank 312A of the mask memory 310.

[0077] At timing T1, the control circuit 110 outputs seeds for generating weights 14 for each of the kernels #0 to #15 to the random number generator 202 of the weight generation circuit 200. As a result, the random number generator 202 generates weights 14 for the kernels #0 to #15 and outputs the generated weights 14 to the arithmetic circuit 400. Meanwhile, at timing T1, the control circuit 110 outputs a read signal for reading out the supermasks 20 for each of the kernels #0 to #15 to the bank 312A. As a result, the supermasks 20 for each of the kernels #0 to #15 are output to the arithmetic circuit 400. Therefore, the arithmetic circuit 400 performs arithmetic operations for the kernels #0 to #15 (output channels #0 to #15) between timings T1 and T2.

[0078] Furthermore, at timing T1, the control circuit 110 outputs information on the code lengths of the compressed supermasks for the next 16 kernels #16 to #31 (output channels #16 to #31) to the decoder 306 of the mask expansion circuit 300. Between timings T1 and T2, the decoder 306 expands the compressed supermasks for kernels #0 to #15 and stores the expanded supermasks 20 in bank 312B of the mask memory 310. In this way, the calculations for kernels #0 to #15 (output channels #0 to #15) and the expansion of the compressed supermasks for the next kernels #16 to #31 (output channels #16 to #31) are executed in parallel.

[0079] At timing T2, the control circuit 110 outputs seeds for generating weights 14 for each of kernels #16 to #31 to the random number generator 202 of the weight generation circuit 200. As a result, the random number generator 202 generates weights 14 for kernels #16 to #31 and outputs the generated weights 14 to the arithmetic circuit 400. Meanwhile, at timing T2, the control circuit 110 outputs a read signal for reading out the supermasks 20 for each of kernels #16 to #31 to the bank 312B. As a result, the supermasks 20 for each of kernels #16 to #31 are output to the arithmetic circuit 400. Therefore, the arithmetic circuit 400 performs arithmetic operations for kernels #16 to #31 (output channels #16 to #31) between timings T2 and T3.

[0080] Furthermore, at timing T2, the control circuit 110 outputs information on the code lengths of the compressed supermasks for the next 16 kernels #32 to #47 (output channels #32 to #47) to the decoder 306 of the mask expansion circuit 300. Between timings T2 and T3, the decoder 306 expands the compressed supermasks for kernels #16 to #31 and stores the expanded supermasks 20 in bank 312A of the mask memory 310. In this way, the calculations for kernels #16 to #31 (output channels #16 to #31) and the expansion of the compressed supermask for the next kernels #32 to #47 (output channels #32 to #47) are performed in parallel. Then, the processing for the next 16 kernels #32 to #47 (output channels #32 to #47) and the processing for the following 16 kernels #48 to #63 (output channels #48 to #63) are performed in a similar manner.

[0081] 13 is a diagram illustrating a configuration of a post-processing circuit 140 according to the first embodiment. The post-processing circuit 140 includes a floating-point type converter 141, a batch norm unit 142, a block floating-point type converter 143, an activation function processor 144, and a pooling unit 145. The floating-point type converter 141 (INT2FP) converts integer type PSUM data into floating-point type data. This enables the post-processing circuit 140 to perform floating-point arithmetic.

[0082] The batch normalization unit 142 (BN) performs batch normalization on the data converted to floating-point format. The block floating-point conversion unit 143 (FP2BFP) converts the normalized floating-point data to block floating-point format, thereby reducing the amount of data. The activation function processing unit 144 (ReLU) applies an activation function such as ReLU (Rectified Linear Unit) to the data converted to block floating-point format to activate the data. This provides an output result of the activation function. The pooling unit 145 (MaxPool) performs max pooling on the output result.

[0083] The post-processing circuit 140 stores the obtained data for each output channel (kernel) in the activation value data memory 500. Then, using the data stored in the activation value data memory 500 as input data, the neural network circuit device 100 repeatedly executes the above-described process for the next neural network layer. In this manner, the neural network circuit device 100 processes multiple layers of the neural network. That is, the neural network circuit device 100 performs processing for each layer of the neural network by repeating the loop process shown in FIG. 5, which starts from the activation value data memory 500, passes through the barrel shifter 130, the arithmetic circuit 400, and the post-processing circuit 140, and returns to the activation value data memory 500.

[0084] <Computation result> 14 to 17 are diagrams showing the results of calculations performed by the neural network circuit device 100 according to the first embodiment. FIG. 14 shows that a reduction in external memory access can be achieved when the technique according to the first embodiment is applied. By generating weights 14 inside the neural network circuit device 100 (chip) using the weight generation circuit 200, external memory access can be reduced by 48.0%. Furthermore, by expanding the compressed supermask using the mask expansion circuit 300, external memory access can be reduced by 45.9%. In this way, the neural network circuit device 100 (On-chip Model Construction) according to the first embodiment can reduce external memory access.

[0085] 15 shows that when the technique according to the first embodiment is applied to ResNet50, the amount of data related to the weights 14 stored in the device (chip) can be reduced. By generating the weights 14 by generating random numbers using a different seed for each output channel (Random Seed), the amount of data stored for setting the weights 14 can be reduced by 98.2% compared to when this technique is not used (Binaried Weight). Furthermore, by generating the weights 14 by generating a seed using a hash function that takes the output channel number and the layer number as input (Hashed Seed), the amount of data stored for setting the weights 14 can be reduced by 100%. In this way, the neural network circuit device 100 according to the first embodiment can reduce the amount of data stored for setting the weights 14.

[0086] 16 is a diagram showing that deterioration in accuracy can be suppressed even when the technique according to the first embodiment is applied. Compared to the existing case (PyTorch Default), deterioration in accuracy can be suppressed even when the weights 14 are generated by generating random numbers using different seeds for each output channel (Random Seed). Also, compared to the existing case (PyTorch Default), deterioration in accuracy can be suppressed even when the weights 14 are generated by generating seeds using a hash function that takes the output channel number and the layer number as input (Hashed Seed).

[0087] 17 shows the effect of compressing the supermask 20 using the method according to the first embodiment. FIG. 17 shows the compression ratios obtained when compressing the supermask 20 with k values ​​of 10%, 20%, and 30%. The supermask 20 with k of 10% contains many "0"s and few "1"s. Therefore, in this case, the compression ratio can be increased by compressing using a compression method with a code length of "4 bits" (4-bit encoding). The compression ratio in this case is approximately 0.5, which is a 49.6% improvement (data volume reduction) compared to when the supermask 20 is not compressed.

[0088] Furthermore, the supermask 20 with k set to 20% has fewer "0"s and more "1"s than the supermask 20 with k set to 10%. Therefore, in this case, the compression ratio can be increased by compressing using a compression method with a code length of "3 bits" (3-bit encoding). The compression ratio is approximately 0.75. Furthermore, the supermask 20 with k set to 30% has fewer "0"s and more "1"s than the supermask 20 with k set to 20%. Therefore, in this case, the compression ratio can be increased by compressing using a compression method with a code length of "2 bits" (2-bit encoding). The compression ratio is approximately 0.9. Thus, the sparser the corresponding supermask 20 (the fewer the number of "1"s), the higher the compression ratio of the compressed supermask.

[0089] (Variation) The present invention is not limited to the above-described embodiments, and various modifications can be made without departing from the spirit and scope of the present invention. For example, the neural network circuit device 100 according to the first embodiment described above may not have the mask expansion circuit 300. In other words, the supermask 20 may not be provided in a compressed state. On the other hand, if the supermask 20 is provided in a compressed state and the mask expansion circuit 300 expands the compressed supermask, memory accesses can be reduced, as described above.

[0090] Furthermore, in the above-described embodiment, the number of input channels to be processed is 16, but this is not a limitation. The number of input channels to be processed does not have to be 16. Furthermore, in the above-described embodiment, the number of output channels (kernels) to be processed is 16, but this is not a limitation. The number of output channels (kernels) to be processed does not have to be 16. Furthermore, in the above-described embodiment, the horizontal and vertical sizes of the chunk 520 are 4×4, but this is not a limitation. The horizontal and vertical sizes of the chunk 520 do not have to be 4×4.

[0091] In the above examples, the program includes instructions (or software code) that, when loaded into a computer, cause the computer to perform one or more functions described in the embodiments. The program may be stored on a non-transitory computer-readable medium or a tangible storage medium. By way of example and not limitation, computer-readable medium or tangible storage medium includes random-access memory (RAM), read-only memory (ROM), flash memory, solid-state drive (SSD) or other memory technology, CD-ROM, digital versatile disk (DVD), Blu-ray® disk or other optical disk storage, magnetic cassette, magnetic tape, magnetic disk storage or other magnetic storage device. The program may also be transmitted on a transitory computer-readable medium or communication medium. By way of example and not limitation, transitory computer-readable medium or communication medium includes electrical, optical, acoustic, or other forms of propagated signals. [Explanation of symbols]

[0092] 10 Early Model 12 nodes 13 Connection 14 Weight 20 Super Mask 30 score 40 subnetworks 100 Neural network circuit device 110 control circuit 130 Barrel Shifter 140 Post-processing circuit 141 Floating-point type conversion unit 142 Batch Norm Section 143 Block floating-point type conversion unit 144 Activation function processing section 145 Pooling Department 200 Weight generation circuit 202 Random Number Generator 204 Selector 210 Weight generation unit 220 Kernel Weights 222 Kernel weight part 300 Mask deployment circuit 302 Compressed Data Memory 304 FIFO buffer 306 Decoder 310 Mask Memory 312 Bank 320 Kernel Masks 322 Kernel Mask Part 400 Arithmetic circuit 410 PE Matrix 420 PE Vector 430 arithmetic elements 500 activity data memories 510 Input Data 520 chunks 522 chunk part

Claims

1. A neural network circuit device that realizes a neural network model in which, in a learning stage, weights of each connection between nodes are set by fixed random numbers, a mask is generated by determining whether each of the connections is valid or invalid, and the neural network model is configured using the weights and the mask, a weight generation circuit that generates the weights using a random number generator configured to generate the same random numbers as those generated in the learning stage using the same seed as those used in the learning stage when performing inference; an arithmetic circuit that performs a product-sum operation using input data that is inference target data or activation value data corresponding to the inference target data, the weight, and the mask; and the weight generation circuit generates the random numbers as the weights, the random numbers being generated using a different seed for each output channel; Neural network circuit device.

2. a control circuit that generates the seed using an output channel number and a layer number of a layer that constitutes the neural network model; 2. The neural network circuit device according to claim 1, further comprising:

3. the control circuit generates the seed by a hash function that receives the output channel number and the layer number as input.

3. The neural network circuit device according to claim 2.

4. a mask expansion circuit for expanding the compressed mask when making an inference; and the arithmetic circuit performs a product-sum operation using the mask expanded by the mask expansion circuit; 4. The neural network circuit device according to claim 1.

5. The mask in a compressed state is compressed using a zero run length compression method.

5. The neural network circuit device according to claim 4.

6. The mask in the compressed state is compressed at a higher compression rate as the mask becomes sparser.

6. The neural network circuit device according to claim 5.

7. the calculation circuit performs a calculation such that a product of the input data and the weight is enabled or disabled by the mask.

7. The neural network circuit device according to claim 1.

8. the arithmetic circuit multiplies a logical product of the input data input to a certain connection and a value in the mask indicating validity or invalidity corresponding to the connection by the weight corresponding to the connection; 8. The neural network circuit device according to claim 7.

9. The mask is generated for at least one of the accuracy of inference and the inference target.

9. The neural network circuit device according to claim 1.

Images