Neural network circuit device and fault detection method

Abstract

To provide a neural network circuit device and a fault detection method that enable fault detection while reducing circuit area and power consumption.SOLUTION: A neural network device has a plurality of arithmetic circuits that perform an arithmetic operation corresponding to a specific layer of a plurality of layers constituting a neural network, The neural network device has a first arithmetic circuit, which is an arithmetic circuit that performs an arithmetic operation corresponding to a specific layer, for performing an arithmetic operation using a two-dimensional input feature map of N bits for the specific layer, a second arithmetic circuit, which is an arithmetic circuit that performs arithmetic operations corresponding to a specific layer, for performing an arithmetic operation using a generated two-dimensional input feature map of M bits by quantizing a two-dimensional input feature map for the specific layer, and a fault detection circuit for detecting a fault in the first arithmetic circuit or the second arithmetic circuit on the basis of the difference between the first output of the first arithmetic circuit and the second output of the second arithmetic circuit.SELECTED DRAWING: Figure 3

Description

[Technical field]

[0001] The present invention relates to a neural network circuit device and a fault detection method. [Background technology]

[0002] A circuit that performs calculations of a deep neural network (DNN) (hereinafter, also referred to as a neural network circuit) may perform unintended operations due to, for example, aging. For this reason, a function capable of detecting failures may be implemented in the neural network circuit (see Patent Documents 1 and 2). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2017-120966 A [Patent Document 2] JP 2003-316599 A Summary of the Invention [Problem to be solved by the invention]

[0004] Here, as a method for detecting a fault in the above-mentioned neural network circuit, for example, n-MR (n-modular redundancy) is known, which detects a fault by comparing outputs from the same n circuits.

[0005] However, a neural network circuit in which n-MR is implemented has a larger circuit area and consumes more power than a neural network circuit in which n-MR is not implemented, and therefore, for example, the applications in which the neural network circuit in which n-MR is implemented may be limited.

[0006] SUMMARY OF THE PRESENT EMBODIMENT An object of the present invention is to provide a neural network circuit device and a fault detection method that are capable of detecting faults while suppressing circuit area and power consumption. [Means for solving the problem]

[0007] In order to achieve the above-mentioned object, the neural network circuit device of the present invention is a neural network circuit device having a plurality of arithmetic circuits performing an operation corresponding to a specific layer out of a plurality of layers constituting a neural network, the plurality of arithmetic circuits including a first arithmetic circuit performing an operation corresponding to the specific layer, the first arithmetic circuit performing an operation using an N-bit two-dimensional input feature map for the specific layer, a second arithmetic circuit performing an operation corresponding to the specific layer, the second arithmetic circuit generating an M-bit two-dimensional input feature map by quantizing the two-dimensional input feature map for the specific layer and performing an operation using the generated M-bit two-dimensional input feature map, and a fault detection circuit detecting a fault in the first arithmetic circuit or the second arithmetic circuit based on a difference between a first output of the first arithmetic circuit and a second output of the second arithmetic circuit. Effect of the Invention

[0008] According to the neural network circuit device and the fault detection method of the present invention, it is possible to perform fault detection while suppressing the circuit area and power consumption. [Brief description of the drawings]

[0009] [Figure 1] FIG. 1 is a diagram showing an example of a configuration of an information processing device 1 according to the first embodiment. [Diagram 2] FIG. 2 is a diagram for explaining a specific example of the multiple arithmetic circuits 120 in the first embodiment. [Diagram 3] FIG. 3 is a diagram illustrating an example of the configuration of the AI ​​processor 110 in the first embodiment. [Figure 4] FIG. 4 is a flow chart illustrating the processing performed in the AI ​​processor 110. [Diagram 5] FIG. 5 is a flow chart illustrating the processing performed in the AI ​​processor 110. [Figure 6] FIG. 6 is a diagram for explaining the processing performed in the AI ​​processor 110. [Figure 7] FIG. 7 is a diagram for explaining the processing performed in the AI ​​processor 110. [Figure 8] FIG. 8 is a diagram for explaining the processing performed in the AI ​​processor 110. [Figure 9] FIG. 9 is a diagram for explaining the processing performed in the AI ​​processor 110. [Figure 10] FIG. 10 is a diagram for explaining the processing performed in the AI ​​processor 110. [Figure 11] FIG. 11 is a diagram for explaining the processing performed in the AI ​​processor 110. [Figure 12] FIG. 12 is a diagram illustrating a plurality of arithmetic circuits 120 according to the second embodiment. [Figure 13] FIG. 13 is a diagram illustrating the learning process of the decision tree model. [Figure 14] FIG. 14 is a diagram illustrating the learning process of the decision tree model. [Figure 15] FIG. 15 shows an example of the configuration of the decision tree model T. [Figure 16] Figure 16 shows an example of the configuration of a random forest model R. [Figure 17] FIG. 17 is a diagram illustrating an example of the configuration of the arithmetic circuit 124 in the first embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. However, such descriptions should not be interpreted in a limiting sense and do not limit the subject matter described in the claims. In addition, various changes, substitutions, and modifications can be made without departing from the spirit and scope of the present disclosure. In addition, different embodiments can be appropriately combined.

[0011] [Configuration Example of Information Processing Device 1 in First Embodiment] First, a description will be given of a configuration example of an information processing device 1 in the first embodiment. Fig. 1 is a diagram showing a configuration example of an information processing device 1 in the first embodiment.

[0012] The information processing device 1 in this embodiment is a computer device that performs a process (hereinafter simply referred to as an inference process) of inferring output data from input data by using, for example, a learning model (a deep neural network learning model) that has learned multiple teacher data.

[0013] Specifically, the information processing device 1 includes, for example, a CPU (Central Processing Unit) 101, a memory 102, a communication interface 103, a storage medium 104, and an AI processor 110 (hereinafter also referred to as a neural network circuit device) as shown in Fig. 1. Each part is connected to each other via a bus 105. Note that the information processing device 1 may include, for example, a dedicated circuit (not shown) that performs the same processing as the CPU 101, instead of the CPU 101.

[0014] The storage medium 104 has, for example, a program storage area (not shown) that stores a program (not shown) for performing inference processing.

[0015] The storage medium 104 also has a storage area (not shown) for storing information used in performing the inference process, for example. The storage medium 104 may be, for example, a hard disk drive (HDD) or a solid state drive (SSD).

[0016] The CPU 101 performs inference processing by, for example, executing a program (not shown) loaded from the storage medium 104 to the memory 102. The memory 102 is, for example, a dynamic random access memory (DRAM).

[0017] The AI ​​processor 110 is, for example, an accelerator such as an FPGA (Field Programmable Gate Array), and has a plurality of arithmetic circuits 120 that perform calculations corresponding to each layer that constitutes the learning model.

[0018] The communication interface 103 communicates with the operator terminal 2 via a network NW such as the Internet. The operator terminal 2 is, for example, a PC (Personal Computer), and is a terminal through which an operator (hereinafter simply referred to as an operator) who generates a learning model inputs necessary information.

[0019] Moreover, the AI ​​processor 110 in this embodiment has an arithmetic circuit 120 (hereinafter also referred to as the arithmetic circuit 121 or the first arithmetic circuit 121), which is a circuit that performs an operation corresponding to a specific layer (hereinafter also simply referred to as the specific layer) constituting a learning model, and performs a convolution operation using an N-bit input feature map (a two-dimensional input feature map consisting of N-bit data) for the specific layer. Hereinafter, a case where the specific layer is one convolution layer will be described, but this is not limited to this. Specifically, the specific layer may be, for example, one layer other than a convolution layer (for example, a pooling layer, a batch normalization layer, or a fully connected layer, etc.). Also, the specific layer may correspond to, for example, multiple consecutive layers.

[0020] Specifically, the arithmetic circuit 121 performs a convolution operation using, for example, an N-bit input feature map corresponding to a specific layer and a weight corresponding to the specific layer. The N-bit input feature map corresponding to the specific layer is, for example, an output feature map from a layer preceding the specific layer. The N-bit input feature map corresponding to the specific layer is, for example, an N-bit map generated by quantizing an output feature map from a layer preceding the specific layer. The weight corresponding to the specific layer is, for example, an N-bit weight or a weight of bits other than N bits.

[0021] The AI ​​processor 110 also includes a calculation circuit 120 (hereinafter also referred to as the calculation circuit 122 or the second calculation circuit 122), which is a circuit that performs calculations corresponding to a specific layer and performs convolution calculations using an M-bit input feature map (a two-dimensional input feature map consisting of M-bit data) for the specific layer. M bits is a number of bits smaller than N bits.

[0022] That is, for example, the arithmetic circuit 122 performs the same operation as that performed in the arithmetic circuit 121 by using data with lower bits than the arithmetic circuit 121.

[0023] Specifically, the arithmetic circuit 122 generates an M-bit input feature map by, for example, quantizing an input feature map for a specific layer. The input feature map corresponding to the specific layer is, for example, an output feature map itself from a layer preceding the specific layer. Also, the input feature map corresponding to the specific layer is, for example, a map generated by quantizing an output feature map from a layer preceding the specific layer. Then, the arithmetic circuit 122 generates an L-bit weight by, for example, quantizing a weight for the specific layer. The weight corresponding to the specific layer is, for example, an N-bit weight or a weight of bits other than N bits. Also, the L bits is, for example, the same number of bits as the M bits or a different number of bits from the M bits. After that, the arithmetic circuit 122 performs a convolution operation using, for example, the generated M-bit input feature map and the generated L-bit weight.

[0024] Furthermore, the AI ​​processor 110 has an arithmetic circuit 120 (hereinafter also referred to as a fault detection circuit 123) that detects a fault in the arithmetic circuit 121 or the arithmetic circuit 122 based on, for example, the difference between the output from the arithmetic circuit 121 (hereinafter also referred to as the first output) and the output from the arithmetic circuit 122 (hereinafter also referred to as the second output).

[0025] Specifically, the fault detection circuit 123 judges whether or not the difference (hereinafter also simply referred to as the difference) between the output from the arithmetic circuit 121 and the output from the arithmetic circuit 122 is equal to or greater than a predetermined threshold (hereinafter also referred to as the predetermined threshold). If it is judged that the difference is equal to or greater than the predetermined threshold, the fault detection circuit 123 judges that, for example, the arithmetic circuit 121 or the arithmetic circuit 122 is faulty, and outputs information indicating that a fault may have occurred in the arithmetic circuit 121.

[0026] As a result, the AI ​​processor 110 in this embodiment can detect a fault that has occurred in the arithmetic circuit 121, for example, by using the arithmetic circuit 122, which has a smaller circuit area and power consumption than the arithmetic circuit 121, without having a plurality of circuits identical to the arithmetic circuit 121. Therefore, the AI ​​processor 110 can output information indicating that a fault may have occurred in the arithmetic circuit 121, while suppressing, for example, the circuit area and power consumption of each arithmetic circuit 120. Therefore, the AI ​​processor 110 can be implemented in devices such as edge AI devices that have restrictions on the circuit area, power consumption, and the like.

[0027] The AI ​​processor 110 may have, for example, multiple combinations of an arithmetic circuit 121, an arithmetic circuit 122, and a fault detection circuit 123. That is, the AI ​​processor 110 may have, for example, multiple arithmetic circuits 121 that perform calculations corresponding to different layers, and may further have, for each of the multiple arithmetic circuits 121, an arithmetic circuit 122 and a fault detection circuit 123 for detecting a fault that occurs in each arithmetic circuit 121.

[0028] Furthermore, the CPU 101 may be mounted, for example, in the AI ​​processor 110. Below, specific examples of each arithmetic circuit 120 (arithmetic circuit 121, arithmetic circuit 122, and fault detection circuit 123) will be described.

[0029] [Specific Examples of the Arithmetic Circuits 120 in the First Embodiment] 2 is a diagram for explaining a specific example of each arithmetic circuit 120 in the first embodiment. Below, a case where N bits are 8 bits and M bits are 5 bits will be explained.

[0030] Specifically, the multiple arithmetic circuits 120 shown in FIG. 2 are, for example, arithmetic circuits 120 that perform calculations corresponding to a specific layer L1 that constitutes the neural network NN.

[0031] The multiple arithmetic circuits 120 shown in FIG. 2 include, for example, an arithmetic circuit 121 that performs a convolution operation using an input feature map consisting of 8-bit data, an arithmetic circuit 122 that performs a convolution operation using an input feature map consisting of 5-bit data (for example, data obtained by quantizing 8-bit data), and a fault detection circuit 123 that detects a fault occurring in the arithmetic circuit 121 or the arithmetic circuit 122 depending on the determination result of whether or not the difference between the output from the arithmetic circuit 121 and the output from the arithmetic circuit 122 is equal to or greater than a predetermined threshold value.

[0032] Note that the output (second output) from the arithmetic circuit 122 when 5-bit data is used has a larger error (calculation error) than the output (first output) from the arithmetic circuit 121 when 8-bit data is used, for example. Therefore, in the fault detection circuit 123 in this embodiment, for example, instead of judging whether the first output and the second output match, it judges whether the difference between the first output and the second output is within an allowable range.

[0033] As a result, the AI ​​processor 110 in this embodiment is able to detect faults that occur in the arithmetic circuit 121 or the arithmetic circuit 122, even when, for example, the arithmetic circuit 121 performs a convolution operation on data with a large number of bits, while the arithmetic circuit 122 performs a convolution operation on data with a small number of bits.

[0034] [Example of the configuration of the AI ​​processor 110 in the first embodiment] Next, a description will be given of an example of the configuration of the AI ​​processor 110 in the first embodiment. Fig. 3 is a diagram for explaining an example of the configuration of the AI ​​processor 110 in the first embodiment.

[0035] The AI ​​processor 110 includes, for example, an arithmetic circuit 121, an arithmetic circuit 122, and a fault detection circuit 123, as well as a quantization circuit 122a and an inverse quantization circuit 122b.

[0036] The quantization circuit 122a is, for example, a circuit that quantizes N-bit data included in an input feature map IM for a specific layer L1 into M-bit data. In this case, the calculation circuit 122 performs a convolution calculation using the input feature map IM consisting of M-bit data after quantization by the quantization circuit 122a.

[0037] The inverse quantization circuit 122b is, for example, a circuit that inversely quantizes, into N-bit data, M-bit data included in the output feature map OM (the output feature map OM after the operation is performed in the operation circuit 122) output from the operation circuit 122. In this case, the fault detection circuit 123 detects a fault in the operation circuit 121 or the operation circuit 122 based on, for example, the difference between the output feature map OM output from the operation circuit 121 and the output feature map OM output from the inverse quantization circuit 122b.

[0038] The arithmetic circuit 121 includes, for example, a storage circuit (sliding window unit) 121a and a convolution arithmetic circuit (dot product units) 121b.

[0039] In the following, the number of input feature maps IM input to the arithmetic circuit 121 is C in and the number of output feature maps OM output from the arithmetic circuit 121 is C out In the following description, it is assumed that the number of pixels included in the input feature map IM that are used in the calculation of one pixel included in the output feature map OM is 3×3.

[0040] The memory circuitry 121a sequentially stores data corresponding to pixels included in an input feature map IM input to the arithmetic circuitry 121, for example.

[0041] The convolution circuit 121b is, for example, a C out The convolution circuit 121b has, for example, out When calculating the pixels contained in each of the output feature maps OM, the calculation is 3 × 3 × C in (pieces) of data are obtained from the memory circuit 121a, and C out The same convolution operation is performed in each of the circuits.

[0042] That is, the memory circuit 121a stores C out In order to enable simultaneous access to the data required for the calculation of pixels contained in each of the (sheets) output feature maps OM, in Data corresponding to pixels contained in the (sheets) input feature map IM is stored.

[0043] The configuration of the arithmetic circuit 122 is similar to that of the arithmetic circuit 121, and therefore a description thereof will be omitted.

[0044] [Processing performed in AI processor 110] Next, the processing performed by the AI ​​processor 110 will be described. FIGS. 4 and 5 are flow charts for explaining the processing performed by the AI ​​processor 110. Also, FIGS. 6 to 11 are diagrams for explaining the processing performed by the AI ​​processor 110. in and C out The case where is 1 will be described.

[0045] [Weight memory processing] First, a process (hereinafter, also referred to as a weight storage process) for storing weight data W8 (hereinafter, also referred to as an 8-bit quantized weight W8) corresponding to a specific layer L1 constituting a learning model of a neural network NN will be described. FIG. 4 is a flow chart for explaining the weight storage process.

[0046] As shown in FIG. 4, the AI ​​processor 110 (arithmetic circuit 121) waits until it receives input of weight data W8 (NO in S11), for example.

[0047] Specifically, the AI ​​processor 110 waits until the operator inputs weight data W8 consisting of 8-bit data to the AI ​​processor 110, for example.

[0048] Then, when the input of the weight data W8 is accepted (YES in S11), the AI ​​processor 110 (arithmetic circuit 121) stores, for example, the accepted weight data W8 in a memory circuit (not shown) included in the arithmetic circuit 121 (S12).

[0049] Next, the AI ​​processor 110 (quantization circuit 122a) generates quantized weight data W5 (hereinafter also referred to as 5-bit quantization weight W5) by, for example, quantizing the weight data W8 received in the process of S11 (S13).

[0050] Specifically, the quantization circuit 122a generates quantized weight data W5 consisting of 5-bit data by quantizing the weight data W8 consisting of 8-bit data, for example.

[0051] More specifically, as shown in FIG. 6, the quantization circuit 122a sets, for example, "-9" calculated from "-73" set in W8[2,1] to W5[2,1].

[0052] Then, the AI ​​processor 110 (arithmetic circuit 122) stores, for example, the quantization weight data W5 generated in the process of S13 in a storage circuit (not shown) included in the arithmetic circuit 122 (S14).

[0053] The process of S12 (processing to store weight data W8) and the process of S13 (processing to generate quantized weight data W5 by quantizing weight data W8) may be performed, for example, in another information processing device (not shown) different from the information processing device 1 (AI processor 110). In this case, the AI ​​processor 110 may, for example, receive input of quantized weight data W5 generated in the other information processing device, and store the received input quantized weight data W5 in a memory circuit included in the arithmetic circuit 122.

[0054] [Convolution operation processing] Next, a process of performing a convolution operation corresponding to a specific layer L1 constituting a learning model of the neural network NN (hereinafter, also referred to as a convolution operation process) will be described. FIG. 5 is a flow chart for explaining the convolution operation process.

[0055] As shown in FIG. 5, the AI ​​processor 110 (arithmetic circuit 121) waits until it receives input of, for example, an input feature map X8 (hereinafter also referred to as 8-bit quantized feature map X8) for the specific layer L1 (NO in S21).

[0056] Specifically, the arithmetic circuit 121 receives, for example, image data (not shown) input by an operator, or an input feature map X8 from another arithmetic circuit (not shown) that performs a calculation corresponding to a layer preceding the specific layer L1. Note that the input feature map X8 for the specific layer L1 may be generated, for example, by quantizing an output feature map (not shown) corresponding to a layer preceding the specific layer L1.

[0057] Then, when input of the input feature map X8 is received (YES in S21), the AI ​​processor 110 (quantization circuit 122a) generates a quantized input feature map X5 (hereinafter also referred to as the 5-bit quantized feature map X5) by, for example, quantizing the input feature map X8 that has been received (S22).

[0058] Specifically, the quantization circuit 122a generates a quantized input feature map X5 made up of 5-bit data by performing quantization on an input feature map X8 made up of 8-bit data, for example.

[0059] More specifically, as shown in FIG. 7, the quantization circuit 122a sets, for example, "26" calculated from "199" set in X8[4,3] to X5[4,3].

[0060] Next, the AI ​​processor 110 (arithmetic circuit 122) generates a quantized output feature map O5 (hereinafter also referred to as output feature map O5) by performing a convolution operation, for example, between the quantized input feature map X5 generated in the processing of S22 and the quantized weight data W5 generated in the processing of S13 (S23).

[0061] Specifically, the arithmetic circuit 122 generates the quantized output feature map O5 by performing a convolution operation between, for example, 5-bit data included in the quantized input feature map X5 and 5-bit data included in the quantized weight data W5.

[0062] More specifically, as shown in FIG. 8, the arithmetic circuit 122 sets, for example, “5” in X5[2,0], “6” in X5[2,1], “2” in X5[2,2], “23” in X5[3,0], “29” in X5[3,1], “25” in X5[3,2], “15” in X5[4,0], “25” in X5[4,1], and “15” in X5[4,2]. The calculation circuit 122 calculates "378" by performing a convolution operation with "15" set in W5[0,0], "3" set in W5[0,1], "9" set in W5[0,2], "12" set in W5[1,0], "1" set in W5[1,1], "6" set in W5[1,2], "-10" set in W5[2,0], "-9" set in W5[2,1], and "-11" set in W5[2,2]. Then, the calculation circuit 122 sets the calculated "378" to O5[2,0], for example.

[0063] After that, the AI ​​processor 110 (inverse quantization circuit 122b) generates an output feature map Y5 by, for example, performing inverse quantization on the quantized output feature map O5 generated in the process of S23 (S24).

[0064] Specifically, as shown in FIG. 9, for example, the inverse quantization circuit 122b sets "24572" calculated from "378" set in O5[2,0] to Y5[2,0].

[0065] In addition, the AI ​​processor 110 (arithmetic circuit 121) generates an output feature map Y8 (S25) by performing a convolution operation between the input feature map X8 received in the processing of S21 and the weighting data W8 received in the processing of S11.

[0066] Specifically, the arithmetic circuit 121 generates the quantized output feature map Y8 by performing a convolution operation between, for example, 8-bit data included in the input feature map X8 and 8-bit data included in the weight data W8.

[0067] More specifically, as shown in FIG. 10, the arithmetic circuit 121 calculates, for example, “43” set in X8[2,0], “50” set in X8[2,1], “24” set in X8[2,2], “178” set in X8[3,0], “222” set in X8[3,1], “193” set in X8[3,2], “121” set in X8[4,0], “193” set in X8[4,1], and “132” set in X8[4,2], and W8[0,0]. The calculation circuit 121 performs a convolution operation with "120" set in Y8[0,1], "23" set in W8[0,2], "71" set in W8[1,0], "5" set in W8[1,1], "51" set in W8[1,2], "-80" set in W8[2,0], "-73" set in W8[2,1], and "86" set in W8[2,2], and further adds "5" which is the bias b to calculate "24355". Then, the calculation circuit 121 sets the calculated "24355" to Y8[2,0], for example.

[0068] The AI ​​processor 110 may perform, for example, the calculation of the process of S25 in parallel with the calculation of the processes from S22 to S24. That is, while the quantization circuit 122a, the calculation circuit 122, and the inverse quantization circuit 122b are performing the processes from S22 to S24, the calculation circuit 121 may perform, for example, the process of S25.

[0069] Then, the AI ​​processor 110 (fault detection circuit 123) calculates, for example, the difference between the output feature map Y5 generated in the process of S24 and the output feature map Y8 generated in the process of S25 (S26).

[0070] Specifically, as shown in FIG. 11, the failure detection circuit 123 calculates, as a difference, for example, the average value of the absolute values ​​of the differences between the pixels in the output feature map Y5 and the output feature map Y8.

[0071] Thereafter, the AI ​​processor 110 (fault detection circuit 123) determines, for example, whether the difference calculated in the process of S26 is equal to or greater than a predetermined threshold. The predetermined threshold is, for example, a value located between the distribution of differences when the arithmetic circuit 121 or the arithmetic circuit 122 has failed (hereinafter also referred to as a first distribution) and the distribution of differences when neither the arithmetic circuit 121 nor the arithmetic circuit 122 has failed (hereinafter also referred to as a second distribution).

[0072] As a result, if the difference calculated in the processing of S26 is equal to or greater than a predetermined threshold value (YES in S27), the AI ​​processor 110 outputs information indicating, for example, that a fault may have occurred in the arithmetic circuit 121 (S28).

[0073] That is, in this case, the AI ​​processor 110 determines that a failure has occurred in the arithmetic circuit 121 or the arithmetic circuit 122, for example.

[0074] Specifically, when the difference calculated in the process of S26 is equal to or greater than a predetermined threshold, the AI ​​processor 110 notifies the CPU 101 of information indicating, for example, that a failure may have occurred in the arithmetic circuit 121. In this case, the CPU 101 then stops the AI ​​processor 110 as necessary, for example.

[0075] On the other hand, if the difference calculated in the process of S26 is less than the predetermined threshold value (NO in S27), the AI ​​processor 110 does not perform the process of S28, for example.

[0076] That is, in this case, the AI ​​processor 110 determines that no failure has occurred in the arithmetic circuit 121 and the arithmetic circuit 121, for example.

[0077] In this manner, the AI ​​processor 110 in this embodiment has, for example, a plurality of arithmetic circuits 120 that perform calculations corresponding to specific layers that constitute a neural network.

[0078] Specifically, the multiple arithmetic circuits 120 include, for example, a first arithmetic circuit 121 that is an arithmetic circuit that performs an arithmetic operation corresponding to a specific layer and that performs a convolution operation using an N-bit two-dimensional input feature map IM for the specific layer. The multiple arithmetic circuits 120 also include, for example, a second arithmetic circuit 122 that is an arithmetic circuit that performs an arithmetic operation corresponding to a specific layer and that generates an M-bit two-dimensional input feature map IM by quantizing the N-bit two-dimensional input feature map IM for the specific layer and performs an arithmetic operation using the generated M-bit two-dimensional input feature map IM. The multiple arithmetic circuits 120 also include, for example, a fault detection circuit 123 that detects a fault in the first arithmetic circuit 121 or the second arithmetic circuit 122 based on the difference between a first output of the first arithmetic circuit 121 and a second output of the second arithmetic circuit 122.

[0079] More specifically, when the difference is equal to or greater than a predetermined threshold, the fault detection circuit 123 determines that the first arithmetic circuit 121 or the second arithmetic circuit 122 is faulty, and when the difference is less than the predetermined threshold, it determines that the first arithmetic circuit 121 or the second arithmetic circuit 122 is not faulty. The predetermined threshold is, for example, a value between a first distribution for the difference when the first arithmetic circuit 121 and the second arithmetic circuit 122 are not faulty and a second distribution for the difference when the first arithmetic circuit 121 or the second arithmetic circuit 122 is faulty.

[0080] That is, for example, the arithmetic circuit 122 performs the same operation as that performed in the arithmetic circuit 121 by using data with fewer bits than the arithmetic circuit 121 (data quantized from N bits to M bits). Then, the fault detection circuit 123, for example, determines whether or not the difference between the output from the arithmetic circuit 121 and the output from the arithmetic circuit 122 is equal to or greater than a predetermined threshold. As a result, if it is determined that the difference is not equal to or greater than the predetermined threshold, the fault detection circuit 123 determines, for example, that the arithmetic circuit 121 or the arithmetic circuit 122 is at fault, and outputs information indicating that a fault may have occurred in the arithmetic circuit 121.

[0081] As a result, the AI ​​processor 110 in this embodiment can detect a fault that has occurred in the arithmetic circuit 121 or the arithmetic circuit 122, for example, by using the arithmetic circuit 122, which has a smaller circuit area and power consumption than the arithmetic circuit 121, without having a plurality of circuits identical to the arithmetic circuit 121. Therefore, the AI ​​processor 110 can output information indicating that a fault may have occurred in the arithmetic circuit 121, while suppressing the circuit area and power consumption of each arithmetic circuit 120, for example.

[0082] In addition, in the process of S26, the AI ​​processor 110 may calculate, for example, the degree of matching of each pixel between the output feature map Y5 and the output feature map Y8. Specifically, the AI ​​processor 110 may calculate, as the degree of matching, the ratio of pixels in which the difference in pixel value between the output feature map Y5 and the output feature map Y8 is equal to or less than a threshold. If the degree of matching calculated in the process of S26 is less than another threshold (hereinafter also simply referred to as another threshold), the AI ​​processor 110 may output, for example, information indicating that a failure may occur in the arithmetic circuit 121 in the process of S28. In addition, if the degree of matching calculated in the process of S26 is equal to or more than the other threshold, the AI ​​processor 110 may not perform the process of S28, for example.

[0083] [AI processor 110 in the second embodiment] Next, the AI ​​processor 110 in the second embodiment will be described. Fig. 12 is a diagram for explaining each arithmetic circuit 120 in the second embodiment.

[0084] Specifically, the AI ​​processor 110 in the second embodiment is different from the AI ​​processor 110 in the first embodiment, and has a calculation circuit 120 (hereinafter also referred to as a calculation circuit 124 or a third calculation circuit 124) that performs a comparison calculation and a branching process corresponding to a decision tree model (hereinafter also referred to as a predetermined decision tree model) by using an N-bit input feature map for a specific layer, instead of the calculation circuit 122 that performs a convolution calculation using an M-bit input feature map for a specific layer. That is, the AI ​​processor 110 in the second embodiment has, for example, a calculation circuit 121 that performs a convolution calculation using an N-bit (e.g., 2 bits, etc.) input feature map for a specific layer, and a calculation circuit 124 that performs a comparison calculation and a branching process corresponding to a predetermined decision tree model by using an N-bit input feature map for a specific layer. Hereinafter, the process (hereinafter also simply referred to as a learning process) when learning (machine learning) about a decision tree model will be described.

[0085] [Specific example of learning process of decision tree model] 13 to 16 are diagrams for explaining the learning process of the decision tree model. In the following, a case where the learning process is performed in an information processing device 11 different from the information processing device 1 will be described.

[0086] As shown in FIG. 13, the information processing device 11 realizes various functions including an information management unit 111, a data generation unit 112, a first model generation unit 113, a map identification unit 114, a second model generation unit 115, a model replacement unit 116, and a model output unit 117 as functions for executing learning processing.

[0087] The information management unit 111 stores, for example, various pieces of information input by the worker via the worker terminal 2 in a storage medium (not shown). Specifically, for example, when the information management unit 111 receives input of the learning model M shown in Fig. 14, the information management unit 111 stores the received learning model M in the storage medium. Also, for example, when the information management unit 111 receives input of image data D0 shown in Fig. 14, the information management unit 111 stores the received image data D0 in the storage medium.

[0088] The data generating unit 112 generates, for example, the teacher data DT shown in FIG.

[0089] 14, for example, in response to input of image data D0 to the learning model M, the data generation unit 112 acquires a plurality of input feature maps IM input to a designated layer L2 constituting the learning model M, and an output feature map OM output from the designated layer L2. The designated layer L2 is, for example, a layer designated in advance by an operator, and may be composed of a plurality of consecutive layers.

[0090] Then, for example, for each output feature map OM, the data generating unit 112 generates teacher data DT from each output feature map OM and a plurality of input feature maps IM corresponding to each output feature map OM.

[0091] More specifically, as shown in Fig. 14, the data generating unit 112 generates the teacher data DT so that, for example, a pixel OPa (e.g., data indicating "1" or "0") in the output feature map OM and a plurality of pixels IPa in a plurality of input feature maps IM used in the calculation of the pixel OPa are included. The data generating unit 112 also generates the teacher data DT so that, for example, a pixel OPb in the output feature map OM and a plurality of pixels IPb in a plurality of input feature maps IM used in the calculation of the pixel OPb are included. Hereinafter, the pixels in the output feature map OM including the pixels OPa and OPb are collectively referred to as pixels OP.

[0092] That is, the data generation unit 112 generates the teacher data DT so that, for example, for each pixel OP in the output feature map OM, the teacher data DT includes each pixel OP and multiple pixels IP in multiple input feature maps IM used in the calculation of each pixel OP.

[0093] The first model generation unit 113 generates a decision tree model T or a random forest model R consisting of one or more decision tree models T, for example, by performing learning (machine learning) of teacher data DT corresponding to each output feature map OM output from the specified layer L2. Hereinafter, the decision tree model T and the random forest model R are collectively referred to simply as the decision tree model T.

[0094] Specifically, the first model generation unit 113 generates a decision tree model T, etc., capable of outputting (predicting) one pixel OP included in one output feature map OM from 16 pixels IP (4 × 4 pixels IP) included in each of three input feature maps IM, as shown in Figure 14.

[0095] The map specifying unit 114 specifies, for example, an output feature map OM whose determination accuracy of the decision tree model T or the like satisfies a first condition, from among the output feature maps OM output from the specified layer L2.

[0096] The second model generation unit 115 updates the learning model M to generate a learning model M1, for example, so that the output of a specific output feature map OM in the specified layer L2 is performed by using a decision tree model T or the like.

[0097] The model replacement unit 116, for example, calculates the determination accuracy of the learning model M1 generated by the second model generation unit 115. Then, the model replacement unit 116, for example, determines whether the calculated determination accuracy satisfies the second condition. As a result, when it is determined that the calculated determination accuracy satisfies the second condition, the model replacement unit 116 replaces the learning model M1 with the learning model M, for example.

[0098] The model output unit 117 outputs, for example, the learning model M on which the model replacement unit 116 has performed replacement. Specifically, the model output unit 117 outputs, for example, the learning model M on which the model replacement unit 116 has performed replacement to the operator terminal 2. Specific examples of the decision tree model T and the random forest model R will be described below.

[0099] [Example of decision tree model T] 15 is a configuration example of a decision tree model T. In the following, the decision tree model T will be described as a learning model that outputs one pixel OP from 16 pixels IP (4×4 pixels IP) included in each of four input feature maps IM.

[0100] The decision tree model T shown in FIG. 15 is a learning model that outputs an inference result for a pixel OP in response to input of, for example, a pixel corresponding to X[2,0,0] (the third pixel from the left and the first pixel from the top among the multiple pixels IP included in the first input feature map IM), a pixel corresponding to X[0,3,1] (the first pixel from the left and the fourth pixel from the top among the multiple pixels IP included in the second input feature map IM), and a pixel corresponding to X[3,2,0] (the fourth pixel from the left and the third pixel from the top among the multiple pixels IP included in the first input feature map IM).

[0101] Specifically, the decision tree model T shown in FIG. 15 predicts that the pixel OP is "0" when it is determined that the pixel corresponding to X[2,0,0] is not greater than "0" and the pixel corresponding to X[0,3,1] is not greater than "0". Also, the decision tree model T shown in FIG. 15 predicts that the pixel OP is "1" when it is determined that the pixel corresponding to X[2,0,0] is not greater than "0" and the pixel corresponding to X[0,3,1] is greater than "0". Also, the decision tree model T shown in FIG. 15 predicts that the pixel OP is "1" when it is determined that the pixel corresponding to X[2,0,0] is greater than "0" and the pixel corresponding to X[3,2,0] is not greater than "0". Furthermore, if the decision tree model T shown in FIG. 15 determines that the pixel corresponding to X[2,0,0] is greater than “0” and the pixel corresponding to X[3,2,0] is greater than “0”, then the decision tree model T predicts that the pixel OP is “0”.

[0102] Therefore, if the pixel corresponding to X[2,0,0] is “0” and the pixel corresponding to X[0,3,1] is “1”, the decision tree model T outputs, for example, “1” as the inference result for pixel OP, as shown in FIG.

[0103] Here, the depth D of the decision tree model T shown in Fig. 15 is "3". Therefore, in the example shown in Fig. 15, when inferring the pixel OP by the decision tree model T, the number of operations is only "2 (times)" of comparison operations and branch operations.

[0104] That is, the information processing device 11 can significantly reduce the number of calculations when inferring the pixel OP using the decision tree model T, as compared to when inferring the pixel OP using a convolution calculation.

[0105] Therefore, in the AI ​​processor 110 of the second embodiment, for example, when the specific layer L1 and the specified layer L2 match, it is possible to reduce the circuit area and power consumption of the calculation circuit 124 by having the calculation circuit 124 execute operations (comparison operations and branch operations) corresponding to the decision tree model T.

[0106] [Example of random forest model R configuration] Figure 16 shows an example of the configuration of a random forest model R.

[0107] 16 is a learning model configured with a plurality of decision tree models T, and performs inference processing by majority vote of output results from the plurality of decision tree models T. Specifically, the random forest model R shown in Fig. 16 performs inference processing by majority vote of output results from each of three decision tree models T (decision tree model Ta, decision tree model Tb, and decision tree model Tc), for example.

[0108] This enables the information processing device 11 to improve the determination accuracy when recognizing the image data D0 compared to the case where only the decision tree model T described in Fig. 15 is used. Specifically, the information processing device 11 can improve, for example, the determination accuracy for unknown image data D0 (hereinafter also referred to as generalization performance) compared to the case where only the decision tree model T described in Fig. 15 is used.

[0109] Therefore, in the AI ​​processor 110 of the second embodiment, for example, when the specific layer L1 and the specified layer L2 match, the calculation circuit 124 is caused to execute operations (comparison operations and branch operations) corresponding to the decision tree model T, thereby making it possible to improve the generalization performance while reducing the circuit area and power consumption of the calculation circuit 124.

[0110] [Example of configuration of arithmetic circuit 124 in the second embodiment] Next, a specific example of the arithmetic circuit 124 will be described. Fig. 17 is a diagram showing a configuration example of the arithmetic circuit 124 in the second embodiment. Specifically, Fig. 17 is a diagram showing a configuration example of the arithmetic circuit 124 when performing a calculation corresponding to a random forest model R.

[0111] The arithmetic circuit 124 includes, for example, a memory circuit 124a and an RF arithmetic circuit 124b, as shown in Fig. 17. Hereinafter, as shown in Fig. 17, the number of input feature maps IM input to the arithmetic circuit 124 is C in and the number of output feature maps OM output from the arithmetic circuit 121 is C out In the following description, it is assumed that the number of pixels IP included in the input feature map IM that are used in the calculation of one pixel OP included in the output feature map OM is R×R.

[0112] The memory circuitry 124a is, for example, a sliding window, and sequentially stores data corresponding to pixels included in a plurality of input feature maps IM input to the random forest model R. Specifically, the memory circuitry 124a sequentially stores data, for example, starting from data corresponding to pixels used in the calculation in the random forest model R.

[0113] The RF arithmetic circuit 124b is, for example, out (units) of random forest model circuits RTU. Each of the random forest model circuits RTU has, for example, T n It has (pieces) of decision tree model circuits DTU.

[0114] Then, the RF calculation circuit 124b calculates C out When performing calculations for pixels OP included in each of the (sheets) output feature maps OM, data corresponding to the pixel OP among the data stored in the memory circuit 124a (R×R×C in (data of each item) and C out The pixel OP is calculated in each of the random forest model circuits RFU. out In this case, each of the (number of) random forest model circuits RFU performs a comparison operation and a branching operation in each decision tree model circuit DTU constituting each random forest model circuit RFU, thereby calculating the pixel OP corresponding to each random forest model circuit RTU.

[0115] That is, the memory circuit 124a stores C out In order to simultaneously access the data required for the calculation of pixels OP contained in each of the (sheets) output feature maps OM, in Data corresponding to pixels contained in the (sheets) input feature map IM is stored.

[0116] In this way, in the arithmetic circuit 124, unlike the arithmetic circuit 121, for example, instead of a multiplier, an adder, a product-sum calculator, etc. (hereinafter also referred to as a multiplier, etc.), a comparison calculator smaller than a multiplier, etc. is mounted, thereby making it possible to realize a circuit with a smaller area than when a multiplier, etc. is mounted. Therefore, in the arithmetic circuit 124, for example, it is possible to improve the performance density. [Explanation of symbols]

[0117] 1: Information processing device 2: Worker terminal 11: Information processing device 101:CPU 102: Memory 103: Communication interface 104:Storage medium 105: Bus 110: AI processor 120: Arithmetic circuit 121: Arithmetic circuit 121a: Memory circuit 121b: Convolution circuit 122: Arithmetic circuit 122a: Quantization circuit 122b: Inverse quantization circuit 123: Fault detection circuit 124: Arithmetic circuit 124a: Memory circuit 124b:RF calculation circuit

Claims

1. A neural network circuit device having a plurality of arithmetic circuits for performing an operation corresponding to a specific layer among a plurality of layers constituting a neural network, The plurality of arithmetic circuits include a first arithmetic circuit that performs an operation corresponding to the specific layer and that performs an operation using an N-bit two-dimensional input feature map for the specific layer; a second arithmetic circuit that performs an operation corresponding to the specific layer, the second arithmetic circuit generating an M-bit two-dimensional input feature map by quantizing a two-dimensional input feature map for the specific layer, and performing an operation using the generated M-bit two-dimensional input feature map; a fault detection circuit that detects a fault in the first arithmetic circuit or the second arithmetic circuit based on a difference between a first output of the first arithmetic circuit and a second output of the second arithmetic circuit.

2. In claim 1, the first arithmetic circuit performs an arithmetic operation using an N-bit two-dimensional input feature map corresponding to the specific layer and a weight corresponding to the specific layer; The second calculation circuit generates L-bit weights by quantizing weights for the specific layer, and performs calculations using the generated M-bit two-dimensional input feature map and the L-bit weights, in a neural network circuit device.

3. In claim 1, The fault detection circuit determines that the first arithmetic circuit or the second arithmetic circuit is faulty when the difference is equal to or greater than a predetermined threshold, and determines that the first arithmetic circuit and the second arithmetic circuit are not faulty when the difference is less than the predetermined threshold.

4. In claim 3, the predetermined threshold is a value between a first distribution for the difference when the first arithmetic circuit and the second arithmetic circuit are not faulty and a second distribution for the difference when the first arithmetic circuit or the second arithmetic circuit is faulty.

5. A neural network circuit device having a plurality of arithmetic circuits for performing an operation corresponding to a specific layer among a plurality of layers constituting a neural network, The plurality of arithmetic circuits include a first arithmetic circuit that performs an operation corresponding to the specific layer and that performs an operation using an N-bit two-dimensional input feature map for the specific layer; a third arithmetic circuit for performing an operation corresponding to the specific layer, the third arithmetic circuit performing a comparison operation and a branch operation corresponding to a predetermined decision tree model by using the N-bit two-dimensional input feature map for the specific layer; a fault detection circuit that detects a fault in the first arithmetic circuit or the third arithmetic circuit based on a difference between a first output of the first arithmetic circuit and a third output of the third arithmetic circuit.

6. A method for detecting a fault in a plurality of arithmetic circuits that perform an operation corresponding to a specific layer among a plurality of layers constituting a neural network, comprising the steps of: a first arithmetic circuit included in the plurality of arithmetic circuits performs an arithmetic operation using an N-bit two-dimensional input feature map for the specific layer; a second arithmetic circuit included in the plurality of arithmetic circuits generates an M-bit two-dimensional input feature map by quantizing a two-dimensional input feature map for the specific layer, and performs an operation using the generated M-bit two-dimensional input feature map; a fault detection circuit included in the plurality of arithmetic circuits detects a fault in the first arithmetic circuit or the second arithmetic circuit based on a difference between a first output of the first arithmetic circuit and a second output of the second arithmetic circuit.

7. A method for detecting a fault in a plurality of arithmetic circuits that perform an operation corresponding to a specific layer among a plurality of layers constituting a neural network, comprising the steps of: a first arithmetic circuit included in the plurality of arithmetic circuits performs an arithmetic operation using an N-bit two-dimensional input feature map for the specific layer; a third arithmetic circuit included in the plurality of arithmetic circuits performs a comparison operation and a branch operation corresponding to a predetermined decision tree model by using the N-bit two-dimensional input feature map for the specific layer; a fault detection circuit included in the plurality of arithmetic circuits detects a fault in the first arithmetic circuit or the third arithmetic circuit based on a difference between a first output of the first arithmetic circuit and a third output of the third arithmetic circuit.

Images