Computing systems and methods for identifying images

Abstract

Disclosed are computing systems and methods for identifying images. The computing systems include a neural network system configured to obtain an input image, drive an artificial neural network (ANN) with the input image as an input sample of the ANN to obtain an image identification result, and a quantization system configured to quantize the ANN.

Description

[0001] This application claims the benefit of Korean Patent Application No. 10-2019-0117485, filed on September 24, 2019, with the Korean Intellectual Property Office, and Korean Patent Application No. 10-2020-0029807, filed on March 10, 2020, the disclosure of which is incorporated herein by reference in its entirety. Technical Field

[0002] The inventive concept relates to a quantization method for an artificial neural network and an operational method using the artificial neural network, and more specifically, to a computational system and method for recognizing images. Background Technology

[0003] Image recognition can be achieved using artificial neural networks. An artificial neural network can represent a computing device or a method executed by a computing device to generate an interconnected set of artificial neurons (or neuron models). An artificial neuron can generate output data by performing simple operations on input data, and the output data can be passed to another artificial neuron. As an example of an artificial neural network, a deep neural network, or deep learning, can have a multi-layered structure. Summary of the Invention

[0004] The inventive concept provides a computational system and method for image recognition.

[0005] According to one aspect of the inventive concept, a computing system is provided, the computing system comprising: a neural network system configured to acquire an input image, using the input image as input samples of an artificial neural network (ANN) to drive the ANN to obtain an image recognition result, wherein the step of driving the ANN includes: for each layer of at least one layer in the ANN, generating output samples of the layer of the ANN based on input samples of the layer of the ANN, quantized parameters, and a compensation bias; and a quantization system configured to quantize the ANN, wherein the quantization system is further configured to: for each layer of the at least one layer in the ANN, quantize the parameters of the layer of the ANN to generate quantized parameters of the layer of the ANN, generate a quantization error of the parameters of the layer of the ANN based on the parameters of the layer of the ANN and the quantized parameters, generate a compensation bias based on the quantized parameters of the layer of the ANN and the quantization error of the parameters, and send the generated quantized parameters and compensation bias to the neural network system.

[0006] According to another aspect of the inventive concept, a method for recognizing an image is provided, the method comprising: obtaining an input image as an input sample of an artificial neural network (ANN); and performing the ANN based on the input sample to obtain an image recognition result, wherein the step of obtaining the image recognition result comprises: quantizing the weights and biases of the ANN for each layer of at least one layer in the ANN; generating a compensation bias by compensating the quantized bias to include errors caused by quantization; quantizing the input samples of the ANN layer; performing a first multiply-accumulate (MAC) operation based on the quantized weights and quantized input samples of the ANN layer; and reflecting the compensation bias in the result of the first MAC operation to generate an output sample of the ANN layer.

[0007] According to another aspect of the inventive concept, a method for recognizing an image is provided, the method comprising: obtaining an input image as an input sample of an artificial neural network (ANN); performing the ANN based on the input sample to obtain an image recognition result, wherein the step of obtaining the image recognition result comprises: quantizing the parameters of the ANN layer for each layer of at least one layer in the ANN; calculating a quantization error of the parameters based on the parameters of the ANN layer and the quantized parameters; generating a compensation bias based on the quantized parameters and the quantization error of the parameters; quantizing the input sample of the ANN layer; and generating an output sample of the ANN layer based on the quantized parameters, the compensation bias, and the quantized input sample.

[0008] According to one aspect of the inventive concept, a computing system is provided, the computing system comprising: a neural network system configured to drive an artificial neural network (ANN); and a quantization system configured to quantize the ANN, wherein the quantization system is further configured to: quantize the parameters of the ANN to generate quantized parameters of the ANN, generate a quantization error of the ANN parameters based on the ANN parameters and the quantized parameters, generate a compensation bias based on the quantized parameters and the quantization error of the ANN parameters, and send the generated quantized parameters and the compensation bias to the neural network system.

[0009] According to another aspect of the inventive concept, an operating method using an artificial neural network (ANN) is provided, the operating method comprising: quantizing the weights and biases of the ANN; generating a compensated bias by compensating the quantized bias to include errors caused by quantization; quantizing an input sample; performing a first multiply-accumulate (MAC) operation based on the quantized weights and the quantized input sample of the ANN; and reflecting the compensated bias in the result of the first MAC operation.

[0010] According to another aspect of the inventive concept, a quantization method for an artificial neural network (ANN) is provided, the quantization method comprising: quantizing the parameters of the ANN; calculating the quantization error of the parameters based on the parameters of the ANN and the quantized parameters; and generating a compensation bias based on the quantized parameters and the quantization error of the parameters. Attached Figure Description

[0011] From the following detailed description taken in conjunction with the accompanying drawings, exemplary embodiments of the inventive concept will become clearer, wherein:

[0012] Figure 1 This is a diagram illustrating an artificial neural network according to an example embodiment;

[0013] Figure 2 This is a diagram illustrating a computing system according to an example embodiment;

[0014] Figure 3 This is a flowchart illustrating the operation of a neural network system, a parameter quantizer, a sample quantizer, and a bias compensator according to an example embodiment;

[0015] Figure 4 It is a diagram used to explain the architecture of the computation graph according to the example embodiment;

[0016] Figure 5 This is a diagram illustrating the operation of reflecting the expected value of the quantization error of a neural network system into the quantization operation according to an example embodiment;

[0017] Figure 6 This is a diagram illustrating a method for generating a compensated bias according to an example embodiment;

[0018] Figure 7 This is a diagram illustrating a method for generating a compensated bias according to an example embodiment;

[0019] Figure 8 This is a diagram illustrating the operation of reflecting the expected value of the quantization error of a neural network system into the quantization operation according to an example embodiment;

[0020] Figure 9 This is a flowchart illustrating a method for determining a reference sample, a quantized reference sample, and a quantization error of the reference sample according to an example embodiment;

[0021] Figure 10 This is a diagram illustrating a computing system according to an example embodiment;

[0022] Figure 11 This is a diagram illustrating a method for predicting the next input sample according to an example embodiment;

[0023] Figure 12 This is a diagram illustrating a method for predicting the next input sample according to an example embodiment;

[0024] Figure 13 This is a flowchart illustrating a method for determining a reference sample, a quantized reference sample, and a quantization error of the reference sample according to an example embodiment;

[0025] Figure 14 This is a diagram illustrating a method for performing operations on the input samples of the next order using a quantized artificial neural network (ANN) according to an example embodiment;

[0026] Figure 15 This is a flowchart illustrating an operational method using an ANN according to an example embodiment;

[0027] Figure 16 This is a block diagram illustrating an electronic device according to an example embodiment; and

[0028] Figure 17 This is a block diagram illustrating an electronic device according to an example embodiment. Detailed Implementation

[0029] Figure 1 This is a diagram illustrating an artificial neural network (ANN) according to an example embodiment. For example, Figure 1 This is a schematic diagram illustrating the structure of a deep neural network 10, which is an example of an ANN according to an exemplary embodiment.

[0030] An ANN can represent a computational system focused on the biological neural networks that constitute an animal's brain. Unlike classical algorithms (such as rule-based programming) that perform tasks according to predefined conditions, an ANN can be trained to perform tasks by considering multiple samples (or examples). An ANN may have a structure in which artificial neurons (or neurons) are connected, and the connections between neurons may be called synapses. A neuron processes the received signal and sends the processed signal to another neuron through a synapse. The output of a neuron may be called an "activation". Neurons and / or synapses may have variable weights, and the influence of the signal processed by the neuron may increase or decrease according to the weights. In particular, the weights associated with a single neuron may be called biases.

[0031] Deep neural networks (DNNs), or deep learning architectures, can have a layered structure, and the output of a particular layer can be the input to subsequent layers. In such a multi-layered structure, each layer can be trained on multiple samples. Artificial neural networks (such as DNNs) can be implemented using multiple processing nodes, each corresponding to an artificial neuron. This may require higher computational complexity to achieve good results (such as higher accuracy), and therefore a significant amount of computational resources may be needed.

[0032] Reference Figure 1The DNN 10 may include multiple layers L1, L2, L3…LN (e.g., N can be an integer greater than 3), and the output of a layer can be input to subsequent layers through at least one channel. For example, the first layer L1 can provide the output of the sample SAM to the second layer L2 through multiple channels CH11…CH1x, and the second layer L2 can also provide the output of the third layer L3 through multiple channels CH21…CH2y. Finally, the Nth layer LN can output the result RES, and the result RES may include at least one value associated with the sample SAM. The number of channels through which the outputs of multiple layers L1, L2, L3…LN are transmitted can be the same or different. For example, the number of channels CH21…CH2y in the second layer L2 and the number of channels CH31…CH3z in the third layer L3 can be the same or different. For example, x, y, and z can be integers greater than 1.

[0033] The sample SAM can be input data processed by the DNN 10. For example, the sample SAM can be an image containing human-written letters, and the DNN 10 can output a result RES containing values ​​representing the letters by recognizing the letters from the images. The result RES can include multiple probabilities corresponding to different letters, and the letter with the greatest influence among the different letters can correspond to the highest probability. Each of the multiple layers L1, L2, L3…LN of the DNN 10 can process the sample SAM and the output of the previous layer based on values ​​(such as weights, biases, etc.) learned from multiple images containing letters to generate its own output.

[0034] According to the example embodiment, the DNN 10 may include a large number of layers or channels, and therefore, the computational complexity of the DNN 10 can increase. A DNN 10 with high computational complexity may require a large amount of resources. Therefore, in order to reduce the computational complexity of the DNN 10, the DNN 10 may be quantized. Quantization of the DNN 10 may represent the process of mapping input values ​​to a number of values ​​smaller than the number of input values ​​(e.g., mapping real numbers to integers through rounding). A quantized DNN 10 may have low computational complexity, but may have reduced accuracy due to errors occurring in the quantization process.

[0035] As described below with reference to the accompanying drawings, the quantized DNN 10 according to the example embodiment can reflect the expected value of the error that occurs in the quantization process in the computation result. Therefore, the DNN 10 can have improved performance and / or reduced complexity.

[0036] Figure 2 This is a diagram illustrating a computing system (e.g., a computing system for recognizing images) 1000 according to an example embodiment.

[0037] Reference Figure 2The computing system 1000 may include a quantization system 100 and a neural network system 200. The neural network system 200 provides an ANN (Application Not Responding) network, and the quantization system 100 quantizes the ANN provided from the neural network system 200, and provides at least a partially quantized ANN to the neural network system 200. Figure 1 In this embodiment, the neural network system 200 and the quantization system 100 are separate from each other, but according to the example embodiment, the neural network system 200 and the quantization system 100 can be implemented as a single system.

[0038] The neural network system 200 can be any system that provides (or drives) an ANN, and may also be referred to as a neural network device. For example, the neural network system 200 can be implemented as processing circuitry (e.g., a computing system including at least one processor and memory). As a non-limiting example, the neural network system 200 can be a fixed computing system (such as a desktop computer, server, etc.) and a mobile computing system (such as a laptop computer, smartphone, etc.). In an example embodiment, the neural network system can obtain an input image, using the input image as input samples to drive the ANN to obtain image recognition results. In an example embodiment, the first layer of the ANN can obtain an input image as input samples for the first layer, and the output samples of the previous layer of the ANN can be input samples of the next layer of the ANN.

[0039] In one example embodiment, the neural network system 200 may drive an ANN and provide information about the ANN to the quantization system 100. In another example embodiment, the neural network system 200 may drive the ANN based on information provided from the quantization system 100 and provide information about the driven ANN to the quantization system 100.

[0040] The quantization system 100 can be any system that performs the quantization of the ANN and may also be referred to as a quantization device. For example, the quantization system 100 can be implemented as a processing circuit (e.g., a computing system including at least one processor and memory). The quantization system 100 can be a fixed computing system or a mobile computing system. The quantization system 100 can quantize the ANN based on information provided by the neural network system 200.

[0041] Reference Figure 2The quantization system 100 may include a neural network interface 110, a parameter quantizer 120, a sample quantizer 130, and / or a bias compensator 140. In one example embodiment, each of the neural network interface 110, parameter quantizer 120, sample quantizer 130, and bias compensator 140 may be implemented as, for example, a processing circuit as a logic block implemented by logic synthesis, a software block executed by a processor, or a combination thereof. In one example embodiment, each of the neural network interface 110, parameter quantizer 120, sample quantizer 130, and bias compensator 140 may be a process as a set of multiple instructions executed by a processor and may be stored in accessible memory by the processor.

[0042] The neural network interface 110 can provide an interface to the neural network system 200 to the parameter quantizer 120 and the sample quantizer 130. For example, the neural network interface 110 can provide the parameters of the ANN received from the neural network system 200 to the parameter quantizer 120, and provide the quantized parameters received from the parameter quantizer 120 to the neural network system 200. Furthermore, the neural network interface 110 can provide the samples received from the neural network system 200 to the sample quantizer 130, and provide the quantized samples received from the sample quantizer 130 to the neural network system 200. Additionally, the neural network interface 110 can provide the compensation bias received from the bias compensator 140 to the neural network system 200.

[0043] The parameter quantizer 120 can generate quantized parameters from parameters received from the neural network system 200 via the neural network interface 110. For example, the parameter quantizer 120 can receive weights and biases as parameters of the ANN and generate quantized weights and quantized biases. The parameter quantizer 120 can provide the quantized weights to the neural network system 200 via the neural network interface 110.

[0044] Furthermore, the parameter quantizer 120 can use the quantized weights and quantized biases to generate quantization errors for the weights and biases. The quantization error for the weights can include errors that occur during the quantization process of the ANN's weights. By using the property that the unquantized weights are equal to the sum of the quantized weights and the quantization errors of the weights, the parameter quantizer 120 can generate the quantization errors for the weights from the received weights and the quantized weights. Similarly, the quantization error for the bias can include errors that occur during the quantization process of the ANN's biases. By using the property that the unquantized bias is equal to the sum of the quantized bias and the quantization errors of the biases, the parameter quantizer 120 can generate the quantization error for the biases from the received biases and the quantized biases.

[0045] Furthermore, the parameter quantizer 120 can provide information for generating a compensation bias to compensate for errors caused by quantization to the bias compensator 140. For example, the parameter quantizer 120 can provide the quantization weights, quantization bias, quantization error of the weights, and quantization error of the bias to the bias compensator 140.

[0046] The sample quantizer 130 can generate quantized samples from samples received from the neural network system 200 via the neural network interface 110. For example, the sample quantizer 130 can receive multiple images and generate quantized images. The sample quantizer 130 can provide the quantized samples to the neural network system 200 via the neural network interface 110. Because parameters and samples have different characteristics in an ANN, parameter quantization and sample quantization can be separated.

[0047] The bias compensator 140 can use information received from the parameter quantizer 120 to generate a compensating bias to compensate for errors caused by quantization. In one example embodiment, the bias compensator 140 can generate a compensating bias by compensating the quantized bias to include errors caused by quantization. Reference will be made below. Figure 6 and Figure 7 An example is provided to illustrate the operation of bias compensator 140 in generating a compensation bias. Bias compensator 140 can provide the generated compensation bias to neural network system 200 via neural network interface 110.

[0048] The neural network system 200 can perform a multiply-accumulate (MAC) operation based on the quantized weights received from the parameter quantizer 120 and the quantized samples received from the sample quantizer 130. Furthermore, the neural network system 200 can generate the final computation result by reflecting the compensation bias received from the bias compensator 140 into the MAC operation result.

[0049] Figure 3 This is a flowchart illustrating the operation of a neural network system 200, a parameter quantizer 120, a sample quantizer 130, and a bias compensator 140 according to an example embodiment. For example, Figure 3 It is shown Figure 2 The flowchart describes the quantization operation of the ANN in the neural network system 200, parameter quantizer 120, sample quantizer 130, and bias compensator 140, as well as the operation of the quantized ANN.

[0050] Reference Figure 2 and Figure 3The neural network system 200 may include an ANN (S100). Furthermore, the neural network system 200 may provide weights W and biases as parameters of the ANN along with a quantization request to the parameter quantizer 120 (S105). Furthermore, the parameter quantizer 120 may quantize the received weights W and biases (S110). Furthermore, the parameter quantizer 120 may provide the quantized weights q_W to the neural network system 200 (S115). The neural network system 200 may store the received quantized weights q_W. Furthermore, the parameter quantizer 120 may provide the quantized weights q_W, the quantized bias q_bias, the quantization error e_W of the weights, and the quantization error e_bias of the bias to the bias compensator 140 (S120). The order of operations S115 and S120 may be changed.

[0051] Furthermore, the bias compensator 140 can generate a compensation bias (S125) based on the received information to compensate for errors caused by quantization. For example, the bias compensator 140 can generate the compensation bias based on the received quantization weight q_W, quantization bias q_bias, quantization error e_W of the weight, and quantization error e_bias of the bias. The following will refer to... Figure 6 and Figure 7 An example is described of the operation of the bias compensator 140 in generating a compensation bias. Furthermore, the bias compensator 140 can provide the generated compensation bias q_bias1 to the neural network system 200 (S130). The neural network system 200 can store the received compensation bias q_bias1.

[0052] The neural network system 200 can receive a new sample X (S135). Then, the neural network system 200 can provide the sample X along with a quantization request to the sample quantizer 130 (S140). Furthermore, the sample quantizer 130 can quantize the received sample X (S145). Additionally, the sample quantizer 130 can provide the quantized sample q_X to the neural network system 200 (S150). The neural network system 200 can perform a MAC operation based on the quantized weights q_W and the quantized sample q_X (S155). Furthermore, the neural network system 200 can generate a final operation result by reflecting a compensation bias q_bias1 in the MAC operation result (S160).

[0053] Figure 4 This is a diagram used to explain the architecture of computational diagram 20 according to an example embodiment.

[0054] Reference Figure 4 Computational graph 20 is a graph illustrating a mathematical model represented using nodes and edges. The architecture of computational graph 20 may correspond to the architecture of an ANN or a quantized ANN. For example, an ANN or a quantized ANN may be implemented as a convolutional neural network (CNN), but the inventive concept is not limited thereto. Figure 4 When a quantized ANN represents a CNN, computation graph 20 can correspond to some layers of the CNN. For example, computation graph 20 can correspond to a layer in the CNN that performs MAC operations (such as a convolutional layer, a fully connected layer, etc.). In the following text, for ease of description, computation graph 20 will be given as a convolutional layer. Figure 2 The description of the method by which the neural network system 200 performs MAC operations using quantized weights and quantized samples.

[0055] Reference Figure 2 and Figure 4 The neural network system 200 can provide the weights W of the ANN along with a quantization request to the parameter quantizer 120, and receive the quantized weights q_W from the parameter quantizer 120. Furthermore, the neural network system 200 can provide the received input sample X along with a quantization request to the sample quantizer 130, and receive the quantized input sample q_X from the sample quantizer 130. Then, the neural network system 200 can perform a MAC operation based on the quantized weights q_W and the quantized input sample q_X, and reflect a bias (not shown) in the MAC operation result to generate a quantized output sample q_Y.

[0056] The quantized input sample q_X and the quantized output sample q_Y can be two-dimensional or higher-dimensional matrices and can each have its own activation parameters. If the quantized input sample q_X and the quantized output sample q_Y correspond to, for example, a three-dimensional matrix, then the quantized input sample q_X and the quantized output sample q_Y can have a width w (or columns), a height H (or rows), and a depth D. In some example embodiments, the depth D can be referred to as the number of channels.

[0057] In a convolutional layer, a convolution operation is performed on a quantized input sample q_X and quantized weights q_W, resulting in a quantized output sample q_Y. The quantized weights q_W can filter the quantized input sample q_X and can be referred to as a filter or kernel. The quantized weights q_W can have a kernel size K (i.e., the size of the weights), and the depth of the quantized weights q_W (i.e., the number of channels of the quantized weights q_W) can be the same as the depth D of the quantized input sample q_X. The quantized weights q_W can be shifted by traversing the quantized input sample q_X as a sliding window. During each shift, each weight included in the quantized weights q_W can be multiplied and added to all values ​​in the region overlapping with the quantized input sample q_X. When the quantized input sample q_X and the quantized weights q_W are convolved, one channel of the quantized output sample q_Y is generated. Although in Figure 4The diagram shows a quantized weight q_W, but in reality, multiple quantized weights q_W and quantized input samples q_X can be convolved to generate multiple channels of quantized output samples q_Y.

[0058] Performing MAC operations using quantized ANNs and quantized inputs may introduce errors compared to example implementations that use non-quantized ANNs and non-quantized inputs to perform MAC operations.

[0059] For example, the operations of ANN can be represented by the following equation.

[0060] [Equation 1]

[0061]

[0062] (C is the number of input channels, K is the core size, It is a quantified weight. It is the quantization error of the weight W, q bias It is a quantized bias, e bias It is the quantization error of the bias. It is a quantized input sample. (This refers to the quantization error of the input sample X).

[0063] The operations of an ANN can be divided into quantization operations (①) and quantization errors (②). The prior art neural network system 200 focuses on implementing quantization operations (①) optimized to reduce the quantization errors of the weights W and the input sample X, without directly considering the quantization errors (②) (i.e., not reflecting the quantization errors (②) in the ANN operations). The neural network system 200 according to the example embodiment can calculate the expected value of the quantization error (②) and reflect the calculated expected value in the quantization operations (①), thereby improving the accuracy of the quantized ANN. Therefore, the computational system 1000 for image recognition can improve the accuracy of image recognition.

[0064] exist Figure 4 For ease of description, the operations in one layer of an ANN are shown and described, but the operations of an ANN according to the inventive concept can be applied in substantially the same way to each of the multiple layers constituting an ANN.

[0065] Figure 5 This is a diagram illustrating the operation of reflecting the expected value of the quantization error of the neural network system 200 into the quantization operation according to an example embodiment.

[0066] Reference Figure 2 , Figure 4 and Figure 5According to the example embodiment, the neural network system 200 can perform a MAC operation based on quantized input samples q_X and quantized weights q_W received from the quantization system 100. Furthermore, the neural network system 200 can generate a quantized output sample q_Y by reflecting a compensated bias q_bias1 generated by the bias compensator 140 of the quantization system 100 (instead of a quantized bias q_bias obtained by simply quantizing the bias of the ANN) onto the MAC operation result. As will be discussed later... Figure 6 and Figure 7 As described above, the compensation bias q_bias1 can have the same size as the result of the MAC operation. Therefore, the compensation bias q_bias1 can be added to all values ​​in the region overlapping with the result of the MAC operation. Figure 5 In this context, the quantized output sample q_Y is indicated for any channel k, but the quantized output sample q_Y for each channel can be substantially generated.

[0067] The bias compensator 140 can generate a compensating bias q_bias1 that includes the quantization error and the bias q_bias of the ANN's quantization. Specifically, the bias compensator 140 can generate a compensating bias q_bias1 (as in equation 2) for including formula ③ in the following equation for the quantization operation of the ANN.

[0068] [Equation 2]

[0069]

[0070] In the above equation for the quantization operation of ANN, q bias e bias , and These are values ​​previously known, obtained by performing quantization of the ANN using sample quantizer 130. However, and The value is previously unknown when the operation is performed on the input sample X. Therefore, the bias compensator 140 can use a reference sample X' instead of the actual input sample X. and Instead of about the actual input sample X and To generate the compensation bias q_bias1.

[0071] For example, sample quantizer 130 can generate multiple quantized samples and quantization errors of multiple samples by quantizing multiple samples selected from the sample pool. Bias compensator 140 can use the quantized multiple samples to generate... Furthermore, it can generate quantization errors from multiple samples. In addition, the bias compensator 140 can be used and Generate the compensated bias q_bias1. (See below for further details.) Figure 9 Describe the specific operations for generating the compensation bias q_bias1 using multiple samples.

[0072] As another example, the bias compensator 140 can use input samples that have already been processed by the ANN to generate the expected next-order input samples (e.g., the next sample or next set of samples in time or space). Furthermore, the quantization error of the processed input samples is used to generate the input samples for the next sequence. In addition, the bias compensator 140 can be used and Generate the compensated bias q_bias1. In one example, the sample pool may include input samples that have already been processed by the ANN; therefore, the input samples for the next order can be predicted based on the input samples that have already been processed by the ANN in the sample pool. See later... Figures 10 to 14 Describe the specific steps involved in generating the compensation bias q_bias1 using the processed input samples.

[0073] Figure 6 This is a diagram illustrating a method for generating a compensated bias q_bias1 according to an example embodiment. For example, Figure 6 This is a diagram illustrating a method for generating a bias for quantization of an ANN and a compensation bias for quantization error.

[0074] The operation of compensating for bias can be expressed as the following equation.

[0075] [Equation 3]

[0076]

[0077] (E is the expected value)

[0078] In the last equation above, x c，k It can represent a reference sample X' instead of the actual input sample X, and This can be generated using a reference sample X'. More details will follow later. Figure 9 and Figure 13 The method for determining the reference sample X' and calculating the quantization error of the reference sample X' are described in the text. The method.

[0079] Reference Figure 6The final equation above can be implemented through the operation of bias compensator 140. Bias compensator 140 can perform a first MAC operation based on the quantization error e_X' of the reference sample X' and the quantization weight q_W. Furthermore, bias compensator 140 can perform a second MAC operation based on the quantization error e_W of the reference sample X' and the weight. Then, bias compensator 140 can combine the result of the first MAC operation, the result of the second MAC operation, and the bias of the ANN (which is compared with the quantization bias q_bias (as in equation 3)). bias ) and the quantization error e_bias of the bias (e in Equation 3) bias The sums of the two are equal, and they are added together to generate the compensation bias q_bias1.

[0080] Figure 7 This is a diagram illustrating a method for generating a compensated bias q_bias1 according to an example embodiment. For example, Figure 7 It is shown Figure 6 A diagram illustrating a modifiable example embodiment.

[0081] Figure 6 The operation of the compensation bias can be expressed as the following equation.

[0082] [Equation 4]

[0083]

[0084] In the final equation above, W c，k The weights W of an ANN can be represented as non-quantized weights. and Reference sample X' can be used instead of the actual input sample X to generate it. This will be discussed later. Figure 9 and Figure 13 The text describes the method for determining the reference sample X' and the calculation of the quantized reference sample. Quantization error with reference sample X' The method.

[0085] Reference Figure 7 The final equation above can be implemented through the operation of bias compensator 140. Bias compensator 140 can perform a third MAC operation based on the quantization error e_X' of the reference sample X' and the unquantized weights W. Furthermore, bias compensator 140 can perform a fourth MAC operation based on the quantized reference sample q_X' and the quantization error e_W of the weights. Additionally, bias compensator 140 can add the results of the third MAC operation, the fourth MAC operation, and the bias of the ANN to generate a compensating bias q_bias1.

[0086] exist Figure 6 and Figure 7The methods for generating the compensation bias q_bias1 are different, but the value of the compensation bias q_bias1 can be the same. Figure 5 The quantized input sample q_X and Figure 6 Reference sample X' and Figure 7 The quantization error e_X' of the quantized reference sample q_X' and the reference sample X' can have the same width w and height H. Figure 5 The quantized weights q_W Figure 6 The quantized weights q_W and Figure 7 The quantization error e_W of the weights can have the same kernel size K. Therefore, Figure 6 and Figure 7 The size of the compensation bias q_bias1 generated in the process can be the same as... Figure 5 The quantized weights q_W and the MAC operation result of the quantized input sample q_X in the channel have the same size. Therefore, the generated compensation bias q_bias1 can be added to all values ​​in the region that overlaps with the MAC operation result of the quantized weights q_W and the quantized input sample q_X.

[0087] Figure 8 This is a diagram illustrating the operation of reflecting the expected value of the quantization error of the neural network system 200 into the quantization operation according to an example embodiment. For example, Figure 8 It is shown Figure 5 A diagram illustrating a modifiable example embodiment.

[0088] As above Figure 7 As described above, the compensation bias q_bias1 may have the same size as the MAC operation result of the quantized weights q_W and the quantized input samples q_X in the channel. According to a modified example embodiment, the compensation bias q_bias1 may have scalar values ​​instead of a two-dimensional matrix structure. In an example embodiment, the bias compensator 140 can generate a second compensation bias q_bias2 with scalar values ​​by obtaining the average of the values ​​constituting the compensation bias q_bias1. The method for generating the second compensation bias q_bias2 with scalar values ​​performed by the bias compensator 140 is not limited to the above example, and various methods may be applied.

[0089] The bias compensator 140 may provide the generated second compensation bias q_bias2 to the neural network system 200. According to an example embodiment, the bias compensator 140 may provide the compensation bias q_bias1 to the neural network system 200, and use the compensation bias q_bias1 received by the neural network system 200 to generate a second compensation bias q_bias2 with a scalar value.

[0090] Reference Figure 8The neural network system 200 can perform MAC operations based on quantized input samples q_X and quantized weights q_W. Furthermore, the neural network system 200 can generate quantized output samples q_Y by reflecting a second compensation bias q_bias2 (instead of a quantized bias q_bias obtained by simply quantizing the bias of the ANN) in the MAC operation result. As described above, when using the second compensation bias q_bias2 instead of the compensation bias q_bias1, the efficiency of memory storage space can be improved, and only one scalar value can be reflected in the MAC operation result, thus improving the operation speed and the overall operation speed of the neural network system 200 used for image recognition.

[0091] Figure 9 This is a flowchart illustrating a method for determining a reference sample, a quantized reference sample, and a quantization error of the reference sample according to an example embodiment. For example, Figure 9 This is a flowchart illustrating a method for determining a reference sample, a quantized reference sample, and the quantization error of the reference sample to generate a compensating bias, which can be generated by... Figure 2 The neural network system 200 and the quantization system 100 are used in a fixed manner.

[0092] Reference Figure 2 and Figure 9 The neural network system 200 can select multiple first samples from a sample pool (S210). Here, the multiple first samples can be training samples used to train the ANN. However, the inventive concept is not limited to this; the multiple first samples can be samples not used to train the ANN, or can include both training samples used for training and samples not used for training.

[0093] Furthermore, the neural network system 200 can perform operations on multiple first samples via an ANN (S220). For example, the neural network system 200 can perform operations on multiple first samples via a non-quantized neural network.

[0094] Furthermore, the neural network system 200 can select at least one second sample from a plurality of first samples based on the statistical distribution of multiple output samples constituting each layer of the ANN (S230). For example, the neural network system 200 can identify the layers constituting a non-quantized ANN (e.g., Figure 1The statistical distribution of the output samples of each of the first layers L1 to the Nth layer LN. Here, the statistical distribution may include at least one of the mean, variance, expected value, asymmetry, and kurtosis of the output samples, and is not limited to the examples described above. Furthermore, the neural network system 200 may select at least one second sample from a plurality of first samples based on the statistical distribution. For example, the neural network system 200 may select at least one second sample from a plurality of first samples that corresponds to an output sample having a value close to the mean of each layer. The method of selecting at least one second sample based on the statistical distribution is not limited to the examples described above.

[0095] Then, the neural network system 200 can use the selected second sample to calculate the quantization error of the second sample and the quantization error of the second sample (S240). For example, the neural network system 200 can perform operations on the second sample using a quantized ANN to calculate the quantized second sample. Then, using the property that the input sample X is equal to the sum of the quantized input sample q_X and the quantization error e_X of the input sample, the neural network system 200 can calculate the quantization error of the second sample from the second sample and the quantized second sample.

[0096] Furthermore, the quantization system 100 can determine a reference sample, a quantized reference sample, and / or the quantization error of the reference sample (S250). In an example embodiment, the quantization system 100 can calculate the average value of at least one second sample, the quantized second sample, and the quantization error of the second sample, and determine the calculated average value of the second sample, the average value of the quantized second sample, and the average value of the quantization error of the second sample as the reference sample, the quantized reference sample, and the quantization error of the reference sample, respectively. The method performed by the quantization system 100 to determine the reference sample, the quantized reference sample, and / or the quantization error of the reference sample using the second sample, the quantized second sample, and the quantization error of the second sample is not limited to the above example, and various methods can be applied. In one embodiment, the quantization system 100 can generate a reference sample based on at least one second sample, quantize at least one second sample to generate at least one quantized second sample, and generate a quantized reference sample using the at least one quantized second sample.

[0097] Furthermore, the quantization system 100 can generate a compensation bias based on a defined reference sample, a quantized reference sample, and / or the quantization error of the reference sample. For example, referring to... Figure 6The bias compensator 140 of the quantization system 100 can perform a first MAC operation based on the quantization error e_X' of the reference sample and the quantization weight q_W. Furthermore, the bias compensator 140 can perform a second MAC operation based on the quantization error e_W of the reference sample X' and the weight. Then, the bias compensator 140 can add the result of the first MAC operation, the result of the second MAC operation, and the bias of the ANN (which is equal to the sum of the quantization bias q_bias and the quantization error e_bias of the bias) to generate a compensated bias q_bias1.

[0098] Optionally, refer to Figure 7 The bias compensator 140 of the quantization system 100 can perform a third MAC operation based on the quantization error e_X' of the reference sample and the unquantized weights W. Furthermore, the bias compensator 140 can perform a fourth MAC operation based on the quantized reference sample q_X' and the quantization error e_W of the weights. Additionally, the bias compensator 140 can add the result of the third MAC operation, the result of the fourth MAC operation, and the bias of the ANN to generate a compensating bias q_bias1.

[0099] As referenced above Figure 8 The bias compensator 140 can obtain the average value of the generated compensation bias q_bias1 to generate a second compensation bias q_bias2 with a scalar value.

[0100] As described above, the bias compensator 140 can use multiple samples to determine a reference sample, a quantized reference sample, and / or the quantization error of the reference sample, and generate a compensation bias q_bias1 or q_bias2 based on the determined reference sample, the quantized reference sample, and / or the quantization error of the reference sample. The neural network system 200 can use the generated compensation bias q_bias1 or q_bias2 in the computation of input samples. The inventive concept is not limited thereto; according to the example embodiment, the bias compensator 140 can generate the compensation bias periodically or non-periodically.

[0101] The bias compensator 140 can use the processed input samples to determine the reference sample, the quantized reference sample, and the quantization error of the reference sample. This will be discussed later. Figures 10 to 14 Provide a detailed description.

[0102] Figure 10 This is a diagram illustrating a computing system 1000a according to an example embodiment. For example, Figure 10 It is shown Figure 2 A diagram illustrating a modifiable example embodiment.

[0103] Reference Figure 10The computing system 1000a may include a quantization system 100a and a neural network system 200. The quantization system 100a may include a neural network interface 110, a parameter quantizer 120, a sample quantizer 130, a bias compensator 140a, and a sample generator 150a. According to... Figure 10 The neural network system 200, neural network interface 110, parameter quantizer 120, and sample quantizer 130 in the example embodiment may correspond to Figure 1 The neural network system 200, neural network interface 110, parameter quantizer 120 and sample quantizer 130 are described in a redundant manner, so their redundant descriptions are omitted.

[0104] The sample generator 150a can be implemented as processing circuitry (e.g., a logic block implemented through logic synthesis, a software block executed by a processor, or a combination thereof). See reference. Figure 10 Although the sample generator 150a is shown and described as being included in the quantization system 100a, according to the example embodiment, the sample generator 150a may be included in the neural network system 200, or may be implemented in a configuration separate from the neural network system 200 and the quantization system 100a.

[0105] The sample generator 150a can use at least one input sample that has already been processed by the neural network system 200 to generate the expected input sample for the next order. For example, the sample generator 150a can analyze at least one processed input sample and predict the input sample for the next order based on the analysis results. Furthermore, the sample generator 150a can provide the predicted input sample to the bias compensator 140a. See below for further details. Figure 11 and Figure 12 A detailed description is given of the operation performed by sample generator 150a to generate the next order of input samples using at least one processed input sample.

[0106] The bias compensator 140a can generate a compensated bias using the received, anticipated input samples. More on this later... Figure 11 The document describes a specific method for generating a compensation bias using the expected input sample, performed by the bias compensator 140a.

[0107] Figure 11 This is a diagram illustrating a method for predicting the next input sample according to an embodiment. For example, Figure 11 It is shown by Figure 10 A diagram illustrating a method performed by the sample generator 150a to predict the next sequence of input samples using at least one processed input sample.

[0108] The input samples processed by the neural network system 200 can be sequentially captured images. Each of the sequentially captured images has a small difference from images in a previous or subsequent order, and the differences between the images can be directional over time. Therefore, the sample generator 150a can predict the input samples for the next order by analyzing the differences between the input samples that have already been processed (e.g., sequentially captured images). Furthermore, the sample generator 150a can provide the predicted input samples for the next order to the bias compensator 140a. The sample generator 150a can read the processed input samples from a memory (not shown) included in the quantization system 100a, or receive the processed input samples from the neural network system 200 or any other different configuration.

[0109] For example, refer to Figure 11 The sample generator 150a can use a Kalman filter or similar method to calculate the motion vectors of the (N-2)th and (N-1)th frames, which are the closest images to the present image among the processed images. Furthermore, the sample generator 150a can use the calculated motion vectors to identify moving objects (e.g., people) in the image. Additionally, the sample generator 150a can generate the expected Nth frame by predicting the motion path in the image of the object to be identified.

[0110] Reference Figure 11 Although the sample generator 150a is shown and described as using two recent images to predict the next image, the inventive concept is not limited thereto, and two or more recent images can be used to predict the next image.

[0111] Figure 12 This is a diagram illustrating a method for predicting the next input sample according to an example embodiment. For example, Figure 12 It is shown by Figure 10 A diagram illustrating a method performed by the sample generator 150a to predict the next sequence of input samples using at least one processed input sample.

[0112] The sample generator 150a may include an artificial intelligence (AI) module 151a. The AI ​​module 151a may be an AI module trained to predict the next sample based on an input sample. The sample generator 150a may input at least one processed input sample into the AI ​​module 151a and provide an output sample from the AI ​​module 151a as the next input sample to the bias compensator 140a. The AI ​​module 151a may be implemented as a logic block implemented through logic synthesis, a software block executed by a processor, or a combination thereof.

[0113] Reference Figure 11 and Figure 12The input samples processed by the neural network system 200 are described as continuously captured images, but the inventive concept is not limited to this, and various types of data can be used.

[0114] Figure 13 This is a flowchart illustrating a method for determining a reference sample, a quantized reference sample, and a quantization error of the reference sample according to an example embodiment. Specifically, Figure 13 This is a flowchart illustrating a method for determining a reference sample, a quantized reference sample, and the quantization error of the reference sample to generate a compensating bias. Figure 10 The neural network system 200 and quantization system 100a can use this compensation bias in the computation of the next sequence of input samples.

[0115] Reference Figure 10 and Figure 13 The quantization system 100a can select at least one first sample from the processed input samples (S310). For example, the sample generator 150a of the quantization system 100a can select a preset number of first samples from the processed input samples. Here, the preset number can be set by the manufacturer or the user. For example, when the input samples are continuously captured images, the sample generator 150a can select 100 most recently processed images as first samples from the processed continuous images.

[0116] Furthermore, the quantization system 100a can predict the second sample for the next order based on the first sample (S320). For example, the sample generator 150a can analyze a preset number of first samples and predict the second sample for the next order based on the analysis results. The method for predicting the second sample for the next order performed by the sample generator 150a can be compared with... Figure 11 and Figure 12 The methods described herein are the same or substantially the same. Furthermore, the sample generator 150a can provide a second sample to the bias compensator 140a.

[0117] Then, the quantization system 100a can determine the expected second sample as a reference sample (S330). The quantization system 100a can then determine the quantization error of the reference sample using the quantization error of at least one first sample (S340). In an example embodiment, the bias compensator 140a can determine the quantization error of the reference sample by obtaining the average of the quantization errors of at least one first sample. Optionally, the bias compensator 140a can identify the quantization error of the closest currently processed sample among the at least one first sample and determine the identified quantization error as the quantization error of the reference sample. That is, the bias compensator 140a can determine the quantization error of the reference sample without using the property that the input sample X is equal to the sum of the quantized input sample q_X and the quantization error e_X of the input sample X.

[0118] Furthermore, the quantization system 100a can generate a compensation bias based on a defined reference sample and the quantization error of the reference sample. For example, the reference... Figure 6 The bias compensator 140a of the quantization system 100a can perform a first MAC operation based on the quantization error e_X' of the reference sample and the quantization weight q_W. Furthermore, the bias compensator 140a can perform a second MAC operation based on the quantization error e_W of the reference sample X' and the weight. Then, the bias compensator 140a can add the result of the first MAC operation, the result of the second MAC operation, and the bias of the ANN (which is equal to the sum of the quantization bias q_bias and the quantization error e_bias of the bias) to generate a compensated bias q_bias1.

[0119] According to a modifiable example embodiment, quantization system 100a can quantize a determined reference sample and generate a compensation bias based on the quantized reference sample and the quantization error of the reference sample. For example, referring to... Figure 7 The bias compensator 140a of the quantization system 100a can perform a third MAC operation based on the quantization error e_X' of the reference sample and the unquantized weights W. Furthermore, the bias compensator 140a can perform a fourth MAC operation based on the quantized reference sample q_X' and the quantization error e_W of the weights. Additionally, the bias compensator 140a can add the result of the third MAC operation, the result of the fourth MAC operation, and the bias of the ANN to generate a compensated bias q_bias1.

[0120] As referenced above Figure 8 The bias compensator 140a can generate a second compensation bias q_bias2 with a scalar value by obtaining the average value of the values ​​constituting the generated compensation bias q_bias1.

[0121] As described above, the bias compensator 140a can use the processed input samples to determine the reference sample, the quantized reference sample, and / or the quantization error of the reference sample, and generate a compensation bias q_bias1 or q_bias2 based on the determined reference sample, the quantized reference sample, and the quantization error of the reference sample. The neural network system 200 can use the generated compensation bias q_bias1 or q_bias2 in the computation of the next sequence of input samples.

[0122] Figure 14 This is a diagram illustrating a method for performing operations on the next sequence of input samples using a quantized ANN, according to an example embodiment. For example, Figure 14 It is shown by Figure 10 A diagram illustrating the method by which the quantization system 100a and the neural network system 200 perform operations on the input samples of the next sequence. In the following description, for ease of explanation, the description will be given under the assumption that the input samples are continuously captured images.

[0123] Reference Figure 10 and Figure 14 Image frames 0 through N, which are captured consecutively as multiple input samples, can be processed sequentially. When processing of frame N-1 is complete, the quantization system 100a can prepare for processing of frame N.

[0124] For example, quantization system 100a can select at least one image from processed image frames 0 to N-1. For example, refer to Figure 14 The sample generator 150a of the quantization system 100a can select frames N-1. Furthermore, the quantization system 100a can predict the next sequence of images based on the selected images. For example, referring to... Figure 14 The sample generator 150a can generate the expected frame N based on the expected frame N-1.

[0125] Furthermore, the quantization system 100a can use the image from the next order to generate a compensation bias. For example, refer to Figure 14 The bias compensator 140a can receive the expected frame N from the sample generator 150a and determine the received expected frame N as the reference sample X'. Furthermore, the bias compensator 140a can determine the quantization error e_X' of the reference sample using the quantization error of at least one of the processed image frames 0 to N-1. In an example embodiment, the bias compensator 140a can determine the quantization error e_X' of the reference sample by the quantization error of frame N-1, or by obtaining the average of the quantization errors of a preset number of images closest to the currently processed image.

[0126] The quantization errors of processed image frames 0 to N-1 can be calculated in previous operations on the processed image frames 0 to N-1 and stored in memory 160. For example, using the property that the sum of the input sample X and the quantized input sample q_X and the quantization error e_X of the input sample is equal, the quantization errors of the processed image frames 0 to N-1 can be calculated and stored in memory 160. Therefore, bias compensator 140a can read the quantization error of at least one of the processed image frames 0 to N-1 from memory 160 and use the read quantization error to determine the quantization error e_X' of the reference sample. Figure 14 Although memory 160 is shown and described as being included in quantization system 100a, memory 160 may be included in neural network system 200 or may be implemented in a configuration separate from neural network system 200 and quantization system 100a.

[0127] Furthermore, the bias compensator 140a can generate a compensation bias q_bias1 or q_bias2 based on the reference sample X' and the quantization error e_X' of the reference sample X'. Additionally, the bias compensator 140a can provide the generated compensation bias q_bias1 or q_bias2 to the neural network system 200 via the neural network interface 110.

[0128] Then, the sample quantizer 130 can quantize the frame N to generate a quantized frame N. Furthermore, the sample quantizer 130 can provide the quantized frame N to the neural network system 200 via the neural network interface 110.

[0129] Furthermore, the neural network system 200 can perform operations based on the received quantized frame N and the compensation bias q_bias1 or q_bias2. For example, the neural network system 200 can perform a MAC operation based on the quantized frame N and the quantized weights, and reflect the compensation bias q_bias1 or q_bias2 in the MAC operation result to generate a quantized output sample q_Y.

[0130] Figure 15 This is a flowchart illustrating an operational method using an ANN according to an embodiment. For example, Figure 15 It shows the use Figure 2 Computing system 1000 or Figure 10 A flowchart of the operation method of the ANN in the computing system 1000a.

[0131] Reference Figure 2 , Figure 10 and Figure 15 The computing system 1000 or 1000a can quantize the parameters of the ANN (S410). For example, the quantization system 100 of the computing system 1000 or the quantization system 100a of the computing system 1000a can quantize the parameters of the ANN (such as weights and biases). Furthermore, the computing system 1000 or 1000a can generate a compensated bias by compensating for the quantized bias to include the error caused by quantization (S420). Here, the compensated bias can be generated using at least one of the quantized weights of the ANN, the quantization error of the weights, the quantized bias, the quantization error of the bias, a reference sample, a quantized reference sample, and the quantization error of the reference sample. The reference sample can be determined from multiple samples different from the currently processed input sample, or from at least one of the processed input samples.

[0132] Furthermore, the computing system 1000 or 1000a can quantize the input samples (S430). For example, the quantization system 100 of the computing system 1000 or the quantization system 100a of the computing system 1000a can quantize the input samples. Furthermore, the quantization system 100 or 100a can provide the quantized input samples to the neural network system 200 of the computer system 1000 or 1000a. Furthermore, the computing system 1000 or 1000a can perform a MAC operation based on the quantized weights and the quantized input samples (S440). For example, the neural network system 200 can receive quantized input samples and perform a MAC operation based on the quantized weights and the received quantized input samples.

[0133] Furthermore, the computing system 1000 or 1000a can reflect the compensation bias used to compensate for quantization errors in the MAC operation result (S450). For example, the neural network system 200 of the computing system 1000 or 1000a can generate the final operation result by reflecting the compensation bias received from the quantization system 100 or 100a in the MAC operation result. The computing system according to the example embodiment can generate the expected value of the error that occurs in the quantization process as a compensation bias, and reflect the generated compensation bias in the MAC operation result through the quantized ANN. Therefore, the operation method using ANN according to the example embodiment can have reduced complexity due to the use of the quantized ANN, and has good performance based on the reflection of the compensation bias. Therefore, the complexity of the computing system 1000 or 1000a for image recognition can be reduced, and the computing system 1000 or 1000a for image recognition can have good performance.

[0134] Figure 16 This is a block diagram illustrating an electronic device 300 according to an example embodiment.

[0135] In the example embodiment, Figure 2 Quantization system 100 or Figure 10 The quantization system 100a can be implemented as Figure 16 Electronic device 300. For example... Figure 16 As shown, electronic device 300 may include system memory 310, processor 330, storage device 350, input / output (I / O or IO) device 370 and / or communication connection 390. Components included in electronic device 300 may be communicatively connected to each other, for example, via a bus.

[0136] System memory 310 may include program 312. Program 312 may cause processor 330 to perform quantization of the ANN, quantization of input samples, and generation of compensation bias according to an example embodiment. For example, program 312 may include a plurality of instructions executable by processor 330. Furthermore, the plurality of instructions included in program 312 may be executed by processor 330 to perform quantization of the ANN, quantization of input samples, or generation of compensation bias. As a non-limiting example, system memory 310 may include volatile memory (such as static random access memory (SRAM) and dynamic random access memory (DRAM)) or non-volatile memory (such as flash memory).

[0137] Processor 330 may include at least one core capable of executing any instruction set (e.g., Intel Architecture-32 (IA-32), 64-bit Extended IA-32, x86-64, PowerPC, Sparc, MIPS, ARM, IA-64, etc.). Processor 330 may execute instructions stored in system memory 310 and execute program 312 to perform ANN quantization, input sample quantization, or bias compensation generation.

[0138] Even if the power supply to the electronic device 300 is interrupted, the storage device 350 may not lose the stored data. For example, the storage device 350 may also include non-volatile memory (such as electrically erasable programmable read-only memory (EEPROM), flash memory, phase-change random access memory (PRAM), resistive random access memory (RRAM), nanofloating gate memory (NFGM), polymer random access memory (PoRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), etc.) and storage media (such as magnetic tape, optical disc, and magnetic disk). In some example embodiments, the storage device 350 may be removable from the electronic device 300.

[0139] In an example embodiment, storage device 350 may store a program 312 for generating ANN quantization, input sample quantization, and bias compensation according to the example embodiment. Furthermore, program 312, or at least a portion thereof, may be loaded from storage device 350 into system memory 310 before execution by processor 330. In an example embodiment, storage device 350 may store a file written in a programming language, and program 312, or at least a portion thereof, generated by a compiler or the like, may be loaded from the file into system memory 310.

[0140] In an example embodiment, storage device 350 may store data to be processed by processor 330 and / or data processed by processor 330. For example, storage device 350 may store input samples, quantized input samples and quantization errors of input samples, and generated compensation biases, etc.

[0141] I / O device 370 may include input devices (such as keyboards and indicator devices) and output devices (such as display devices and printers). For example, a user may trigger the processor 330 to execute program 312 through I / O device 370, input input samples, and identify output samples and / or error messages, etc.

[0142] The communication connection 390 provides access to a network external to the electronic device 300. For example, the network may include multiple computing systems and communication links, and the communication links may include wired links, optical links, wireless links, or any other form of link.

[0143] Figure 17 This is a block diagram illustrating an electronic device 400 according to an example embodiment. In the example embodiment, Figure 2 or Figure 10 The neural network system 200 can be implemented as Figure 17 Electronic device 400. As a non-limiting example, electronic device 400 can be any mobile electronic device (such as a mobile phone, tablet PC, wearable device, Internet of Things device, etc.) powered by a battery or self-generated electricity.

[0144] like Figure 17 As shown, electronic device 400 may include a memory subsystem 410, I / O devices 430, a processing unit 450, and / or a network interface 470, and the memory subsystem 410, I / O devices 430, processing unit 450, and / or network interface 470 may communicate with each other via bus 490. In some example embodiments, at least two of the memory subsystem 410, I / O devices 430, processing unit 450, and network interface 470 may be included in a package as a system-on-a-chip (SoC).

[0145] The memory subsystem 410 may include RAM 412 and storage device 414. RAM 412 and / or storage device 414 may store instructions executed by processing unit 450 and data to be processed by processing unit 450. For example, RAM 412 and / or storage device 414 may store parameters (such as signals, ANN weights, and biases). In some example embodiments, storage device 414 may include non-volatile memory.

[0146] The processing unit 450 may include a central processing unit (CPU) 452, a graphics processing unit (GPU) 454, a digital signal processor (DSP) 456, and a neural processing unit (NPU) 458. Figure 17 As shown in the example embodiment, the processing unit 450 may include only some of the CPU 452, GPU 454, DSP 456, and NPU 458.

[0147] The CPU 452 may directly perform the overall operation (e.g., a specific task) of the electronic device 400 in response to external input received through the I / O device 430, or may instruct other components of the processing unit 450 to perform the overall operation. The GPU 454 may generate data for outputting an image through a display device included in the I / O device 430, or may encode data received from a camera included in the I / O device 430. The DSP 456 may generate useful data by processing digital signals (such as digital signals provided from the network interface 470).

[0148] The NPU 458 is dedicated hardware for ANNs and may include multiple computing nodes corresponding to at least some of the artificial neurons constituting the ANN, and at least some of these computing nodes may process signals in parallel. Because the ANN quantized according to the example embodiment has higher accuracy and / or lower computational complexity, ANNs can be easily... Figure 17 The electronic device 400 can be used to implement this, which can have a fast processing speed, and can be implemented, for example, by a small NPU 458.

[0149] I / O device 430 may include input devices (such as touch input devices, voice input devices, cameras) and output devices (such as display devices and sound output devices). For example, when a user's voice is input via a voice input device, the voice can be recognized by an ANN implemented in electronic device 400, and thus an operation can be triggered. Furthermore, when an image is input via a camera, objects included in the image can be recognized by a DNN implemented in electronic device 400, and an output (such as virtual reality) can be provided to the user. Network interface 470 may provide electronic device 400 with access to mobile communication networks (such as LTE, 5G, etc.) or access to a local network (such as Wi-Fi).

[0150] Any of the elements disclosed above (including neural network systems and / or quantization systems) may include or can be implemented as processing circuitry, which may include hardware containing logic circuitry, a hardware / software combination (such as a processor executing software), or a combination thereof. For example, processing circuitry may more specifically include, but is not limited to, a central processing unit (CPU), an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field-programmable gate array (FPGA), a system-on-a-chip (SoC), a programmable logic unit, a microprocessor, an application-specific integrated circuit (ASIC), etc.

[0151] In some embodiments, an artificial neural network (ANN) may include an organizational model and a processing model (such as a convolutional neural network (CNN), a deconvolutional neural network, a recurrent neural network (RNN) that optionally includes long short-term memory (LSTM) units and / or gated recurrent units (GRUs), a stacked neural network (SNN), a state-space dynamic neural network (SSDNN), a deep belief network (DBN), a generative adversarial network, and / or a restricted Boltzmann machine (RBM)).

[0152] Optionally or additionally, artificial neural networks (ANNs) may include other forms of machine learning models (such as linear and / or logistic regression, statistical clustering, Bayesian classification, decision trees, dimensionality reduction (such as principal component analysis), and expert systems and / or combinations thereof (including ensembles such as random forests). Such machine learning models can also be used to provide a variety of services and / or applications (e.g., image classification services, user authentication services based on biometrics or biometric data, advanced driver assistance systems (ADAS) services, voice assistant services, automatic speech recognition (ASR) services, etc.), and such machine learning models may be executed, run, or processed by electronic devices.

[0153] While the inventive concept has been specifically shown and described with reference to exemplary embodiments thereof, it will be understood that various changes in form and detail may be made therein without departing from the spirit and scope of the claims.

Claims

1. A computing system for recognizing an image, comprising: a neural network system configured to obtain an input image, drive an artificial neural network with the input image as an input sample of the artificial neural network to obtain an image recognition result, wherein the step of driving the artificial neural network comprises, for each layer of the at least one layer in the artificial neural network, generating an output sample of the layer of the artificial neural network based on an input sample of the layer of the artificial neural network, a quantized parameter and a compensation bias; and a quantization system configured to quantize the artificial neural network, wherein the quantization system is further configured to, for each layer of the at least one layer in the artificial neural network, quantize a parameter of the layer of the artificial neural network to generate a quantized parameter of the layer of the artificial neural network, generate a quantization error of the parameter of the layer of the artificial neural network based on the parameter and the quantized parameter of the layer of the artificial neural network, generate a compensation bias of the layer of the artificial neural network based on the quantized parameter and the quantization error of the parameter of the layer of the artificial neural network, and send the generated quantized parameter and the compensation bias to the neural network system.

2. The computing system of claim 1, wherein, a first layer of the artificial neural network obtains the input image as an input sample of the first layer, and an output sample of a previous layer of the artificial neural network is an input sample of a next layer of the artificial neural network.

3. The computing system of claim 1, wherein the quantization system is further configured to, upon receiving the input sample of each layer of the at least one layer in the artificial neural network from the neural network system, quantize the input sample and send the quantized input sample to the neural network system, and wherein the neural network system is further configured to, upon receiving the quantized input sample of each layer of the at least one layer in the artificial neural network, perform a first product and accumulation operation based on the quantized input sample and the quantized parameter, and reflect the compensation bias to a result of the first product and accumulation operation to generate the output sample of each layer of the at least one layer in the artificial neural network.

4. The computing system of any one of claims 1 to 3, wherein, the parameter of the artificial neural network comprises a weight and a bias of the artificial neural network, wherein the quantized parameter comprises a quantized weight and a quantized bias, and wherein the quantization error of the parameter comprises a quantization error of the weight and a quantization error of the bias.

5. The computing system of claim 4, wherein, the quantization system is further configured to identify a reference sample to generate the compensation bias, quantize the reference sample to generate a quantized reference sample, generate a quantization error of the reference sample based on the reference sample and the quantized reference sample, and generate the compensation bias based on at least one of the reference sample, the quantized reference sample, the quantization error of the reference sample, the quantized parameter and the quantization error of the parameter of the artificial neural network.

6. The computing system of claim 5, wherein, the quantization system is further configured to perform a second product and accumulation operation based on the reference sample and the quantization error of the weight, perform a third product and accumulation operation based on the quantization error of the reference sample and the quantized weight, and generate the compensation bias based on a result of the second product and accumulation operation and a result of the third product and accumulation operation.

7. The computing system of claim 6, wherein, The quantization system is further configured to sum the result of the second multiply-accumulate operation, the result of the third multiply-accumulate operation, and a bias of the artificial neural network to generate the compensation bias.

8. The computing system of claim 7, wherein, The quantization system is further configured to calculate a mean value of the result of the second multiply-accumulate operation, the result of the third multiply-accumulate operation, and the bias of the artificial neural network, and generate the compensation bias having the calculated mean value as a scalar value.

9. The computing system of claim 5, wherein, The quantization system is further configured to perform a fourth multiply-accumulate operation based on a quantization error of the reference sample and the weight, perform a fifth multiply-accumulate operation based on the quantization error of the reference sample and a weight of the artificial neural network, and generate the compensation bias based on a result of the fourth multiply-accumulate operation and a result of the fifth multiply-accumulate operation.

10. The computing system of claim 5, wherein, The neural network system is further configured to select at least one first sample from the sample pool and transmit the at least one first sample to the quantization system, and wherein the quantization system is further configured to generate a reference sample based on the at least one first sample, quantize the at least one first sample to generate a quantized at least one first sample, and generate the quantized reference sample using the quantized at least one first sample.

11. The computing system of claim 10, wherein, The neural network system is further configured to select a plurality of second samples from the sample pool, perform an operation on the plurality of second samples by the artificial neural network, and select the at least one first sample based on a result of the operation.

12. The computing system of claim 11, wherein, The neural network system is further configured to identify a statistical distribution of output samples constituting each layer of the artificial neural network based on the result of the operation, and select the at least one first sample based on the identified statistical distribution.

13. The computing system of claim 4, wherein, The quantization system is further configured to project a next-in-sequence sample based on at least one third sample processed by the neural network system based on the quantized parameters to generate a projected input sample, generate a quantization error of the projected input sample based on a quantization error of the at least one third sample, and generate the compensation bias based on the projected input sample, the quantization error of the projected input sample, the quantized parameters, and a quantization error of the parameters of the artificial neural network.

14. The computing system of claim 13, wherein, The quantization system is further configured to calculate a motion vector of the at least one third sample, and generate the projected input sample using the calculated motion vector.

15. A method for recognizing an image, the method comprising: obtaining an input image as an input sample of an artificial neural network; and performing the artificial neural network based on the input sample to obtain an image recognition result, wherein the step of obtaining the image recognition result comprises quantizing weights and biases of at least one layer of the artificial neural network, generating a compensation bias by compensating the quantized biases to include errors due to quantization, quantizing input samples of the layer of the artificial neural network, performing a first multiply-accumulate operation based on the quantized weights and the quantized input samples of the layer of the artificial neural network, and reflecting the compensation bias to a result of the first multiply-accumulate operation to generate output samples of the layer of the artificial neural network.

16. The method of claim 15, wherein, The step of generating the compensated bias includes compensating the quantized bias based on a first error defined as a quantization error of the reference sample and a second error defined as a quantization error of the weight.

17. The method of claim 16, wherein, The step of generating the compensated bias includes: performing a second product-sum operation based on the quantized weight and the first error; performing a third product-sum operation based on the reference sample and the second error; and compensating the quantized bias based on a result of the second product-sum operation and a result of the third product-sum operation.

18. The method of claim 17, wherein, The step of compensating the quantized bias based on the result of the second product-sum operation and the result of the third product-sum operation includes: summing the result of the second product-sum operation, the result of the third product-sum operation, and a bias of the artificial neural network; and compensating the quantized bias to include a result of the summing.

19. The method of claim 16, wherein, The step of generating the compensated bias includes: performing a fourth product-sum operation based on the weight and the first error; quantizing the reference sample; performing a fifth product-sum operation based on the quantized reference sample and the second error; and compensating the quantized bias based on a result of the fourth product-sum operation and a result of the fifth product-sum operation.

20. A method for recognizing an image, the method comprising: obtaining an input image as an input sample of an artificial neural network; and executing the artificial neural network based on the input sample to obtain an image recognition result, wherein the step of obtaining the image recognition result includes, for each of at least one layer of the artificial neural network, quantizing parameters of the layer of the artificial neural network, calculating a quantization error of the parameters of the layer of the artificial neural network based on the parameters of the layer of the artificial neural network and the quantized parameters, generating a compensated bias of the layer of the artificial neural network based on the quantized parameters of the layer of the artificial neural network and the quantization error of the parameters, quantizing an input sample of the layer of the artificial neural network, and generating an output sample of the layer of the artificial neural network based on the quantized parameters, the compensated bias, and the quantized input sample.

Images