Method and system for bit quantization of artificial neural networks

Abstract

The present disclosure provides a method for bit quantization of an artificial neural network. The method may include: (a) selecting a parameter or a parameter group to be quantized in the artificial neural network; (b) bit quantization to reduce the size of the data representation of the selected parameter or parameter group to a bit unit; (c) determining whether the accuracy of the artificial neural network is equal to or greater than a predetermined target value; (d) when the accuracy of the artificial neural network is equal to or greater than the target value, repeating steps (a) to (c).

Description

Technical Field

[0001] The present disclosure relates to a method and system for bit quantization of an artificial neural network, and more particularly, to a method and system for bit quantization capable of reducing storage usage while maintaining basic accuracy of the artificial neural network. Background Art

[0002] An artificial neural network is a computer structure that simulates the biological brain. In an artificial neural network, the nodes corresponding to the neurons in the brain are connected to each other, and the strength of the synaptic coupling between neurons is represented by weights. The artificial neural network changes the strength of the synaptic coupling between nodes by training artificial neurons (nodes), thereby building a model with the ability to solve a given problem.

[0003] Artificial neural network can refer to a multi-layer perceptron, a feedforward neural network in a narrow sense, but is not limited to this, and may include various types of neural networks, such as radial basis function networks, self-organizing networks, and recurrent neural networks.

[0004] Recently, multi-layer deep neural networks are widely used as image recognition technology, and a representative example of multi-layer deep neural networks is convolutional neural networks (CNNs). In the case of general multi-layer feedforward neural networks, input data is limited to one-dimensional form, but if image data consisting of two or three dimensions is flattened into one-dimensional data, spatial information is lost, and it may be difficult to train the neural network while maintaining the spatial information of the image. However, convolutional neural networks can train visual information while maintaining two-dimensional (2D) or three-dimensional (3D) spatial information.

[0005] Specifically, convolutional neural networks are effective in recognizing patterns in visual data because they include a max-pooling process that effectively recognizes features of adjacent images while maintaining image spatial information, and collects and strengthens the features of extracted images. However, deep neural networks with multi-layer structures such as convolutional neural networks use deep structures to provide high recognition performance, but their structures are very complex and require a lot of computation and a lot of storage. In multi-layer deep neural networks, most of the operations that occur internally are performed using multiplication and addition or accumulation. The number of connections between nodes in artificial neural networks is large, and the parameters that require multiplication, such as weight data, feature map data, activation map data, etc., are large, so a lot of computation is required during the training process or recognition process. Summary of the invention

[0006] Technical issues

[0007] As mentioned above, in the training and recognition process of multi-layer deep neural networks such as convolutional neural networks, a large amount of calculation and storage is required. As a method to reduce the amount of calculation and storage of multi-layer deep neural networks, a bit quantization method can be used to reduce the data representation size of parameters used in artificial neural network calculations in bits. The traditional bit quantization method uses uniform bit quantization to quantize all parameters of the artificial neural network with the same number of bits. The problem with the traditional uniform bit quantization method is that it cannot accurately reflect the impact of changing the number of bits for each parameter used in the artificial neural network on the overall performance.

[0008] The embodiments disclosed in the present disclosure are intended to provide a method and system for quantizing each parameter data constituting an artificial neural network or parameter data grouped according to a specific standard into a specific number of bits, thereby maintaining the accuracy of artificial intelligence while improving the overall performance of the artificial neural network.

[0009] Solutions to the problem

[0010] According to an embodiment of the present disclosure, a method for quantizing bits of an artificial neural network is provided. The method includes the following steps: (a) selecting at least one parameter from a plurality of parameters used in the artificial neural network; (b) bit quantization to reduce the data size required for the operation of the selected parameter to a bit unit; (c) determining whether the accuracy of the artificial neural network is greater than or equal to a predetermined target value; (d) if the accuracy of the artificial neural network is greater than or equal to the target value, repeating steps (b) to (c) for the parameter to further reduce the number of bits in the data representation of the parameter. In addition, the method also includes the step (e) if the accuracy of the artificial neural network is less than the target value, restoring the number of bits of the parameter to the number of bits when the accuracy of the artificial neural network is greater than the target value, and then repeating steps (a) to (d).

[0011] According to an embodiment of the present disclosure, a method for quantizing bits of an artificial neural network is provided. The method comprises the following steps: (a) selecting at least one layer of a plurality of layers by a parameter selection module; (b) bit quantization by a bit quantization module to reduce the size of the data representation of the parameters of the selected layer to a bit unit; (c) determining by an accuracy determination module whether the accuracy of the artificial neural network is greater than or equal to a predetermined target value; and (d) when the accuracy of the artificial neural network is greater than or equal to the target value, repeating steps (a) to (c).

[0012] According to an embodiment of the present disclosure, a method for quantizing bits of an artificial neural network is provided. The method comprises the following steps: (a) selecting one or more data or one or more groups of data from weights, feature maps, and activation map data from an artificial neural network; (b) bit quantizing by a bit quantization module to reduce the data representation size of the selected data to a bit unit; (c) measuring whether the artificial intelligence accuracy of the artificial neural network is greater than or equal to a target value; and (d) repeating steps (a) to (c) until there is no more data in the data of the artificial neural network that needs to be quantized.

[0013] According to an embodiment of the present disclosure, a method for quantizing bits of an artificial neural network is provided. The method includes: training the artificial neural network according to one or more parameters of the artificial neural network; performing bit quantization on one or more parameters of the artificial neural network according to the bit quantization method of the artificial neural network according to the embodiment; and training the artificial neural network according to one or more parameters of the artificial neural network on which bit quantization is performed.

[0014] According to another embodiment of the present disclosure, a system for quantizing bits of an artificial neural network is provided. The system may include: a parameter selection module for selecting at least one parameter in the artificial neural network; bit quantization to reduce the size of the data representation of the selected parameter to a bit unit; and an accuracy determination module for determining whether the accuracy of the artificial neural network is greater than or equal to a predetermined target value. If the accuracy of the artificial neural network is greater than the target value, the accuracy determination module controls the parameter selection module and the bit quantization module to perform quantization, so that each of the multiple parameters has a minimum number of bits while keeping the accuracy of the artificial neural network higher than the target value.

[0015] According to an embodiment of the present disclosure, a system for quantizing bits of an artificial neural network is provided. The system includes a parameter selection module, which is used to select at least one layer from multiple layers constituting the artificial neural network; a bit quantization module, which is used to reduce the size of the data representation of the parameters of the selected layer to a bit unit; and an accuracy determination module, which determines whether the accuracy of the artificial neural network is greater than or equal to a predetermined target value. If the accuracy of the artificial neural network is equal to or greater than the target value, the accuracy determination module controls the parameter selection module and the bit quantization module to perform bit quantization on another layer of the multiple layers, and the bit quantization module sets n bits (where n is an integer of n>0) for all weights of the multiple layers, and sets m bits (where m is an integer of m>0) for the output data of the multiple layers.

[0016] According to an embodiment of the present disclosure, a system for quantizing bits of an artificial neural network is provided. The system includes: a parameter selection module, which is used to select at least one layer from multiple layers constituting the artificial neural network; a bit quantization module, which is used to reduce the size of the data representation of the parameters of the selected layer to a bit unit; and an accuracy determination module, which determines whether the accuracy of the artificial neural network is greater than or equal to a predetermined target value, wherein when the accuracy of the artificial neural network is greater than or equal to the target value, the accuracy determination module controls the parameter selection module and the bit quantization module to perform bit quantization on another layer of the multiple layers, wherein the bit quantization module allocates n bits (n is an integer of n>0) to the weights and output data of the multiple layers, and sets the number of bits allocated to each layer of the multiple layers differently.

[0017] According to an embodiment of the present disclosure, a system for quantizing bits of an artificial neural network is provided. The system includes: a parameter selection module, which is used to select at least one layer from multiple layers constituting the artificial neural network; a bit quantization module, which is used to reduce the size of the data representation of the parameters of the selected layer to a bit unit; and an accuracy determination module, which determines whether the accuracy of the artificial neural network is greater than or equal to a predetermined target value, when the accuracy of the artificial neural network is greater than or equal to the target value, wherein the accuracy determination module controls the parameter selection module and the bit quantization module to perform bit quantization on another layer of the multiple layers, and wherein the bit quantization module allocates the weights of the multiple layers and the number of bits of the output data separately and differently.

[0018] According to an embodiment of the present disclosure, a system for quantizing bits of an artificial neural network is provided. The system includes: a parameter selection module for selecting at least one layer from a plurality of layers constituting the artificial neural network; a bit quantization module for reducing the storage size for storing the parameters of the selected layer to a bit unit; and an accuracy determination module for determining whether the accuracy of the artificial neural network is greater than or equal to a predetermined target value, wherein when the accuracy of the artificial neural network is greater than or equal to the target value, the accuracy determination module controls the parameter selection module and the bit quantization module to perform bit quantization on another layer of the plurality of layers, and wherein the bit quantization module allocates a different number of bits to each weight used in the plurality of layers.

[0019] According to an embodiment of the present disclosure, a system for quantizing bits of an artificial neural network is provided. The system includes: a parameter selection module for selecting at least one layer from a plurality of layers constituting the artificial neural network; a bit quantization module for reducing the size of the data representation of the parameters of the selected layer to a bit unit; and an accuracy determination module for determining whether the accuracy of the artificial neural network is equal to or greater than a predetermined target value, wherein when the accuracy of the artificial neural network is greater than or equal to the target value, the accuracy determination module controls the parameter selection module and the bit quantization module to perform bit quantization on another layer of the plurality of layers, and wherein the bit quantization module individually allocates different numbers of bits to specific units of output data output from the plurality of layers.

[0020] According to an embodiment of the present disclosure, a system for quantizing bits of an artificial neural network is provided. The system includes: a parameter selection module for selecting at least one layer from a plurality of layers constituting the artificial neural network; a bit quantization module for reducing the size of the data representation of the parameters of the selected layer to a bit unit; and an accuracy determination module for determining whether the accuracy of the artificial neural network is greater than or equal to a predetermined target value, wherein when the accuracy of the artificial neural network is greater than or equal to the target value, the accuracy determination module controls the parameter selection module and the bit quantization module to perform bit quantization on another layer of the plurality of layers, and wherein the bit quantization module allocates different bits to each value of the output data output from the plurality of layers.

[0021] Effects of the Invention

[0022] According to various embodiments of the present disclosure, overall operation performance may be improved by quantizing the number of bits of data required for operations such as training or inference in an artificial neural network.

[0023] In addition, an artificial neural network can be implemented that does not compromise artificial intelligence accuracy while reducing hardware resources required to implement the artificial neural network and reducing power consumption and memory usage. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Embodiments of the present disclosure will be described with reference to the drawings described below, wherein like reference numerals refer to like elements, but are not limited thereto.

[0025] Figure 1 An example of an artificial neural network for obtaining output data for input data using multiple layers and multiple layer weights according to an embodiment of the present disclosure is shown.

[0026] Figures 2 to 3 According to the embodiments of the present disclosure Figure 1 A specific implementation example of the artificial neural network shown.

[0027] Figure 4Another example of an artificial neural network including multiple layers according to an embodiment of the present disclosure is shown.

[0028] Figure 5 The weight kernels used for input data and convolution operations in the convolutional layers according to an embodiment of the present disclosure are shown.

[0029] Figure 6 A process of generating a first activation map by performing convolution on input data using a first weight kernel according to an embodiment of the present disclosure is shown.

[0030] Figure 7 A process of generating a second activation map by performing convolution on input data using a second weight kernel according to an embodiment of the present disclosure is shown.

[0031] Figure 8 The process of calculating the convolution layer as a matrix according to an embodiment of the present disclosure is shown.

[0032] Fig. 9 The operation process of the fully connected layer in the matrix according to the embodiment of the present disclosure is shown.

[0033] Fig.10 The bit quantization process of the convolutional layer as a matrix according to an embodiment of the present disclosure is shown.

[0034] Fig.11 is a flow chart illustrating a method of quantizing bits of an artificial neural network according to an embodiment of the present disclosure.

[0035] Fig.12 is a flow chart illustrating a method of quantizing bits of an artificial neural network according to another embodiment of the present disclosure.

[0036] Fig.13 is a flowchart illustrating a bit quantization method of an artificial neural network according to another embodiment of the present disclosure.

[0037] Fig.14 is a graph showing an example of the amount of computation per layer of an artificial neural network according to an embodiment of the present disclosure.

[0038] Fig.15 is a graph showing the number of bits per layer of an artificial neural network in which bit quantization is performed by a forward bit quantization method according to an embodiment of the present disclosure.

[0039] Fig.16 is a graph showing the number of bits per layer of an artificial neural network in which bit quantization is performed by an inverse bit quantization method according to an embodiment of the present disclosure.

[0040] Fig.17is a graph showing the number of bits per layer of an artificial neural network in which bit quantization is performed by a high computational cost layer first bit quantization method according to an embodiment of the present disclosure.

[0041] Fig.18 is a graph showing the number of bits per layer of an artificial neural network in which bit quantization is performed by a low computational cost layer-first bit quantization method according to an embodiment of the present disclosure.

[0042] Fig.19 An example of a hardware implementation of an artificial neural network according to an embodiment of the present disclosure is shown.

[0043] Fig. 20 An example of a hardware implementation of an artificial neural network according to another embodiment of the present disclosure is shown.

[0044] Fig.21 An example of hardware implementation of an artificial neural network according to another embodiment of the present disclosure is shown.

[0045] Fig. 22 A configuration of a system for performing bit quantization on an artificial neural network according to an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0046] The specific details for implementing the present disclosure will be described in detail below in conjunction with the accompanying drawings. However, in the following description, if the subject matter of the present disclosure may be unnecessarily obscure, a detailed description of well-known functions or configurations may be omitted.

[0047] In the accompanying drawings, identical or corresponding elements are assigned the same reference numerals. In addition, in the description of the following embodiments, redundant descriptions of identical or corresponding elements may be omitted. However, even if the description of an element is omitted, it is meant that such an element is not included in any embodiment.

[0048] In the present disclosure, "parameter" may refer to one or more of an artificial neural network or weight data, feature map data, and activation map data constituting each layer of an artificial neural network. In addition, "parameter" may refer to an artificial neural network or each layer constituting an artificial neural network represented by such data. In addition, in the present disclosure, "bit quantization" may refer to an operation or an operation for reducing the number of bits in a data representation representing a parameter or a group of parameters.

[0049] The present disclosure provides various embodiments of quantization methods and systems for reducing the size of data representations of parameters used in related operations to bits to reduce the computation, storage usage, and power consumption of digital hardware systems. In some embodiments, the bit quantization methods and systems of the present disclosure can reduce the size of parameters used in artificial neural network operations to bits. Typically, the calculation of artificial neural networks uses data structures (e.g., CPU, GPU, memory, cache, buffer, etc.) in 32-bit, 16-bit, or 8-bit units. Therefore, the quantization methods and systems of the present disclosure can reduce the size of parameters used to calculate artificial neural networks to a number of bits other than 32, 16, and 8 bits. In addition, a specific number of bits can be allocated to each parameter or parameter group of the artificial neural network individually and differently.

[0050] In some embodiments, the bit quantization method and system of the present disclosure can set n bits for all weights of the artificial neural network model, where n is an integer of n>0, and set m bits for the output data of each layer, where m is an integer of m>0.

[0051] In another embodiment, the bit quantization method and system of the present disclosure may allocate n bits to the weights and output data of each layer of an artificial neural network model, where n may be set to a different number for each layer.

[0052] In another embodiment, the bit quantization method and system of the present disclosure allocates different bits to the weights and output data of each layer of the artificial neural network model, and can allocate different numbers of bits to each layer for the weights and output feature map parameters of the corresponding layer.

[0053] The bit quantization method and system of the present disclosure can be applied to various artificial neural networks. For example, when the bit quantization method and system of the present disclosure are applied to a convolutional artificial neural network (CNN), different bits can be individually allocated to the weight kernel used in each layer of the artificial neural network.

[0054] In another embodiment, the bit quantization method and system of the present disclosure can allocate different bits to each weight used in each layer of a multi-layer artificial neural network model, allocate separate bits to specific units of the output data of each layer, or allocate different bits to each value of the output data.

[0055] The above-mentioned bit quantization method and system according to various embodiments of the present disclosure may apply any one of the above-mentioned embodiments to an artificial neural network model, but is not limited thereto, and one or more of these embodiments may be combined and applied to an artificial neural network model.

[0056] Figure 1An example of an artificial neural network 100 that uses multiple layers and multiple layer weights to obtain output data for input data according to an embodiment of the present disclosure is shown.

[0057] Typically, a multi-layer artificial neural network such as the artificial neural network 100 includes a statistical training algorithm implemented based on the structure of a biological neural network in machine learning technology and cognitive science, or a structure that executes its algorithm. That is, in the artificial neural network 100, like a biological neural network, nodes (i.e., artificial neurons that form a network by combining synapses) repeatedly adjust the weights of synapses, and a machine learning model with problem-solving capabilities can be created through training to reduce the error between the correct output corresponding to a specific input and the inferred output.

[0058] In one example, the artificial neural network 100 can be implemented as a multilayer perceptron (MLP), which is composed of layers including one or more nodes and connections therebetween. However, the artificial neural network 100 according to the present embodiment is not limited to the structure of the MLP, but can be implemented using one of various artificial neural network structures having a multilayer structure.

[0059] like Figure 1 As shown, when input data from the outside is provided, the artificial neural network 100 is configured to output output data corresponding to the input data through a plurality of layers 1101, 1102, ..., 110N each consisting of one or more nodes.

[0060] In general, the training method of the artificial neural network 100 includes a supervised learning method that trains to optimize to solve a problem by inputting a teacher signal (correct answer), an unsupervised learning method that does not require a teacher signal, and a semi-supervised learning method that uses supervised learning and unsupervised learning. Figure 1 The artificial neural network 100 shown uses at least one of a supervised learning method, an unsupervised learning method, and a semi-supervised learning method according to the user's selection. Therefore, the artificial neural network 100 that generates output data can be trained.

[0061] Figures 2 to 3 According to an embodiment of the present disclosure Figure 1 A specific implementation example of the artificial neural network 100 is shown.

[0062] refer to Figure 2 , the artificial neural network 200 may include an input node (X 0 , X 1 ...X n-1 , X n ), the input data 210 is input to the input node (X 0 , X 1 ...X n-1 , Xn ) in which the output node (Y) outputs the output data corresponding to the input data 210. 0 , Y 1 ...Y n-1 , Y n ), as well as hidden nodes and multiple parameters between the input nodes and the output nodes. Input nodes (X 0 , X 1 ...X n-1 , X n ) is a node that configures the input layer 220 and receives input data 210 (such as an image) from the outside, and the output node (Y 0 , Y 1 ...Y n-1 , Y n ) is a node configuring the output layer 240 and can output output data to the outside. The hidden node located between the input node and the output node is a node configuring the hidden layer 230 and can connect the output data of the input node to the input data of the output node. Figure 2 As shown, each node of the input layer 220 may be completely connected to each output node of the output layer 240, or may not be completely connected. In addition, the input node may be used to receive input data from the outside and transmit it to the hidden node. In this case, the hidden node and the output node may perform calculations on the data, and the calculations may be performed by multiplying the received input data by a parameter or weight. When the calculation of each node is completed, all the calculation results are added, and then the output data may be output using a preset activation function.

[0063] Hidden nodes and output nodes (Y 0 , Y 1 ...Y n-1 , Y n ) has an activation function. The activation function may be one of a function, a sign function, a linear function, a logistic sigmoid function, a hypertangent function, a ReLU function, and a normalized exponential function (softmax function). The activation function may be appropriately determined by a technician according to a learning method of an artificial neural network.

[0064] The artificial neural network 200 performs machine learning by repeatedly updating or modifying weight values ​​to appropriate values. Representative methods of machine learning of the artificial neural network 200 include supervised learning and unsupervised learning.

[0065] Supervised learning is a learning method that updates weight values ​​in a state where the target output data that any neural network is to calculate for input data is clearly defined, so that the output data obtained by putting the input data into the neural network is closer to the target data. Figure 2The multi-layer artificial neural network 200 can be generated based on supervised learning.

[0066] refer to Figure 3 As another example of a multi-layer artificial neural network, there is a convolutional neural network (CNN) 300, which is a deep neural network (DNN). A convolutional neural network (CNN) is a neural network composed of one or more convolutional layers, a pooling layer, and a fully connected layer. A convolutional neural network (CNN) has a structure suitable for training two-dimensional data and can be trained by a backpropagation algorithm. It is one of the representative models of DNN and is widely used in various application fields such as object classification and object detection in images.

[0067] It should be noted that the multi-layer artificial neural network disclosed in the present invention is not limited to Figure 2 and 3 The artificial neural network shown, and the training model can be obtained by machine learning other types of data in various other artificial neural networks.

[0068] Figure 4 Another example of an artificial neural network including multiple layers according to an embodiment of the present disclosure is shown. Figure 4 The artificial neural network 400 shown is a convolutional artificial neural network (CNN) including a plurality of convolutional layers (CONV) 420 , a plurality of subsampling layers (SUBS) 430 , and a plurality of fully connected layers (FC) 440 .

[0069] CONV 420 of CNN 400 generates a feature map by applying a convolution weight kernel to input data 410. Here, CONV 420 can be used as a template for extracting features from high-dimensional input data (e.g., images or videos). Specifically, a convolution can be repeatedly applied multiple times while changing the position of a portion of the input data 410 to extract features for the entire input data 410. In addition, SUBS 430 is used to reduce the spatial resolution of the feature map generated by CONV 420. The role of subsampling is to reduce the dimensionality of the input data (e.g., feature map), and thereby reduce the complexity of the analysis problem of the input data 410. SUBS 430 can use a maximum pooling operator or a feature extraction operator to take the maximum value. Figure 1 The average pooling operator of the average of the partial values. SUBS 430 not only reduces the dimension of the feature map through the pooling operation, but also has the effect of making the feature map robust to shift and distortion. Finally, FC 440 can perform a function to classify the input data based on the feature map.

[0070] CNN 400 may perform various configurations and functions according to the number of layers or types of operators of CONV 420, SUBS 430, and FC 440. For example, CNN 400 may include any of various CNN configurations such as AlexNet, VGGNet, LeNet, and ResNet, but is not limited thereto.

[0071] When image data is input as input data 410, CONV 420 of CNN 400 having the above configuration may apply weights to input data 410 to generate a feature map through a convolution operation, and in this case, a set of weights to be used may be referred to as a weight kernel. The weight kernel is composed of a three-dimensional matrix of nxmxd (here, n represents a row of a specific size like the input image data, m represents a column of a specific size, d represents a channel of the input image data, and the number of these dimensions is an integer greater than or equal to 1), and may generate a feature map through a convolution operation by traversing the input data 410 at specified intervals. At this time, if the input data 410 is a color image having multiple channels (e.g., three channels of RGB), the weight kernel may traverse each channel of the input data 410, calculate convolution, and then generate a feature map for each channel.

[0072] Figure 5 Convolutional layer input data and weight kernels for convolution operations according to an embodiment of the present disclosure are shown.

[0073] As shown, the input data 510 may be an image or video displayed in a two-dimensional matrix consisting of rows 530 of a specific size and columns 540 of a specific size. As described above, the input data 510 may have a plurality of channels 550, wherein the channels 550 may represent the number of color components of the input data image. Meanwhile, the weight kernel 520 may be a weight kernel for convolution to extract features of the corresponding portion when scanning a predetermined portion of the input data 510. Like the input data image, the weight kernel 520 may be configured to have rows 560 of a specific size, columns 570 of a specific size, and channels 580 of a specific number. Typically, the sizes of the rows 560 and columns 570 of the weight kernel 520 are set to be the same, and the number of channels 580 may be the same as the number of channels 550 of the input data image.

[0074] Figure 6 A process for generating a first activation map by performing a convolution on input data using a first kernel according to an embodiment of the present disclosure is shown.

[0075] The first weight kernel 610 may be an indication Figure 2The weight kernel of the first channel of the weight kernel 620 of the first weight kernel 610. The first weight kernel 610 can finally generate a first activation map 630 by traversing the input data at specified intervals and performing convolution. When the first weight kernel 610 is applied to a portion of the input data 510, convolution is performed by adding all values ​​generated by multiplying each input data value at a specific position of the portion by the value at the corresponding position of the weight kernel. Through this convolution process, a first result value 620 is generated, and each time the first weight kernel 610 traverses the input data 510, a convolution result value is generated to form a feature map. Each element value of the feature map is converted into a first activation map 630 by the activation function of the convolution layer.

[0076] Figure 7 A process of generating a second activation map by performing convolution on input data using a second weight kernel according to an embodiment of the present disclosure is shown.

[0077] like Figure 6 As shown, after performing convolution on the input data 510 using the first weight kernel 610 to generate the first activation map 620, as shown in Figure 7 As shown, a second activation map 730 may be generated by performing convolution on the input data 510 using a second weight kernel 710 .

[0078] The second weight kernel 710 may be an indication Figure 5 The second weight core 710 can finally generate a second activation map 730 by traversing the input data at specified intervals and performing convolution. Figure 6 As shown, when the second weight kernel 710 is applied to a portion of the input data 510, convolution is performed by adding all values ​​generated by multiplying each input data value at a specific position of the portion by the value at the corresponding position of the weight kernel. Through this convolution process, a second result value 720 is generated, and each time the second weight kernel 710 traverses the input data 510, a convolution result value is generated to form a feature map. Each element value of the feature map is converted into a second activation map 730 by the activation function of the convolution layer.

[0079] Figure 8 The calculation process of the convolution layer in the matrix when the input feature map has one channel according to an embodiment of the present disclosure is shown.

[0080] Figure 8 The convolutional layer 420 shown may correspond to Figure 4 CONV 420 shown in FIG. Figure 8, the input data 810 input to the convolution layer 420 is shown as a two-dimensional matrix of size 6×6, and the weight kernel 814 is shown as a two-dimensional matrix of size 3×3. However, the sizes of the input data 810 and the weight kernel 814 of the convolution layer 420 are not limited thereto, and may be variously changed according to the performance and requirements of the artificial neural network including the convolution layer 420.

[0081] As shown in the figure, when input data 810 is input to the convolution layer 420, the weight kernel 814 traverses the input data 810 at a predetermined interval (e.g., 1), and thus, element-wise multiplication can be performed, in which the input data 810 and the value at the same position in the weight kernel 814 are multiplied. The weight kernel 814 traverses the input data 810 at regular intervals and sums 816 the values ​​obtained by element-wise multiplication.

[0082] Specifically, the weight kernel 814 assigns the value of the element-by-element multiplication calculated at a specific position 820 of the input data 810 (e.g., "3") to the corresponding element 824 of the feature map 818. Next, the weight kernel 814 assigns the value of the element-by-element multiplication calculated at the next position 822 of the input data 810 (e.g., "1") to the corresponding element 826 of the feature map 818. In this way, when the weight kernel 814 traverses the input data 810 and assigns the calculated element-by-element multiplication value, a feature map 818 of size 4×4 is completed. At this time, if the input data 810 consists of, for example, three channels (R channel, G channel, B channel), a feature map of each channel can be generated by convolution, where the same weight kernel for each channel or different channels traverses the data for each channel of the input data 810 and performs element-by-element multiplication 812 and summation 816.

[0083] Refer to Figure 4 , CONV 420 can be implemented by applying an activation function according to the reference Figure 2-8 The activation map is generated by the feature map generated by the method described above to generate the activation map as the final output result of the convolution layer. The activation function here can be any of various activation functions such as sigmoid function, radial basis function (RBF), rectified linear unit (ReLU), or any of various activation functions, their modified functions, or other functions.

[0084] At the same time, SUBS 430 receives the activation map as input data, which is the output data of CONV420. SUBS430 performs a function of reducing the size of the activation map or highlighting specific data. When SUBS 430 uses maximum pooling, the maximum value of the value in a specific area of ​​the activation map is selected and output. In this way, the noise of the input data can be removed by the pulling process of SUBS 430, and the size of the data can be reduced.

[0085] In addition, FC 440 may receive output data of SUBS 430 and generate final output data 450. Figure 1 The θ is flattened to be input to the fully connected layer 440.

[0086] Fig. 9 The operation process of the fully connected layer as a matrix according to an embodiment of the present disclosure is shown.

[0087] Fig. 9 The fully connected layer 440 shown may correspond to Figure 4 FC 440. As described above, the activation map extracted from the max pooling layer 430 may be flattened into one dimension to be input to the fully connected layer 440. The activation map flattened in one dimension may be received as input data 910 from the fully connected layer 440. In the fully connected layer 440, an element-wise multiplication 912 of the input data 910 and the weight kernel 914 may be performed using a one-dimensional weight kernel 914. The results of the element-wise multiplication of the input data 910 and the weight kernel 914 may be summed 916 and output as output data 918. In this case, the output data 918 may represent an inferred value of the input data 410 input to the CNN 400.

[0088] The CNN 400 having the above configuration receives input data of a two-dimensional or one-dimensional matrix for each layer in the multilayer, and performs training and inference processes on the input data through complex operations such as element-by-element multiplication and weight kernel summation. Therefore, depending on the number of layers or the complexity of the operations constituting the CNN 400, the resources required for data training and inference (e.g., the number of operators or the amount of storage) may increase significantly. Therefore, in order to reduce the amount of computation and storage of an artificial neural network having multiple layers such as the CNN 400, bit quantization may be performed on the input and output data for each layer. In one embodiment, bit quantization of the CNN 400 having multiple layers may be performed for CONV 420 and FC 440 that require a lot of computation and storage.

[0089] Fig.10 The bit quantization process of the convolutional layer as a matrix according to an embodiment of the present disclosure is shown.

[0090] The bit quantization performed in the convolution layer may include weight or weight kernel quantization (1028) for reducing the number of bits per element value of the weight kernel used in the convolution operation; and / or feature map quantization or activation map quantization (1030) for reducing the number of bits per element value of the feature map or activation map.

[0091] The bit quantization process of the convolution layer according to the embodiment can be performed as follows. Before performing convolution by applying the weight kernel 1014 to the input data 1010 of the convolution layer, the weight kernel 1014 is subjected to a quantization 716 process to generate a quantized weight kernel 1018. In addition, by applying the quantized weight kernel 1018 to the input data 1010 and performing element-wise multiplication (1012) and summation (1020) to output the convolution value to generate a feature map, an activation map 1022 can then be generated by an activation function. Next, a final quantized activation map 1026 can be generated by quantizing the activation map 1024.

[0092] In the bit quantization process of the above convolutional layer, weight kernel quantization 1028 can be performed using the following equation.

[0093]

[0094] Among them, a f is the weight value to be quantized, such as a real number weight and each weight in the weight kernel, k represents the number of bits to be quantized, a q Represents a f The result after quantization with k bits. That is, according to the above formula, first let a f Multiply by a predetermined binary number 2 k , thus a f k bits are added, hereinafter referred to as the "first value". Next, the first value is rounded or truncated to remove a f The number after the decimal point is referred to as the "second value" below. The second value is divided by a binary number 2 k , the number of bits is reduced by k bits, so that the element value of the final quantized weight core can be calculated. Such weight or weight core quantization 1028 is repeatedly performed on all element values ​​of the weight or weight core 1014 to generate quantized weight values ​​1018.

[0095] Meanwhile, feature map or activation map quantization 1030 may be performed by the following equation.

[0096] a f =a f ·clip(-11)

[0097]

[0098] In the feature map or activation map quantization 1030, the same formula as the weight or weight kernel quantization 1028 method can be used. However, in the feature map or activation map quantization, each element value a of the feature map or activation map 1022 can be added by applying clipping before applying quantization to each element value. f The process of normalizing a real number (e.g., a feature map or activation map) to a value between 0 and 1.

[0099] Next, the normalized value a f Multiply by a predetermined binary number 2 k , thus a f k bits are added, hereinafter referred to as the "first value". Next, the first value is rounded or truncated to remove a f The number after the decimal point is referred to as the "second value" below. The second value is divided by a binary number 2 k , the number of bits is reduced by k bits, so that the element values ​​of the final quantized feature map or activation map 1026 can be calculated. Such quantization 1030 of the feature map or activation map is repeatedly performed on all element values ​​of the feature map or activation map 1022 to generate the quantized feature map or activation map 1026.

[0100] Through the described weight or weight kernel quantization 1028 and feature map or activation map quantization 1030, the storage size and computational effort required for the convolution operation of the convolution layer 420 of the convolutional neural network can be reduced in bit units.

[0101] Fig.11 1 is a flowchart showing a method of quantizing bits of an artificial neural network according to an embodiment of the present disclosure. This embodiment is an example in which the unit of a data group that can be quantized in an artificial neural network is assumed to be all parameters belonging to each layer constituting the artificial neural network.

[0102] As shown in the figure, the bit quantization method 1100 of the artificial neural network can start by selecting at least one layer from multiple layers included in the artificial neural network S1110. The layer to be selected from the multiple layers included in the artificial neural network can be determined based on the influence of the selected layer on the overall performance, calculation amount or storage amount of the artificial neural network. In one embodiment, in the above reference Figure 1-3 In the multi-layer artificial neural network described in , we can arbitrarily select the layer that has the greatest impact on the overall performance or computational complexity of the artificial neural network. Figure 4-10 In the described convolutional artificial neural network (CNN) 400, since the convolutional layer 420 and / or the fully connected layer 440 have a great impact on the overall performance or computational complexity of the CNN 400, at least one of these layers 420 and 440 may be selected.

[0103] The method of selecting at least one layer among the multiple layers included in the artificial neural network may be determined according to the influence of the selected layer on the overall performance or the amount of computation of the artificial neural network. However, the present disclosure is not limited thereto and may include one of various methods. For example, selecting at least one layer from the multiple layers included in the artificial neural network may be performed according to the following methods: (i) a method of sequentially selecting layers from the first layer receiving input data to subsequent layers according to the arrangement order of the multiple layers constituting the artificial neural network, (ii) a method of sequentially selecting layers from the last layer generating final output data to the front layer according to the arrangement order of the multiple layers constituting the artificial neural network, (iii) a method of selecting the layer with the highest amount of computation among the multiple layers constituting the artificial neural network, or (iv) a method of selecting the layer with the least amount of computation from the multiple layers constituting the artificial neural network.

[0104] When the layer selection of the artificial neural network is completed in S1110, the operation may proceed to step S1120 to reduce the data representation size of the parameters (eg, the weight of the selected layer) to a bit unit.

[0105] In one embodiment, when the weight or size of the output data in the parameters of the selected layer is reduced to a bit unit, a reference may be performed. Figure 4-10 The weight kernel quantization 1028 and activation map quantization 1024 described above can be calculated by the following equation.

[0106]

[0107] Here, a f represents the element value of the weight kernel to be quantized, such as a real weight kernel coefficient, k represents the number of bits to be quantized, a q Indicates a f The result of k-bit quantization. That is, according to the above equation, first, a f Multiply by a predetermined binary number 2 k , thus a f Add k bits, i.e., the "first value". Next, remove a by rounding or truncating the first value. f The number after the decimal point is the "second value". The second value is divided by the binary number 2 k , the number of bits is reduced by k bits, so that the element value of the final quantized weight core can be calculated. Such weight core quantization 1028 is repeatedly performed on all element values ​​of the weight core 1014 to generate a quantized weight core 1018.

[0108] Meanwhile, activation map quantization 1030 may be performed by the following equation.

[0109] a f =a f ·clip(-1, 1)

[0110]

[0111] In activation map quantization 1030, clipping is applied before quantizing each element value (e.g., real coefficient) of activation map 1022, thereby adding a process of normalizing each element value of activation map 1022 to a value between 0 and 1. Next, the normalized af is multiplied by a predetermined binary number 2 k , thus a f Add k bits, i.e., the "first value". Next, remove a by rounding or truncating the first value. f The number after the decimal point is the "second value". The second value is divided by the binary number 2 k , the number of bits is reduced by k bits, so that the element values ​​of the final quantized activation map 1026 can be calculated. This quantization 1030 of the activation map is repeatedly performed on all element values ​​of the activation map 1022, and the quantized activation map 1026 is generated.

[0112] In the above-mentioned embodiments, an example of reducing the number of bits of weight values ​​or activation map data has been described to reduce the size of the data representation of the parameters of the layer selected in the artificial neural network. However, the bit quantization method of the present disclosure is not limited to this. In another embodiment, different bits can be allocated to the data in the intermediate stages between multiple calculation steps of various data contained in the selected layer in the artificial neural network. Therefore, when implemented in the hardware of the artificial neural network, in order to reduce the memory (such as a buffer, register or cache) storing various data, the number of bits of each data stored in the corresponding memory can be reduced and the number of bits of the corresponding memory can be reduced. In another embodiment, the size of the data bits of the data path can be reduced in bit units, and the data of the selected layer in the artificial neural network is transmitted through the data path.

[0113] After executing step S1120, step S1130 may be performed to determine whether the accuracy of the artificial neural network is equal to or greater than a predetermined target value. If the accuracy of the output result of the artificial neural network (e.g., the training result or inference result of the artificial neural network) is greater than the predetermined target value after the data representation size of the parameters of the selected layer of the artificial neural network is reduced to a bit unit, it can be expected that the overall performance of the artificial neural network can be maintained even if the bits of the data are additionally reduced.

[0114] Accordingly, when it is determined in step S1130 that the accuracy of the artificial neural network is greater than or equal to the target value, the process proceeds to step S1120 to further reduce the data representation size of the selected layer into a bit unit.

[0115] In step S1130, if the accuracy of the artificial neural network is not higher than the target value, it can be determined that the accuracy of the artificial neural network has decreased due to the currently performed bit quantization. Therefore, in this case, the minimum number of bits that meets the accuracy target value in the bit quantization performed immediately before can be determined as the final number of bits of the parameter of the selected layer, that is, step S1140.

[0116] Next, it is determined whether the bit quantization of all layers of the artificial neural network has been completed, that is, step S1150. In this step, if it is determined that the bit quantization of all layers of the artificial neural network has been completed, the whole process ends. On the other hand, if there are still layers in the layers of the artificial neural network that have not been bit quantized, step S1110 is executed to perform bit quantization on the corresponding layers.

[0117] Here, in step S1110, selecting other layers from the multiple layers included in the artificial neural network can be performed according to the following methods: (i) a method of sequentially selecting the next layer of the previously selected layer according to the arrangement order of the multiple layers constituting the artificial neural network, i.e., "forward bit quantization", (ii) a method of selecting the previous layer of the previously selected layer in the reverse direction according to the arrangement order of the multiple layers constituting the artificial neural network, i.e., "reverse bit quantization", (iii) a method of selecting a layer with higher computational amount after the previously selected layer among the multiple layers constituting the artificial neural network according to the order of computational amount, i.e., "high computational cost bit quantization", or (iv) a method of selecting a layer with lower computational amount after the previously selected layer among the multiple layers constituting the artificial neural network according to the order of computational amount, i.e., "low computational cost bit quantization".

[0118] In one embodiment, the accuracy of the artificial neural network may refer to the probability that the artificial neural network provides a solution to a given problem (e.g., identifying an object included in an image as input data) in the inference phase after learning the solution to the problem. In addition, the target value used in the above-mentioned bit quantization method may represent the minimum accuracy to be maintained after the bit quantization of the artificial neural network. For example, assuming that the target value is 90% accuracy, even after the parameters of the layer selected by the bit quantization are reduced in bit units, if the accuracy of the artificial neural network is 90% or higher, additional bit quantization may be performed. For example, after the first bit quantization is performed, if the accuracy of the artificial neural network is measured to be 94%, additional bit quantization may be performed. After the second bit quantization is performed, if the accuracy of the artificial neural network is measured to be 88%, the number of bits determined by the first bit quantization, the result of the currently performed bit quantization is abandoned, and the number of bits determined by the first bit quantization (i.e., the number of bits representing the corresponding data) is determined as the final bit quantization result.

[0119] In one embodiment, in an artificial neural network including a plurality of layers according to a computational cost bit quantization method, when a layer is selected to perform bit quantization based on the computational amount among the plurality of layers, the computational amount of each layer may be determined as follows. That is, when an addition operation performs the addition of n bits and m bits at a specific layer of the artificial neural network, the computational amount is calculated as (n+m) / 2. In addition, when a specific layer of the artificial neural network multiplies n bits and m bits, the computational amount of the corresponding operation may be calculated as n×m. Accordingly, the computational amount of a specific layer of the artificial neural network may be the result of summing the computational amounts of all additions and multiplications performed by the layer.

[0120] In addition, according to the computational cost bit quantization method, a method of performing bit quantization by selecting one layer from multiple layers based on the computational amount in the artificial neural network is not limited to Fig.11 , and various modifications are possible.

[0121] In another embodiment, for Fig.11 The bit quantization of the parameters of each layer in the illustrated embodiment can be performed separately for the weights and activation maps. For example, quantization is first performed on the weights of the selected layer, whereby the weights have n bits. Separately, by performing bit quantization on the output activation data of the selected layer, the number of bits representing the activation map data can be determined to be m bits. Alternatively, quantization can be performed when the same bits are allocated to the weights and activation map data of the corresponding layer, whereby the same bit n can be used for the weights and activation map data.

[0122] Fig.12 is a flowchart illustrating a bit quantization method of an artificial neural network according to another embodiment of the present disclosure.

[0123] As shown in the figure, the bit quantization method 1200 of the artificial neural network can start by selecting the layer with the highest computational complexity among the multiple layers included in the artificial neural network S1210.

[0124] When the layer selection of the artificial neural network is completed in step S1210, the operation can proceed to step S1220 of reducing the size of the data representation of the parameters of the selected layer to a bit unit. In one embodiment, when the size of the data of the selected layer is reduced to a bit unit, a reference can be performed. Figure 4-10 Described weight kernel quantization 1028 and activation map quantization 1024.

[0125] After executing step S1220, step S1230 may be performed to determine whether the accuracy of the artificial neural network reflecting the bit quantization result so far is greater than or equal to the predetermined target value. If it is determined in step S1230 that the accuracy of the artificial neural network is greater than or equal to the target value, the data size of the corresponding layer is set to the current bit quantization result, and after proceeding to step S1210, steps S1210 to S1230 may be repeatedly performed. That is, by proceeding to step S1210, the calculation amount of all layers in the artificial neural network is recalculated, and on this basis, the layer with the largest calculation amount is selected again.

[0126] In step S1230, if the accuracy of the artificial neural network is not higher than the target value, the bit reduction quantization of the currently selected layer is canceled, and the layer is excluded from the layer target selectable in the layer selection step S1210. Then, in step S1240, a layer with a larger amount of calculation after the corresponding layer can be selected. Next, in step S1250, the data size of the selected layer can be reduced to a bit unit.

[0127] In step S1260, it is determined whether the accuracy of the artificial neural network reflecting the bit quantization result so far is greater than or equal to the target value. If the accuracy of the artificial neural network is not higher than the target value, it is determined whether the bit quantization of all layers of the artificial neural network is completed (S1270). If it is determined in step S1270 that the bit quantization of all layers of the artificial neural network is completed, the entire bit quantization process ends. On the other hand, if it is determined in step S1270 that the bit quantization of all layers of the artificial neural network is not completed, the process can proceed to step S1240.

[0128] If it is determined in step S1260 that the accuracy of the artificial neural network is greater than or equal to the target value, the process proceeds to step 1220 to perform subsequent steps.

[0129] Fig.13 is a flowchart illustrating a bit quantization method of an artificial neural network having multiple layers according to another embodiment of the present disclosure.

[0130] As shown in the figure, the bit quantization method 1300 of the artificial neural network with multiple layers includes steps S1310 to S1350, and searches for precision fluctuation points for each layer of all layers included in the artificial neural network. The method 1300 first fixes the bit size of the data of all layers included in the artificial neural network to a maximum value, and selects a layer in which the search for precision fluctuation points is not performed (S1310).

[0131] When the layer selection of the artificial neural network is completed in step S1310, the process may proceed to step S1320 of reducing the size of the data of the selected layer to a bit unit. In one embodiment, when the size of the data of the selected layer is reduced to a bit unit, reference may be performed. Figure 4-10 Described weight kernel quantization 1028 and activation map quantization 1024.

[0132] After performing step S1320, step S1330 of determining whether the accuracy of the artificial neural network reflecting the bit quantization result so far is greater than or equal to a predetermined target value may be performed. If it is determined in step S1330 that the accuracy of the artificial neural network is greater than or equal to the target value, the process proceeds to step S1320 to perform additional bit reduction quantization on the currently selected layer.

[0133] In step S1330, if the accuracy of the artificial neural network is not higher than the target value, the number of data bits of the currently selected layer is set to the minimum number of bits that most recently meets the target value. Thereafter, it is determined whether the search for accuracy change points of all layers of the artificial neural network has been completed (S1340). In this step, if the search for accuracy change points of all layers has not been completed, the process can proceed to step S1310. In step S1310, another layer is selected in which the bit size of data of all layers included in the artificial neural network is the largest, and the search for performance change points is not performed.

[0134] If it is determined in step S1340 that the search for the accuracy change points of all layers of the artificial neural network has been completed, the bit quantization results corresponding to the accuracy change points of each layer of the artificial neural network can be reflected in the artificial neural network S1350. In one embodiment, in step S1350, the corresponding layer is set to the bit size of the data immediately before the accuracy change point of each layer of the artificial neural network determined according to the above steps S1310 to S1350 (for example, the point where the accuracy of the artificial neural network in each layer decreases).

[0135] In another embodiment, in step S1350, the corresponding layer is set to a size greater than the resource required for the calculation of the parameters immediately before the precision change point of each layer of the artificial neural network determined according to the above steps S1310 to S1340. For example, the number of bits of the parameters of each layer of the artificial neural network can be set to be 2 bits greater than the number of bits immediately before the precision change point. Then, the bit quantization method S1360 is performed on the artificial neural network for which the data size of each layer is set in step 1350. The bit quantization method performed in step S1360 may include, for example Fig.11 or Fig.12 The method shown.

[0136] The bit quantization method of the artificial neural network according to the various embodiments described above is not limited to being performed on the weight kernel and feature map or activation map of each layer in the multiple layers of the artificial neural network. In one embodiment, the bit quantization method of the present disclosure first executes the weight kernels or weights of all layers of the artificial neural network, and then bit quantization can be performed on the feature maps of all layers of the artificial neural network reflecting the quantization of the weight kernels. In another embodiment, the feature maps of all layers of the artificial neural network can be bit quantized first, and then the kernels of all layers of the artificial neural network reflecting the quantization of the feature maps can be bit quantized.

[0137] In addition, the bit quantization method of the artificial neural network disclosed in the present invention is not limited to applying the same degree of bit quantization to the weight core of each layer of the artificial neural network. In one embodiment, the bit quantization method disclosed in the present invention can perform bit quantization in units of the weight core of each layer of the artificial neural network, or can perform separate bit quantization so that each weight unit as an element of each weight core has different bits.

[0138] Hereinafter, examples of execution results of a method for quantizing bits of an artificial neural network according to various embodiments of the present disclosure will be described with reference to the accompanying drawings.

[0139] Fig.14 is a graph showing an example of the amount of computation per layer of an artificial neural network according to an embodiment of the present disclosure. Fig.14 The artificial neural network shown is an example of a convolutional artificial neural network of a VGG-16 model including 16 layers, and each layer of the artificial neural network has a different amount of calculation.

[0140] For example, since the second, fourth, sixth, seventh, ninth, and tenth layers have the highest computational cost, when following the high computational cost bit quantization method, bit quantization can be applied first. In addition, after performing bit quantization on the second, fourth, sixth, seventh, ninth, and tenth layers, bit quantization can be performed on the 14th layer, which has the second highest computational cost.

[0141] Fig.15 is a graph showing the number of bits per layer of an artificial neural network in which bit quantization has been performed by a forward bit quantization method according to an embodiment of the present disclosure.

[0142] As described above, forward quantization is a method of sequentially performing bit quantization from the earliest layer (eg, from the layer that first receives input data) based on the arrangement order of multiple layers included in the artificial neural network. Fig.15 Shown in Fig.14The number of bits per layer of the artificial neural network of the VGG-16 model shown in the figure after applying forward vectorization, and the reduction rate of the amount of computation of the artificial neural network by forward vectorization. For example, when performing n-bit and m-bit addition, the amount of computation of the corresponding calculation is calculated as (n+m) / 2. In addition, when performing n-bit and m-bit multiplication, the amount of computation of the corresponding calculation can be calculated as n×m. Therefore, the total amount of computation of the artificial neural network can be the result of summing up all the amounts of computation of addition and multiplication performed by the artificial neural network.

[0143] As shown in the figure, when the bit quantization is performed on the artificial neural network of the VGG-16 model using forward quantization, the number of bits of the layers arranged on the front side of the artificial neural network is relatively reduced more, while the number of bits of the layers arranged on the back side of the artificial neural network is relatively reduced less. For example, the bit number of the first layer of the artificial neural network is reduced to 12 bits, the number of bits of the second and third layers is reduced to 9 bits, and the 16th layer is reduced to 13 bits, and the number of bits of the 15th layer is only reduced to as much as 15 bits. As described above, when forward quantization is sequentially applied from the first layer to the 16th layer of the artificial neural network, the reduction rate of the total computation amount of the artificial neural network is calculated to be 56%.

[0144] Fig.16 is a graph showing the number of bits per layer of an artificial neural network in which bit quantization has been performed by an inverse bit quantization method according to an embodiment of the present disclosure.

[0145] Inverse quantization is a method of sequentially performing bit quantization from the last layer (eg, from a layer that finally outputs output data) based on the arrangement order of multiple layers included in the artificial neural network. Fig.16 Shown in Fig.14 The number of bits per layer of the artificial neural network of the VGG-16 model shown in Figure 1 after inverse quantization is applied, and the reduction rate of the computational complexity of the artificial neural network by inverse quantization.

[0146] As shown in the figure, when the bit quantization is performed on the artificial neural network of the VGG-16 model using inverse quantization, the number of bits of the layers arranged on the rear side of the artificial neural network is relatively reduced more, while the number of bits of the layers arranged on the front side of the artificial neural network is relatively reduced less. For example, the number of bits of the first layer, the second layer, and the third layer are reduced to 15 bits, respectively, the number of bits of the fourth layer is reduced to 14 bits, and the number of bits of the 16th layer is reduced to 9 bits, and the number of bits of the 15th layer is reduced to 15 bits. As described above, when inverse quantization is sequentially applied from the first layer to the 16th layer of the artificial neural network, the reduction rate of the total computation amount of the artificial neural network is calculated to be 43.05%.

[0147] Fig.17is a graph showing the number of bits per layer of an artificial neural network according to an embodiment of the present disclosure, wherein bit quantization is performed by a high computational cost layer first bit quantization method.

[0148] High computation layer first quantization or high computation quantization is a method of sequentially performing bit quantization from a layer with a higher computation load among multiple layers included in an artificial neural network. Fig.17 It shows that applying high computational quantization to Fig.14 The number of bits per layer and the reduction rate of the amount of computation of the artificial neural network after high computational quantization of the artificial neural network are shown.

[0149] As shown in the figure, when the artificial neural network of the VGG-16 model is bit quantized using high computational quantization, the number of bits of the layers with high computational load in the multiple layers of the artificial neural network is relatively reduced more. For example, the number of bits of the second and tenth layers is reduced to 5 and 6 bits, respectively, while the number of bits of the first layer is reduced to 14 bits. In this way, when high computational quantization is applied to the layers of the artificial neural network in the order of computational load, the computational load reduction rate of the entire artificial neural network is calculated to be 70.70%.

[0150] Fig.18 is a graph showing the number of bits per layer of an artificial neural network according to an embodiment of the present disclosure, wherein bit quantization is performed by a low computational cost bit quantization method.

[0151] Low-computation layer-first quantization or low-computation quantization is a method of sequentially performing bit quantization from a layer with a lower computational load among multiple layers included in an artificial neural network. Fig.18 Shows Fig.14 The number of bits per layer of the artificial neural network of the VGG-16 model shown after low-computation quantization is applied and the reduction rate of the computational complexity of the artificial neural network by low-computation quantization.

[0152] As shown in the figure, even when the artificial neural network of the VGG-16 model is bit quantized using low computational quantization, the number of bits of the high computational layers in the multiple layers of the artificial neural network is relatively reduced. For example, the number of bits of the sixth and seventh layers is reduced to 6 and 5 bits, respectively, while the number of bits of the first layer is reduced to 13 bits. In this way, when low computational quantization is applied to the layers of the artificial neural network in the order of computational amount, the reduction rate of the computational amount of the entire artificial neural network is calculated to be 49.11%.

[0153] Hereinafter, a hardware implementation example of an artificial neural network to which bit quantization is applied according to various embodiments of the present disclosure will be described in detail. When a convolutional artificial neural network including multiple layers is implemented in hardware, a weight kernel may be arranged outside and / or inside a processing unit for performing convolution of the convolutional layer.

[0154] In one embodiment, the weight kernel may be stored in a memory (e.g., a register, a buffer, a cache, etc.) separate from a processing unit that performs convolution of the convolution layer. In this case, after bit quantization is applied to the weight kernel to reduce the number of bits of the element value of the weight kernel, the size of the storage may be determined according to the number of bits of the weight kernel. In addition, the bit width of a multiplier or adder may be set according to the number of bits depending on the bit quantization result, and the multiplier or adder is arranged in a processing unit that performs multiplication and / or addition operations by receiving the element value of the weight kernel stored in the memory and the element value of the input feature map.

[0155] In another embodiment, the weight core can be implemented in a convolution processing unit for performing a convolution layer in a hardwired form. In this case, after applying bit quantization to the weight core to reduce the number of bits of the element value of the weight core, hard wiring representing each element value of the weight core can be implemented in the processing unit according to the number of bits of the weight core. In addition, the bit size of the multiplier or adder can be set according to the number of bits from the bit quantization result, and the multiplier or adder is arranged in a processing unit that receives the element value of the hardwired weight core and the element value of the input feature map to perform multiplication and / or addition operations.

[0156] Described below Figures 19 to 21 An example of a hardware implementation of an artificial neural network including multiple layers according to another embodiment of the present disclosure is shown. The method and system for bit quantization of an artificial neural network including multiple layers according to the present disclosure can reduce the required amount of calculation, the bit size of operators, and the memory by applying the present disclosure to any artificial neural network (ANN) computing system (e.g., CPU, GPU, FPGA, and ASIC). In addition, in this example, an implementation has been shown based on integers, but floating-point operations can also be performed.

[0157] Fig.19 An example of a hardware implementation of an artificial neural network according to an embodiment of the present disclosure is shown. The illustrated artificial neural network shows an example of a convolution multiplication processing device 1900 for a convolution layer of a convolutional artificial neural network implemented in hardware. Here, the convolution layer will be described by assuming that the convolution is performed by applying a weight kernel of size 3x3x3 to a portion of the input feature map, i.e., data of size 3x3x3. The size and number of weight kernels for each layer may vary depending on the application field and the number of input / output feature map channels.

[0158] As shown, the weight kernel can be stored in a weight kernel cache 1910, which is separated from a processing unit 1930 for performing convolution of the convolution layer. In this case, after bit quantization is applied to the weight kernel to reduce the number of bits of the element value (w1, w2, ..., w9) of the weight kernel, the size of the cache can be determined according to the number of bits of the weight kernel. In addition, the bit size of the multiplier or adder can be set according to the number of bits of the weight kernel element value resulting from the bit quantization, and the multiplier or adder is arranged in the processing unit 1930 that receives the element value of the weight kernel stored in the memory and the element value of the input feature map and performs multiplication and / or addition operations.

[0159] According to the embodiment, the input feature map buffer 1920 can receive and store a portion of the input data, that is, a portion corresponding to the size of the weight kernel. The weight kernel traverses the input data, and the input feature map buffer 1920 can sequentially receive and store a portion of the input data corresponding to the position of the weight kernel. A portion of the input data (x 1, x 2, ..., x 9) stored in the input feature map buffer 1920 and some element values ​​(w 1, w2, ..., w 9) of the weight kernel stored in the weight kernel buffer 1910 are respectively input to the corresponding multiplier 1932 to perform element-by-element multiplication. The result value of the element-by-element multiplication by the multiplier 1932 is summed by the tree adder 1934 and input to the adder 1940. When the input data consists of multiple channels, for example, when the input data is an RGB color image, the adder 1940 can add the value stored in the accumulator 1942 (initial value is 0) to the sum value of the input specific channel and store it in the accumulator 1942 again. The sum value stored in the accumulator 1942 may be input to the accumulator 1942 by adding it back to the sum value of the adder 1940 of the next channel. The summing process of the adder 1940 and the accumulator 1942 is performed on all channels of the input data, and all the sum values ​​may be input to the output activation map buffer 1950. The above convolution process may be repeated for the weight kernel corresponding to the traversed position in the weight kernel input data and the partial input data.

[0160] As described above, when the element values ​​of the weight core are stored in the weight core cache 1910 arranged outside the processing unit 1930, the number of bits of the weight core element values ​​can be reduced by bit quantization according to the present disclosure. Therefore, the size of the weight core cache 1910 and the size of the multiplier and adder of the processing unit 1930 can be reduced. In addition, as the size of the processing unit 1930 is reduced, the calculation speed and power consumption of the processing unit 1930 are also reduced.

[0161] Fig. 20 An example of hardware implementation of an artificial neural network according to another embodiment of the present disclosure is shown.

[0162] The illustrated artificial neural network shows an example of a convolution multiplication processing device 2000 that implements the convolution layer of the convolutional artificial neural network in hardware. Here, the convolution layer performs convolution by applying a weight kernel of size 3x3x3 to a portion of the input activation map (i.e., data of size 3x3x3).

[0163] As shown, the weight core can be stored in a weight core cache 2010 separated from the processing unit 2030 for performing convolution of the convolution layer. In this case, after bit quantization is applied to the weight core, the number of bits of the element value (w1, w2, ..., w9) of the weight core is reduced, and then the size of the cache can be determined according to the number of bits of the weight core. In addition, the bit size of the multiplier or adder arranged in the processing unit 2030 can be set according to the number of bits of the weight core element value obtained by bit quantization, and the processing unit 2030 receives the element value of the weight core stored in the memory and the element value of the input activation map or feature map and performs multiplication and / or addition operations.

[0164] According to the embodiment, the input activation map cache 2020 can receive and store a portion of the input data consisting of multiple channels (e.g., three RGB channels), that is, a portion corresponding to the size of the weight kernel. The weight kernel traverses the input data, and the input activation map cache 2020 can sequentially receive and store the portion of the input data corresponding to the position of the weight kernel. Each portion of the input data (x1, x2, ..., x27) stored in the input activation map cache 2020 and each element value (w1, w2, ..., w27) of the weight kernel cache 2010 are input to the corresponding multiplier to perform element-by-element multiplication. At this time, the kernel element value (w1, w2, ..., w9) of the weight kernel cache 2010 and a portion of the first channel (x1, x2, ..., x9) of the input data stored in the input activation map weight cache 2020 are input to the first convolution processing unit 2032. In addition, the weight kernel element values ​​(w10, w11, ..., w18) of the weight kernel buffer 2010 and the second channel input data (x10, x11, ..., x18) stored in the input activation map buffer 2020 are input to the second convolution processing unit 2034. In addition, the weight kernel element values ​​(w19, w20, ..., w27) of the weight kernel buffer 2010 and a portion of the third channel of the input data (x10, x11, ..., x18) stored in the input activation map buffer 2020 are input to the third convolution processing unit 2036.

[0165] Each of the first convolution processing unit 2032, the second convolution processing unit 2034, and the third convolution processing unit 2036 can be Fig.19The processing unit 1930 shown in FIG. 10 operates in the same manner. The result value of the convolution calculated by each of the first convolution processing unit 2032, the second convolution processing unit 2034, and the third convolution processing unit 2036 can be summed by the tree adder 2038 and input to the output activation map buffer 2040.

[0166] As described above, when the element values ​​of the weight core are stored in the weight core cache 2010 arranged outside the processing unit 2030, the number of bits of the weight core element values ​​can be reduced by bit quantization according to the present disclosure. Therefore, the size of the weight core cache 2010 and the size of the multiplier and adder of the processing unit 2030 can be reduced. In addition, as the size of the processing unit 2030 is reduced, the calculation speed and power consumption of the processing unit 2030 are also reduced.

[0167] Fig.21 An example of hardware implementation of an artificial neural network according to another embodiment of the present disclosure is shown.

[0168] The illustrated artificial neural network shows an example of a convolution multiplication processing device 2200 that implements the convolution layer of the convolutional artificial neural network in hardware. Here, the convolution layer performs convolution by applying a weight kernel of size 3x3x3 to a portion of the input activation map (i.e., data of size 3x3x3).

[0169] As shown in the figure, the weight core can be implemented in a hard-wired form in a processing unit 2220 for performing convolution of a convolution layer. In this case, after bit quantization is applied to the weight core to reduce the number of bits of the element value (w1K, w2K, ..., w27K) of the weight core, the size of the cache can be determined according to the number of bits of the weight core. In addition, the bit size of the multiplier or adder arranged in the processing unit 2030 can be set according to the number of bits of the weight core element value obtained by bit quantization, and the processing unit 2030 receives the element value of the weight core implemented as a connection in the processing unit 2220 and the element value of the input activation map or feature map and performs multiplication and / or addition operations.

[0170] According to one embodiment, the input activation map buffer 2210 may receive and store a portion of input data consisting of a plurality of channels (e.g., three RGB channels), the portion corresponding to the size of the weight kernel. The weight kernel traverses the input data, and the input activation map buffer 2210 may sequentially receive and store a portion of the input data corresponding to the position of the weight kernel. The portion of the input data (x 1, x 2, ..., x 27) stored in the input activation map buffer 2210 and the element values ​​(w 1K, w 2K, ..., w 27K) of the weight kernel implemented as a line in the processing unit 2220 are respectively input to the corresponding multipliers to perform element-by-element multiplication. In this case, the weight kernel element values ​​(w 1K, w 2K, ..., w 9K) implemented as a line in the processing unit 2220 and a portion of the first channel (x 1, x 2, ..., x 9) of the input data stored in the input activation map buffer 2210 are input to the first convolution processing unit 2222. In addition, the weight kernel element value implemented as a line in the processing unit 2220 and a portion of the second channel of the input data stored in the input activation map buffer 2210 are input to the second convolution processing unit 2224. In addition, the weight kernel element value (w 19K, w 20K, ..., w 27K) of the weight kernel buffer 2210 and a portion of the third channel of the input data stored in the input activation map buffer 2210 (x 19, x 20, ..., x 27) are input to the third convolution processing unit 2226.

[0171] The convolution results calculated by each of the first convolution processing unit 2222 , the second convolution processing unit 2224 , and the third convolution processing unit 2226 may be summed by the tree adder 2228 and input to the output activation map buffer 2230 .

[0172] As described above, when the element values ​​of the weight core are implemented in a hard-wired form in the processing unit 2220, the number of bits of the weight core element values ​​can be reduced by bit quantization according to the present disclosure. Therefore, there is an effect of reducing the number of wires implemented therein and the size of the multipliers and adders of the processing unit 2220. Moreover, as the size of the processing unit 2220 decreases, the calculation speed and power consumption of the processing unit 2220 may also decrease.

[0173] Fig. 22 A configuration of a system for performing bit quantization on an artificial neural network according to an embodiment of the present disclosure is shown.

[0174] As shown, the system 2300 may include a parameter selection module 2310, a bit quantization module 2320, and an accuracy determination module 2330. The parameter selection module 2310 may analyze the configuration information of the input artificial neural network. The configuration information of the artificial neural network may include the number of layers included in the artificial neural network, the function and role of each layer, the information of the input / output data of each layer, the type and number of multiplications and additions performed by each layer, the type of activation function performed by each layer, the type and configuration of the weight kernel input to each layer, the size and number of the weight kernel of each layer, the size of the output feature map, the initial value of the weight kernel (e.g., the element value of the weight kernel set incorrectly), etc., but is not limited thereto. According to the type of the artificial neural network (e.g., convolutional artificial neural network, recurrent artificial neural network, multilayer perceptron, etc.), the configuration information of the artificial neural network may include information of various elements.

[0175] The parameter selection module 2310 can refer to the input artificial neural network configuration information and select at least one parameter or parameter group to be quantized from the artificial neural network. How to select a parameter or data or parameter group in the artificial neural network can be determined based on the influence of the selected parameter on the overall performance or the amount of calculation or the amount of resources required for the hardware implementation of the artificial neural network. The selection of parameters can be performed by selecting from a weight, a feature map and activation map, a weight kernel, all weights in a layer, all feature maps or activation maps in a layer.

[0176] In one embodiment, with reference to the above Figures 4 to 10 In the case of the described convolutional artificial neural network (CNN) 400, since the convolutional layer 420 and / or the fully connected layer 440 have a great influence on the overall performance or computational amount of the CNN 400, a weight kernel or a feature map / activation map of at least one of these layers 420 and 440 can be selected as a parameter to be quantized.

[0177] In one embodiment, at least one of the multiple layers included in the artificial neural network can be selected, and all weight kernels in the layer or all activation map data of the layer can be set as a parameter group. The selection method can be determined based on the influence of the selected layer on the overall performance or computational amount of the artificial neural network, but is not limited thereto, and can include one of various methods. For example, selecting at least one layer from the multiple layers included in the artificial neural network can be performed according to the following methods: (i) a method of sequentially selecting layers from the first layer receiving input data to subsequent layers according to the arrangement order of the multiple layers constituting the artificial neural network, (ii) a method of sequentially selecting layers from the last layer generating the final output data to the front layer according to the arrangement order of the multiple layers constituting the artificial neural network, (iii) a method of selecting the layer with the highest computational amount among the multiple layers constituting the artificial neural network, or (iv) a method of selecting the layer with the least computational amount from the multiple layers constituting the artificial neural network.

[0178] When the parameter selection module 2310 completes the selection of the data target for quantization of the artificial neural network, the information of the selected data is input to the bit quantization module 2320. The bit quantization module 2320 can reduce the data representation size of the corresponding parameter to the bit unit by referring to the input information of the selected parameter. The resources required for the operation of the selected parameter may include a memory for storing the selected parameter or a data path for transmitting the selected parameter, but is not limited thereto.

[0179] In one embodiment, when the bit quantization module 2320 reduces the data size of the selected parameter to the bit unit, a reference Figures 4 to 13 Describes weight kernel quantization and / or activation map quantization.

[0180] When the bit quantization module 2320 completes the bit quantization of the selected parameters, it transmits the bit quantized artificial neural network information to the accuracy determination module 2330. The accuracy determination module 2330 can reflect the bit quantized artificial neural network information in the configuration information of the artificial neural network input to the system 2300. The bit quantization module 2320 can determine whether the accuracy of the artificial neural network is greater than or equal to a predetermined target value based on the configuration information of the artificial neural network in which the bit quantized artificial neural network information is reflected. For example, after reducing the size of the data representing the selected parameters in the artificial neural network to a bit unit, if the accuracy of the output result of the artificial neural network (e.g., the inference result of the artificial neural network) is greater than the predetermined target value, the accuracy determination module 2330 can predict that the overall performance of the artificial neural network can be maintained even when additional bit quantization is performed.

[0181] Therefore, when the accuracy determination module 2330 determines that the accuracy of the artificial neural network is greater than or equal to the target value, a control signal is transmitted to the parameter selection module 2310 so that the parameter selection module 2310 selects another parameter or parameter group included in the artificial neural network. Here, the method of selecting a parameter in the artificial neural network can be performed according to the following methods: (i) a method of sequentially selecting the next parameter of the previously selected parameter according to the arrangement order of each parameter or parameter group constituting the artificial neural network ("forward bit quantization"), (ii) a method of selecting the previous parameter of the previously selected parameter in the reverse direction according to the arrangement order of the parameters or parameter groups constituting the artificial neural network ("reverse bit quantization"), (iii) a method of selecting a parameter with a higher computational amount after the previously selected parameter according to the order of computational amount among the multiple parameters constituting the artificial neural network ("high computational cost bit quantization"), or (iv) a method of selecting a parameter with a lower computational amount after the previously selected parameter according to the order of computational amount among the multiple parameters constituting the artificial neural network ("low computational cost bit quantization").

[0182] On the other hand, if the accuracy determination module 2330 determines that the accuracy of the artificial neural network is not greater than or equal to the target value, it can be determined that the accuracy of the artificial neural network is reduced due to the bit quantization performed on the currently selected parameters. Therefore, in this case, the number of bits determined by the bit quantization performed immediately before can be determined as the final number of bits. In one embodiment, the accuracy of the artificial neural network can refer to the probability that the artificial neural network provides a solution to a given problem (e.g., identifying an object included in an image as input data) in the inference stage. In addition, the target value used in the above-mentioned bit quantization method can represent the minimum accuracy to be maintained after the bit quantization of the artificial neural network. For example, assuming that the target value is 90% accuracy, even if the storage size for storing the parameters of the layer selected by the bit quantization is reduced to a bit unit, if the accuracy of the artificial neural network is 90% or higher, additional bit quantization can be performed. For example, after the first bit quantization is performed, if the accuracy of the artificial neural network is measured to be 94%, additional bit quantization can be performed. After performing the second bit quantization, if the accuracy of the artificial neural network is measured to be 88%, the result of the currently performed bit quantization can be ignored, and the number of bits represented by the data determined by the first bit quantization is determined as the final bit quantization result.

[0183] In one embodiment, according to the computational cost bit quantization method, when a parameter or parameter group is selected based on the computational amount to perform bit quantization, the computational amount of each parameter can be determined as follows. That is, when performing the sum of n bits and m bits in a specific operation of the artificial neural network, the computational amount of the corresponding operation is calculated as (n+m) / 2. In addition, in a specific operation of the artificial neural network, when n bits and m bits are multiplied, the computational amount of the corresponding operation can be calculated as n×m. Therefore, the computational amount of a specific parameter of the artificial neural network can be the result of summing the computational amounts of all additions and multiplications performed on the parameter.

[0184] In this bit quantization, a selection method of a specific parameter or parameter group can be selected as weight data or feature map and activation map data belonging to each layer, or each weight kernel belonging to a layer, or a separate parameter group for each weight data in a weight kernel.

[0185] For reference, according to the embodiments of the present disclosure, Fig. 22 The elements shown may be implemented as software or hardware elements, for example, a field programmable gate array (FPGA) or an application specific integrated circuit (ASIC).

[0186] However, the “element” is not meant to be limited to software or hardware, and each element may be configured in an addressable storage medium or may be configured to serve as one or more processors.

[0187] Thus, as an example, elements include, for example, software elements, object-oriented software elements, class elements, and task elements, as well as procedures, functions, properties, procedures, subroutines, program code segments, drivers, firmware, microcode, circuits, data, databases, data structures, tables, arrays, and variables.

[0188] The elements and the functionality provided within the elements may be combined into a smaller number of elements or further separated into additional elements.

[0189] Embodiments of the present disclosure can also be implemented in the form of a recording medium including instructions that can be executed by a computer (e.g., a program module executed by a computer). Computer-readable media can be any available media that a computer can access, and include volatile and non-volatile media, removable and non-removable media. In addition, computer-readable media can include computer storage media and communication media. Computer storage media include volatile and non-volatile, removable and non-removable media implemented by any method or technology for storing information such as computer-readable instructions, data structures, program modules or other data. Communication media typically include other data in computer-readable instructions, data structures, program modules or modulated data signals (e.g., carrier waves, or other transmission mechanisms), and include any information transfer media.

[0190] Although the present disclosure has been described in conjunction with some embodiments, it should be understood that various modifications and changes can be made without departing from the scope of the present disclosure as understood by those skilled in the art. In addition, such modifications and changes should be deemed to fall within the scope of the claims appended to this specification.

Claims

1. A method for quantizing bits of input data of each of a plurality of layers included in an artificial neural network, wherein the input data is image data or video data, the method comprising: (a) selecting at least one layer among the multiple layers of the artificial neural network through a parameter selection module; (b) bit quantizing by a bit quantization module to reduce the size of the data representation of the parameter of the selected layer to a bit unit; (c) determining, by an accuracy determination module, whether the accuracy of the artificial neural network is greater than or equal to a predetermined target value; and (d) if the accuracy of the artificial neural network is greater than or equal to the target value, repeating steps (b) to (c) for the selected layer; (e) if the precision of the artificial neural network is less than the target value, determining a final size of the data representation of the parameters of the selected layer, wherein the final size of the data representation is the minimum number of bits resulting from the previous execution of step (b) such that the precision of the artificial neural network is greater than or equal to the target value; and (f) performing steps (a) to (e) for unquantized layers among the plurality of layers.

2. The method according to claim 1, The artificial neural network is a convolutional neural network, and the multiple layers include multiple convolutional layers and multiple fully connected layers. wherein (a) the selecting comprises selecting at least one of the plurality of convolutional layers, and Wherein (b) the bit quantization includes bit quantization to reduce the size of the data representation of at least one element value in the weight kernel and feature map of the selected convolutional layer to a bit unit.

3. The method according to claim 2, wherein (a) the selecting comprises selecting at least one layer from the plurality of fully connected layers, and Wherein (b) the bit quantization includes reducing the size of the data representation of the element value of the weight kernel of the selected fully connected layer to a bit unit.

4. The method according to claim 1, Wherein (b) the bit quantization includes: Clip the value of the parameter to between 0 and 1; calculating a first value by multiplying the clipped value by a predetermined binary number; calculating a second value by rounding or truncating the first value; and A quantization parameter is calculated by dividing the second value by the binary number.

5. The method according to claim 4, Also included is removing a decimal point of the first value using the rounding or truncation.

6. The method according to claim 1, Wherein (a) the selecting comprises selecting one of the multiple layers according to the position order of the multiple layers.

7. The method according to claim 6, Wherein (a) the selecting includes selecting at least one layer from the multiple layers by using one of a method of sequentially selecting from a first layer to subsequent layers that receive input data according to an arrangement order of the multiple layers, and a method of sequentially selecting from a last layer to a front layer that generates output data according to an arrangement order of the multiple layers.

8. The method according to claim 1, The options described in (a) include: determining a computational effort for each of the plurality of layers; as well as A layer of the plurality of layers is selected in order of the determined computation amount.

9. The method according to claim 8, Wherein (a) the selecting includes selecting at least one layer from the plurality of layers by using one of a method of selecting a layer having the highest computational complexity from the plurality of layers and a method of selecting a layer having the lowest computational complexity from the plurality of layers.

10. The method according to claim 1, Also included is training the artificial neural network according to parameters of the plurality of layers by using the data representation reduced in bit units.

11. The method according to claim 1, Also included is performing an inference process of the artificial neural network based on parameters of the plurality of layers using the data representation reduced in bits.

12. The method according to claim 1, The bit quantization includes setting the size of the data representation of all weights of the multiple layers to n bits, where n is an integer of n>0, and setting the size of the data representation of the output data of the multiple layers to m bits, where m is an integer of m>0.

13. The method according to claim 12, Also included is individually and differently allocating the number of bits of weights and output data in the parameters of the multiple layers.

14. The method according to claim 1, The bit quantization includes allocating the size of the data representation of the weights and output data in the parameters of the multiple layers to n bits, where n is an integer of n>0, and setting the number of bits allocated to the data representation of each layer in the multiple layers to be different.

15. The method according to claim 2, The bit quantization further includes allocating a different number of bits to each weight kernel of the multiple convolutional layers of the convolutional neural network.

16. The method according to claim 1, The bit quantization includes allocating the number of bits of the data representation of the weight differently to each weight in the parameters of the multiple layers.

17. The method according to claim 1, The bit quantization includes individually and differently allocating the number of bits representing a specific unit of output data in the parameters of the plurality of layers.

18. The method according to claim 1, The bit quantization includes allocating differently the number of bits representing the data of each value of the output data in the parameters of the multiple layers.

19. A method for quantizing bits of input data of each of a plurality of layers included in an artificial neural network, the input data being image data or video data, the method comprising: (a) selecting one or more data or one or more groups of data from the weights, feature maps and activation map data of the artificial neural network through a parameter selection module; (b) bit quantizing the selected data to reduce the data representation size to a bit unit by a bit quantization module; (c) measuring whether the artificial intelligence accuracy of the artificial neural network is greater than or equal to a target value; and (d) if the accuracy of the artificial neural network is greater than or equal to the target value, repeating steps (b) to (c) for the selected data; (e) if the precision of the artificial neural network is less than the target value, determining a final size of the data representation of the selected data, wherein the final size of the data representation is the minimum number of bits resulting from the previous execution of step (b) such that the precision of the artificial neural network is greater than or equal to the target value; as well as (f) performing steps (a) to (e) for the unquantized data until there is no more data in the artificial neural network that needs to be quantized.

20. The method according to claim 19, The selected data includes a weight kernel, and (b) the bit quantization includes reducing the size of the data representation of the element value of the weight kernel to a bit unit.

21. The method according to claim 19, Wherein the selected data includes weights, and (b) the bit quantization includes reducing the size of the data representation of the weights to bit units.

22. A method for quantizing bits of an artificial neural network, the method comprising: training the artificial neural network according to one or more parameters of the artificial neural network; According to the method according to any one of claims 1-21, bit quantization is performed on one or more parameters of the artificial neural network; and the artificial neural network is trained based on the one or more parameters of the artificial neural network on which bit quantization is performed.

23. A non-transitory computer-readable storage medium having recorded therein a program including instructions for executing each instruction according to the bit quantization method for an artificial neural network according to any one of claims 1 to 22.

24. A bit quantization system for input data of each of a plurality of layers included in an artificial neural network, wherein the input data is image data or video data, the system comprising: A parameter selection module, which is used to select at least one parameter from a plurality of parameters of the artificial neural network; a bit quantization module for reducing the size of the data representation of the selected parameter to bit units; and an accuracy determination module, which is used to determine whether the accuracy of the artificial neural network is greater than or equal to a predetermined target value, wherein when the accuracy of the artificial neural network is greater than or equal to the target value, the accuracy determination module controls the bit quantization module to repeatedly perform bit quantization for the selected parameter; wherein when the precision of the artificial neural network is less than the target value, the precision determination module determines a final size of the data representation of the selected parameter, wherein the final size of the data representation is a minimum number of bits generated by previously performing the bit quantization so that the precision of the artificial neural network is greater than or equal to the target value; as well as The precision determination module controls the parameter selection module and the bit quantization module so as to perform bit quantization on another parameter of the plurality of parameters whose final size of data representation has not been determined.

25. The bit quantization system according to claim 24, The artificial neural network is a convolutional neural network comprising a plurality of layers, and the plurality of layers comprises a plurality of convolutional layers and a plurality of fully connected layers, wherein the parameter selection module selects at least one of the plurality of convolutional layers, and in, The bit quantization module reduces the size of the data representation of at least one element value in the weight kernel and feature map of the selected convolution layer to a bit unit.

26. The bit quantization system according to claim 25, wherein the parameter selection module selects at least one layer of the plurality of fully connected layers, and in, The bit quantization module reduces the size of the data representation of the element values ​​of the weight kernel of the selected fully connected layer to a bit unit.

27. The bit quantization system according to claim 24, wherein the bit quantization module clips the parameter to a value between 0 and 1, calculating a first value by multiplying the clipped value by a predetermined binary number, calculating a second value by rounding or truncating the first value; and A quantization parameter is calculated by dividing the second value by the binary number.

28. The bit quantization system according to claim 27, The bit quantization module uses the rounding or truncation to remove a decimal point of the first value.

29. The bit quantization system according to claim 24, The parameter selection module selects one of the multiple parameters according to the position order of the multiple parameters.

30. The bit quantization system according to claim 29, The parameter selection module selects at least one layer from the multiple layers by using a method of sequentially selecting a first layer to subsequent layers that receive input data according to the arrangement order of the multiple layers constituting the neural network, and a method of sequentially selecting a last layer to a front layer that generates output data according to the arrangement order of the multiple layers constituting the neural network.

31. The bit quantization system according to claim 24, The parameter selection module determines a calculation amount of each of the plurality of parameters; and selects one of the plurality of parameters in an order of the determined calculation amounts.

32. The bit quantization system according to claim 31, The parameter selection module selects at least one layer from the plurality of layers by using a method of selecting a layer with the highest computational complexity from the plurality of layers constituting the neural network and a method of selecting a layer with the lowest computational complexity from the plurality of layers.

33. The bit quantization system according to claim 24, The system trains the artificial neural network according to the plurality of parameters by using a data representation reduced in bit units.

34. The bit quantization system according to claim 24, The system performs an inference process of the artificial neural network based on the plurality of parameters using a data representation reduced in bit units.

35. The bit quantization system according to claim 25, The system allocates different numbers of bits to the weight kernels of the multiple convolutional layers of the convolutional neural network.

36. A bit quantization system for input data of each of a plurality of layers included in an artificial neural network, the input data being image data or video data, the system comprising: A parameter selection module, configured to select at least one layer from the plurality of layers constituting the artificial neural network; a bit quantization module for reducing the size of the data representation of the parameters of the selected layer to a bit unit; as well as an accuracy determination module, which is used to determine whether the accuracy of the artificial neural network is greater than or equal to a predetermined target value, wherein, if the accuracy of the artificial neural network is equal to or greater than the target value, the accuracy determination module controls the bit quantization module to repeatedly perform bit quantization for the selected layer; wherein, if the accuracy of the artificial neural network is less than the target value, the accuracy determination module determines a final size of the data representation of the parameter of the selected layer, wherein the final size of the data representation is a minimum number of bits generated by previously performing the bit quantization so that the accuracy of the artificial neural network is greater than or equal to the target value; and The precision determination module controls the parameter selection module and the bit quantization module to perform bit quantization on another layer of the multiple layers until all layers of the multiple layers have been quantized.

Images