Network quantization method and network quantization device

Abstract

A network quantization method that quantizes a neural network (14) and includes: a database construction step in which a statistical information database for tensors handled by the neural network (14) is created; a parameter generation step in which a quantization parameter set (22) is generated by quantizing tensor values on the basis of the statistical information database (18) and the neural network (14); and a network construction step in which a quantization network (26) is constructed by using the quantization parameter set (22) and quantizing the neural network (14). The parameter generation step includes a quantization type determination step in which the quantization type for each of a plurality of layers that constitute the neural network is decided.

Description

[Technical field]

[0001] The present disclosure relates to a network quantization method and a network quantization device. [Background technology]

[0002] Traditionally, machine learning has been performed using networks such as neural networks. Here, a model that takes numerical data as input, performs some kind of calculation to obtain a numerical output value is called a network. When implementing a network in hardware such as a computer, in order to reduce hardware costs, it is necessary to build a network with lower calculation accuracy while maintaining the inference accuracy after implementation at the same level as floating-point accuracy.

[0003] For example, implementing a network that performs all calculations with floating-point precision would result in high hardware costs, so there is a need to realize a network that performs calculations with fixed-point precision while maintaining inference accuracy.

[0004] In the following, the network with floating-point precision is also referred to as a pre-quantization network, and the network with fixed-point precision is also referred to as a quantization network.

[0005] Here, quantization refers to the process of dividing floating-point values, which can continuously represent almost any value, into predetermined divisions and encoding them. More generally, quantization is defined as the process of reducing the number of digits or range of numbers handled by a network.

[0006] When expressing real numbers with a limited number of bits by quantization, the distribution of input data may differ from the expected distribution. In this case, the quantization error becomes large, which has a negative effect on the speed of machine learning and even the accuracy of inference after training.

[0007] As a method for solving such a problem, for example, the method described in Patent Document 1 is known. In the method described in Patent Document 1, an individual fixed-point format is defined for each of the weights and data in each layer of the convolutional neural network. Machine learning of the convolutional neural network is started with floating-point numbers, and the input data distribution is analyzed to estimate it. Next, an optimized number format that represents the input data value is determined based on the input data distribution, and quantization is performed using the format. In this way, Patent Document 1 attempts to solve the above problem by first examining the distribution of the input data and selecting a number format suitable for the distribution. [Prior art documents] [Patent documents]

[0008] [Patent Document 1] JP 2018-10618 A Summary of the Invention [Problem to be solved by the invention]

[0009] In the method described in Patent Document 1, the dynamic range of the data to be handled is taken into consideration, and a limited number of bits is allocated to a range in which the data fits. In this case, depending on the characteristics of the data, the number of bits may not be used effectively. For example, the amount of meaningful data may be small relative to the number of bits. In this way, bit allocation may be inefficient.

[0010] Therefore, the present disclosure has been made to solve such problems, and has an object to provide a network quantization method and the like that can construct a quantization network with efficient bit allocation. [Means for solving the problem]

[0011] In order to achieve the above-mentioned object, a network quantization method according to one embodiment of the present disclosure is a network quantization method for quantizing a neural network, comprising: a preparation step of preparing the neural network; a database construction step of constructing a statistical information database of tensors handled by the neural network obtained when multiple test data sets are input to the neural network; a parameter generation step of generating a quantization parameter set by quantizing values ​​of the tensor based on the statistical information database and the neural network; and a network construction step of constructing a quantization network by quantizing the neural network using the quantization parameter set, wherein the parameter generation step includes a quantization type determination step of determining a quantization type for each of multiple layers constituting the neural network.

[0012] In order to achieve the above object, a network quantization device according to one embodiment of the present disclosure is a network quantization device that quantizes a neural network, and includes a database construction unit that constructs a statistical information database of tensors handled by the neural network obtained when multiple test data sets are input to the neural network, a parameter generation unit that generates a quantization parameter set by quantizing values ​​of the tensor based on the statistical information database and the neural network, and a network construction unit that constructs a quantization network by quantizing the neural network using the quantization parameter set, and the parameter generation unit determines the quantization type of each of multiple layers that constitute the neural network. Effect of the Invention

[0013] The present disclosure can provide a network quantization method capable of constructing a quantization network with efficient bit allocation. [Brief description of the drawings]

[0014] [Figure 1] FIG. 1 is a block diagram showing an outline of the functional configuration of a network quantization device according to the first embodiment. [Diagram 2] FIG. 2 is a diagram illustrating an example of a hardware configuration of a computer that realizes the functions of the network quantization device according to the first embodiment by software. [Diagram 3] FIG. 3 is a flowchart showing a flow of the network quantization method according to the first embodiment. [Figure 4] FIG. 4 is a flowchart showing the flow of the quantization parameter set generating method according to the first embodiment. [Diagram 5] FIG. 5 is a table showing an example of the relationship between the redundancy and a suitable quantization type according to the first embodiment. [Figure 6] FIG. 6 is a graph for explaining the ternary conversion of floating-point precision numbers. [Figure 7] FIG. 7 is a block diagram showing an outline of a functional configuration of a network quantization device according to the second embodiment. As shown in FIG. [Figure 8] FIG. 8 is a flowchart showing a flow of the network quantization method according to the second embodiment. [Figure 9] FIG. 9 is a flowchart showing a flow of the parameter generating step according to the second embodiment. [Figure 10] FIG. 10 is a flowchart showing a flow of the quantization type determining step according to the second embodiment. [Figure 11] FIG. 11 is a graph illustrating pseudo-ternarization of a numerical value with floating-point precision. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0015] Hereinafter, the embodiments of the present disclosure will be described in detail with reference to the drawings. Note that each of the embodiments described below shows a specific example of the present disclosure. The numerical values, shapes, materials, standards, components, the arrangement and connection forms of the components, steps, and the order of steps shown in the following embodiments are merely examples and are not intended to limit the present disclosure. In addition, among the components in the following embodiments, components that are not described in the independent claims showing the highest concept of the present disclosure are described as optional components. In addition, each figure is not necessarily a strict illustration. In each figure, substantially the same configuration is given the same symbol, and duplicated explanations may be omitted or simplified.

[0016] (Embodiment 1) A network quantization method and a network quantization device according to a first embodiment will be described.

[0017] [1-1. Network quantization device] First, the configuration of a network quantization device according to this embodiment will be described with reference to Fig. 1. Fig. 1 is a block diagram showing an outline of the functional configuration of a network quantization device 10 according to this embodiment.

[0018] The network quantization device 10 is a device that quantizes the neural network 14. That is, the network quantization device 10 is a device that converts the neural network 14 with floating-point precision into a quantized network that is a neural network with fixed-point precision. The network quantization device 10 does not need to quantize all tensors handled by the neural network 14, and only needs to quantize at least some of the tensors. Here, a tensor is a value expressed as an n-dimensional array (n is an integer equal to or greater than 0) including parameters such as input data, output data, and weights in each of a plurality of layers that constitute the neural network 14. Here, the plurality of layers that constitute the neural network 14 include an input layer to which a signal of the neural network 14 is input, an output layer to which a signal is output from the neural network 14, and a hidden layer to which a signal is transmitted between the input layer and the output layer.

[0019] The tensor may include parameters related to the smallest unit of operation in the neural network 14. If the neural network 14 is a convolutional neural network, the tensor may include weights and bias values, which are functions defined as convolutional layers. In addition, the tensor may include parameters such as normalization processing in the neural network 14.

[0020] 1, network quantization device 10 includes a database construction unit 16, a parameter generation unit 20, and a network construction unit 24. In this embodiment, network quantization device 10 further includes a machine learning unit .

[0021] The database construction unit 16 is a processing unit that constructs a statistical information database 18 of tensors handled by the neural network 14 obtained when multiple test data sets 12 are input to the neural network 14. The database construction unit 16 calculates the redundancy of each tensor handled by the neural network 14 for the multiple test data sets 12, and constructs the statistical information database 18 of each tensor. The statistical information database 18 includes the redundancy of tensors included in each of the multiple layers that constitute the neural network 14. In the database construction unit 16, the redundancy of a tensor may be determined based on the result of tensor decomposition, for example. The redundancy of a tensor will be described later. In addition, the statistical information database 18 may include at least a part of statistics such as the mean value, median, mode, maximum value, minimum value, maximum value, minimum value, variance, deviation, skewness, and kurtosis of each tensor.

[0022] The parameter generating unit 20 is a processing unit that generates a quantization parameter set by quantizing the value of a tensor based on the statistical information database 18 and the neural network 14. The parameter generating unit 20 determines a quantization type for each of a plurality of layers constituting the neural network 14. The quantization type may be selected from a plurality of numerical conversion types that perform different numerical conversions on tensors. The plurality of numerical conversion types include, for example, logarithmic conversion and no conversion. The quantization type may be selected from a plurality of resolution types that differ in the resolution of quantization. The plurality of resolution types include, for example, N-bit fixed point (N: an integer of 2 or more), ternary, and the like. The parameter generating unit 20 determines the quantization type based on the redundancy of the tensor included in each of a plurality of layers constituting the neural network 14. The parameter generating unit 20 quantizes the value of the tensor using the determined quantization type. Detailed processing contents of the parameter generating unit 20 will be described later.

[0023] The network constructing unit 24 is a processing unit that constructs a quantization network 26 by quantizing the neural network 14 using the quantization parameter set 22 .

[0024] The machine learning unit 28 is a processing unit that causes the quantization network 26 to undergo machine learning. The machine learning unit 28 causes machine learning by inputting a plurality of test data sets 12 or other input data sets to the quantization network 26 constructed by the network construction unit 24. In this way, the machine learning unit 28 constructs a quantization network 30 having better inference accuracy than the quantization network 26. Note that the network quantization device 10 does not necessarily have to include the machine learning unit 28.

[0025] With the above-described configuration, network quantization device 10 can construct a highly accurate quantization network.

[0026] [1-2. Hardware configuration] Next, the hardware configuration of network quantization device 10 according to the present embodiment will be described with reference to Fig. 2. Fig. 2 is a diagram showing an example of the hardware configuration of a computer 1000 that realizes the functions of network quantization device 10 according to the present embodiment by software.

[0027] 2, the computer 1000 includes an input device 1001, an output device 1002, a CPU 1003, an internal storage 1004, a RAM 1005, a reading device 1007, a transmission / reception device 1008, and a bus 1009. The input device 1001, the output device 1002, the CPU 1003, the internal storage 1004, the RAM 1005, the reading device 1007, and the transmission / reception device 1008 are connected by the bus 1009.

[0028] The input device 1001 is a user interface device such as an input button, a touch pad, a touch panel display, etc., and accepts user operations. The input device 1001 may be configured to accept voice operations and remote operations using a remote control or the like in addition to accepting touch operations by the user.

[0029] The output device 1002 is a device that outputs a signal from the computer 1000, and may be a device that serves as a user interface, such as a signal output terminal, a display, or a speaker.

[0030] The built-in storage 1004 is a flash memory, etc. Furthermore, the built-in storage 1004 may store in advance at least one of a program for implementing the functions of the network quantization device 10 and an application that utilizes the functional configuration of the network quantization device 10.

[0031] The RAM 1005 is a random access memory, and is used to store data and the like when a program or application is executed.

[0032] The reading device 1007 reads information from a recording medium such as a USB (Universal Serial Bus) memory. The reading device 1007 reads the above-mentioned programs and applications from a recording medium on which the programs and applications are recorded, and stores the programs and applications in the built-in storage 1004.

[0033] The transmitting / receiving device 1008 is a communication circuit for performing wireless or wired communication. The transmitting / receiving device 1008 communicates with, for example, a server device connected to a network, downloads the above-mentioned programs and applications from the server device, and stores them in the built-in storage 1004.

[0034] The CPU 1003 is a central processing unit that copies programs, applications, etc. stored in the internal storage 1004 to the RAM 1005, and sequentially reads out instructions included in the copied programs, applications, etc. from the RAM 1005 and executes them.

[0035] [1-3. Network quantization method] Next, the network quantization method according to the present embodiment will be described with reference to Fig. 3. Fig. 3 is a flowchart showing the flow of the network quantization method according to the present embodiment.

[0036] As shown in Fig. 3, in the network quantization method, first, a neural network 14 is prepared (S10). In this embodiment, a pre-trained neural network 14 is prepared. The neural network 14 is not quantized, that is, it is a neural network with floating-point precision. Note that input data used in training the neural network 14 is not particularly limited, and may include a plurality of test data sets 12 shown in Fig. 1.

[0037] Next, the database construction unit 16 constructs a statistical information database of tensors handled by the neural network 14, which are obtained when multiple test data sets 12 are input to the neural network 14 (S20). In this embodiment, the database construction unit 16 calculates the redundancy of tensors included in each of multiple layers constituting the neural network 14, and constructs a statistical information database 18 including the redundancy of each tensor. In this embodiment, the redundancy is determined based on the result of tensor decomposition of each tensor. The method of calculating the redundancy will be described later.

[0038] Next, the parameter generating unit 20 generates a quantization parameter set 22 by quantizing the tensor values ​​based on the statistical information database 18 and the neural network 14 (S30). The parameter generating step S30 includes a quantization type determining step for determining the quantization type of each of the multiple layers that make up the neural network 14. The quantization type determining step will be described later.

[0039] Next, the network constructing unit 24 constructs the quantization network 26 by quantizing the neural network 14 using the quantization parameter set 22 (S40).

[0040] Next, the machine learning unit 28 causes the quantization network 26 to undergo machine learning (S50). The machine learning unit 28 causes machine learning by inputting a plurality of test data sets 12 or other input data sets to the quantization network 26 constructed by the network construction unit 24. This makes it possible to construct a quantization network 30 having better inference accuracy than the quantization network 26. Note that the network quantization method according to this embodiment does not necessarily have to include the machine learning step S50.

[0041] As described above, according to the network quantization method of the present embodiment, a neural network can be quantized with high accuracy.

[0042] [1-4. Redundancy] Next, the tensor redundancy calculated in the database construction unit 16 will be described. The tensor redundancy is a measure corresponding to the ratio of the amount of information of a tensor that can be reduced while suppressing the decrease in the inference accuracy of the neural network 14 within a predetermined range. In this embodiment, the tensor redundancy is a measure obtained by focusing on the semantic structure of the tensor (i.e., the principal components), and is expressed as the ratio of the amount of information of components (i.e., components far from the principal components) that can be removed while suppressing the reconstruction error correlated with the inference accuracy of the neural network 14 within a predetermined range to the amount of information of the original tensor.

[0043] An example of a method for calculating the redundancy of a tensor will be described below.

[0044] A J-dimensional tensor (a J-dimensional multidimensional array (J is an integer equal to or greater than 2)) can be decomposed into a K-dimensional core tensor (K is an integer equal to or greater than 1 and smaller than J) and J factor matrices using mathematical techniques. Specifically, such tensor decomposition corresponds to solving an optimization problem of approximating a J-dimensional tensor with a K-dimensional tensor. This means that if some noise components are ignored, the relevant J-dimensional tensor can be roughly approximated with a K-dimensional tensor and a factor matrix. In other words, to express the original J-dimensional tensor, it is sufficient to have a degree of complexity sufficient to express each component of the K-dimensional tensor. The value (JK) / J obtained by performing the tensor decomposition in this manner is defined as redundancy. Note that the definition of redundancy is not limited to this. For example, K / J may be defined as redundancy.

[0045] Here, an example of a method of tensor decomposition will be described. As the tensor decomposition, for example, CP decomposition, Tucker decomposition, etc. can be used. For example, as shown in the following formula (1), the J-dimensional tensor W can be approximated by the product of a K-dimensional core tensor U and a factor matrix V by the CP decomposition.

[0046]

number

[0047] In this case, the reconstruction error RecErr correlated with the inference accuracy of the neural network 14 can be expressed as a value obtained by normalizing the difference between the L2 norm of the original tensor and the L2 norm of the restored tensor obtained by restoring the core tensor to the shape of the original tensor, using the L2 norm of the original tensor. In other words, the reconstruction error RecErr can be calculated using the following formula (2).

[0048]

number

[0049] Therefore, the redundancy (K / J) can be obtained by performing tensor decomposition while keeping the reconstruction error RecErr within a predetermined range.

[0050] Similarly, when Tucker decomposition is used as the tensor decomposition, the reconstruction error RecErr can be calculated based on the original tensor W and the core tensor C by the following formula (3).

[0051]

number

[0052] As described above, the redundancy of the tensors included in each of the multiple layers that make up the neural network 14 can be found.

[0053] [1-5. Parameter generation section] Next, a method for generating the quantization parameter set 22 in the parameter generating unit 20 according to this embodiment will be described in detail.

[0054] As described above, the parameter generation unit 20 generates a quantization parameter set by quantizing tensor values ​​based on the statistical information database 18 and the neural network 14. Hereinafter, a method for generating a quantization parameter set in the parameter generation unit 20 will be described with reference to Fig. 4. Fig. 4 is a flowchart showing the flow of the quantization parameter set generation method according to this embodiment.

[0055] As shown in FIG. 4, in the quantization parameter set generation method according to the present embodiment, first, the quantization type of each tensor of a plurality of layers constituting the neural network 14 is determined (S31). In the present embodiment, the quantization type is determined based on the redundancy included in the statistical information database 18. In the present embodiment, before the generation of the quantization parameter set, the relationship between the redundancy and the suitable quantization type is obtained using another neural network as a sample model. Such a relationship between the redundancy and the suitable quantization type will be described with reference to FIG. 5. FIG. 5 is a table showing an example of the relationship between the redundancy and the suitable quantization type according to the present embodiment. In the example shown in FIG. 5, when the redundancy of a tensor is 0.3, the quantization type of the tensor is determined to be 8-bit fixed point (FIX8). Also, when the redundancy of a tensor is 0.4, the quantization type of the tensor is determined to be 6-bit fixed point (FIX6). Also, when the redundancy of a tensor is 0.7, the quantization type of the tensor is determined to be ternary (TERNARY). In this way, in the quantization type determination step S31, a lower resolution quantization type may be selected as the redundancy of the tensor increases. This allows a lower resolution quantization type to be selected while suppressing a decrease in the inference accuracy of the quantization network 26. By selecting a lower resolution quantization type in this way, the hardware cost when implementing the quantization network can be suppressed. This method of determining the relationship between the redundancy and the suitable quantization type in advance by using another neural network as a sample model is particularly effective when the type of the neural network 14 to be quantized and the other neural network to be the sample model are similar. For example, when the neural network 14 is an object detection neural network, a quantization type suitable for the neural network 14 can be selected by using the other object detection neural network as a sample model.

[0056] In addition, in the quantization type determination step S31, each numerical value included in the tensor may be converted nonlinearly. The numerical value conversion type for the tensor in the quantization type may be selected from a plurality of numerical value conversion types including logarithmic conversion and no conversion. For example, when the frequency of the tensor value is particularly high near 0, all elements of the tensor may be logarithmically converted. In other words, all elements of the tensor may be converted to the logarithm of the numerical value. This makes it possible to increase the redundancy of the tensor when the frequency of all elements of the tensor is high in a range close to 0.

[0057] Also, in the quantization type determination step S31, the resolution of the quantization in the quantization type may be selected from among a plurality of resolution types including N-bit fixed point and ternary.

[0058] Next, quantization is performed on the tensors of each of the layers constituting the neural network 14 (S32). Specifically, for example, when quantization with N-bit fixed-point precision is used as the quantization type, the values ​​constituting each tensor are quantized to N-bit fixed-point precision.

[0059] As another example of the quantization type, the case of using ternarization will be described with reference to Fig. 6. Fig. 6 is a graph for explaining the ternarization of a numerical value with floating-point precision. The horizontal axis of the graph shown in Fig. 6 indicates the numerical value with floating-point precision to be quantized (the "original Float value" shown in Fig. 6), and the vertical axis indicates the value after ternarization.

[0060] As shown in Fig. 6, when ternary quantization is used as the quantization type, among numerical values ​​with floating-point precision, numerical values ​​equal to or less than a predetermined first value a are quantized to -1, numerical values ​​greater than the first value a and equal to or less than a predetermined second value b are quantized to 0, and numerical values ​​greater than the second value b are quantized to +1. When such ternary quantization is used as the quantization type, multiplication can be realized by an XOR operation in a convolution operation in the quantization network, etc. This makes it possible to reduce the hardware resources for implementing the quantization network.

[0061] As described above, a quantization parameter set can be generated by quantizing a tensor.

[0062] [1-6. Effects, etc.] As described above, the network quantization method according to the present embodiment is a network quantization method for quantizing a neural network 14, and includes a preparation step of preparing the neural network 14, a database construction step of constructing a statistical information database 18 of tensors handled by the neural network 14 obtained when a plurality of test data sets 12 are input to the neural network, a parameter generation step of generating a quantization parameter set 22 by quantizing tensor values ​​based on the statistical information database 18 and the neural network 14, and a network construction step of constructing a quantization network 26 by quantizing the neural network 14 using the quantization parameter set 22. The parameter generation step includes a quantization type determination step of determining a quantization type for each of a plurality of layers constituting the neural network.

[0063] In this way, since the quantization type can be selected for each of the multiple layers constituting the neural network 14, it is possible to efficiently allocate bits according to the characteristics of each layer, and therefore it is possible to construct a quantization network with efficiently allocated bits.

[0064] In addition, in the network quantization method according to this embodiment, in the quantization type determination step, the quantization type is selected from a plurality of numerical transformation types that perform different numerical transformations on tensors, and the plurality of numerical transformation types may include logarithmic transformation and no transformation.

[0065] This allows us to select a method of numerical transformation of a tensor according to the distribution of values ​​contained in the tensor. For example, by applying a numerical transformation that increases the redundancy of the tensor, more efficient bit allocation becomes possible. Therefore, we can build a quantization network with more efficient bit allocation.

[0066] In addition, in the network quantization method of this embodiment, in the quantization type determination step, the quantization type is selected from a plurality of resolution types having different quantization resolution, and the plurality of resolution types may include N-bit fixed point and ternary.

[0067] This allows the fineness of quantization to be selected according to the redundancy of the tensor, etc. Therefore, quantization can be performed for each layer so as to suppress a decrease in the inference accuracy of the quantized network.

[0068] In addition, in the network quantization method according to the present embodiment, the quantization type may be determined based on the redundancy of the tensors included in each of the multiple layers.

[0069] In general, the greater the redundancy of a tensor, the lower the resolution of quantization can be adopted while suppressing the deterioration of inference accuracy. Therefore, by determining the quantization type based on the redundancy, it is possible to adopt low-resolution quantization while suppressing the deterioration of inference accuracy. By lowering the resolution of quantization in this way, the cost of hardware for implementing the quantization network can be reduced.

[0070] In the network quantization method according to the present embodiment, the redundancy may be determined based on the result of tensor decomposition of the tensor.

[0071] In addition, in the network quantization method according to the present embodiment, the quantization type may be determined to be a type with lower resolution as the redundancy increases.

[0072] This makes it possible to employ low-resolution quantization while suppressing degradation of inference accuracy.

[0073] Moreover, the network quantization device according to this embodiment is a network quantization device 10 that quantizes a neural network 14, and includes a database construction unit 16 that constructs a statistical information database 18 of tensors handled by the neural network 14 obtained when a plurality of test data sets 12 are input to the neural network 14, a parameter generation unit 20 that generates a quantization parameter set 22 by quantizing the values ​​of the tensors based on the statistical information database 18 and the neural network 14, and a network construction unit 24 that constructs a quantization network 26 by quantizing the neural network 14 using the quantization parameter set 22. The parameter generation unit 20 determines the quantization type of each of a plurality of layers that constitute the neural network 14.

[0074] This provides the same effects as the network quantization method according to the present embodiment.

[0075] (Embodiment 2) A network quantization method according to the second embodiment will be described. The network quantization method according to the present embodiment differs from the quantization method according to the first embodiment in the quantization type determination method. The following describes the network quantization method and the network quantization device according to the present embodiment, focusing on the differences from the first embodiment.

[0076] [2-1. Network quantization device] First, the configuration of a network quantization device according to this embodiment will be described with reference to Fig. 7. Fig. 7 is a block diagram showing an outline of the functional configuration of network quantization device 110 according to this embodiment.

[0077] 7, network quantization device 110 includes database construction unit 16, parameter generation unit 120, and network construction unit 24. In this embodiment, network quantization device 110 further includes machine learning unit 28. Network quantization device 110 according to this embodiment differs from network quantization device 10 according to embodiment 1 in parameter generation unit 120.

[0078] The parameter generating unit 120 according to this embodiment, like the parameter generating unit 20 according to the first embodiment, generates the quantization parameter set 22 by quantizing the value of a tensor based on the statistical information database 18 and the neural network 14. The parameter generating unit 120 also determines the quantization type of each of the layers constituting the neural network 14. The parameter generating unit 120 according to this embodiment determines the quantization type based on the redundancy of the tensors of the layers constituting the neural network 14 and the redundancy of the tensor after quantization. Specifically, the quantization type is determined based on the redundancy of the tensor included in the statistical information database 18 and the redundancy of the quantized tensor obtained by quantizing the tensor. The redundancy of the quantized tensor is calculated, for example, in the parameter generating unit 120.

[0079] [2-2. Network quantization method] Next, a network quantization method according to this embodiment and an inference method using the same will be described with reference to Fig. 8. Fig. 8 is a flowchart showing the flow of the network quantization method according to this embodiment.

[0080] As shown in FIG. 8, the network quantization method of this embodiment, like the network quantization method of embodiment 1, includes a neural network 14 preparation step S10, a database construction step S20 for constructing a statistical information database 18, a parameter generation step S130 for generating a quantization parameter set 22, a network construction step S40 for constructing a quantization network, and a machine learning step S50 for training the quantization network 26 to machine learn.

[0081] The network quantization method according to the present embodiment differs from the network quantization method according to the first embodiment in parameter generating step S130.

[0082] The parameter generating step S130 according to the present embodiment will be described with reference to Fig. 9. Fig. 9 is a flowchart showing the flow of the parameter generating step S130 according to the present embodiment. The parameter generating step S130 according to the present embodiment includes a quantization type determining step S131 and a quantization execution step S32, similar to the parameter generating step S30 according to the first embodiment. The parameter generating step S130 according to the present embodiment differs from the parameter generating step S30 according to the first embodiment in the quantization type determining step S131.

[0083] The quantization type determining step S131 according to this embodiment will be described with reference to Fig. 10. Fig. 10 is a flowchart showing the flow of the quantization type determining step S131 according to this embodiment.

[0084] As shown in Fig. 10, in the quantization type determination step S131 according to the present embodiment, first, a type of numerical conversion for tensors in the quantization type is determined (S131a). For example, the numerical conversion type for tensors in the quantization type is selected from a plurality of numerical conversion types including logarithmic conversion. In the present embodiment, the numerical conversion type is selected from (a) logarithmic conversion, (b) pseudo ternary, and (c) uniform quantization (no conversion).

[0085] When deciding on each type of numerical transformation, the following characteristics of the element distribution related to the principal components of the tensor are taken into consideration:

[0086] (a) When the element distribution related to the principal component is concentrated near 0 In this case, logarithmic quantization, which has dense quantization steps around 0, is advantageous.

[0087] (b) When the element distribution associated with the principal component is not near 0 In this case, it is advantageous to quantize by eliminating information around 0, i.e. making it 0. For example, pseudo-ternary quantization is an example.

[0088] (c) When the element distribution associated with the principal component does not fall into either (a) or (b) above In this case uniform quantization is advantageous.

[0089] The above element distribution can be calculated by, for example, repeatedly performing histogram calculations, which require a large amount of calculations. In this embodiment, in order to reduce the amount of calculations, a method of performing numerical conversion in the cases of (a) and (b) and finding redundancy is adopted as an example of a method for simply determining the type of numerical conversion at the above-mentioned focus point.

[0090] A method for selecting a type of numerical conversion according to the present embodiment will be described. The parameter generating unit 120 calculates the redundancy R of a tensor for which a quantization type is to be determined, and the redundancy R of a tensor obtained by performing a logarithmic operation on all elements of the tensor. L , and the redundancy R of the pseudo-ternarized tensor obtained by pseudo-ternarizing all elements of the tensor PT The redundancy R is obtained from the statistical information database 18. L is calculated in the parameter generating unit 120.

[0091] The pseudo ternarization will be described with reference to Fig. 11. Fig. 11 is a graph for explaining the pseudo ternarization of a numerical value with floating-point precision. The horizontal axis of the graph shown in Fig. 11 indicates the numerical value with floating-point precision to be quantized (the "original Float value" shown in Fig. 11), and the vertical axis indicates the value after pseudo ternarization.

[0092] As shown in Figure 11, when pseudo-ternarization is applied to floating-point precision numbers, numbers with floating-point precision that are less than a predetermined first value a and greater than a predetermined second value b are maintained as they are, and numbers that are greater than the first value a and less than or equal to the second value b are converted to 0.

[0093] Next, the redundancy R of the tensor for which the quantization type is to be determined and the redundancy R of the tensor obtained by performing a logarithmic operation on all elements of the tensor are L , and the redundancy R of the pseudo-ternarized tensor obtained by pseudo-ternarizing all elements of the tensor PT Compare with where R L If > R, applying logarithmic operations to all elements of the tensor increases redundancy, meaning that the degradation of inference accuracy can be suppressed even if quantization with lower resolution is performed. Therefore, R L If R, the numeric conversion type is determined to be logarithmic. L If R, it is determined that there is no effect of performing a logarithmic operation on all elements of the tensor.

[0094] Also, R PT If R, applying pseudo-ternary arithmetic to all elements of the tensor results in higher redundancy, meaning that the degradation of inference accuracy can be suppressed even if quantization with lower resolution is performed. Therefore, R PT If R, the numeric conversion type is determined to be pseudo ternary conversion. PT If R, it is determined that there is no effect of applying pseudo ternary operations to all elements of the tensor. Note that the distribution of principal components near 0, where logarithmic transformation and pseudo ternary transformation are assumed to be advantageous, have opposing characteristics. Therefore, R L >R and R PTIf R and R are compatible, it contradicts the assumption, so it is determined that there is no effect of applying either conversion. Based on the results of the effectiveness judgment of the logarithmic conversion and this pseudo ternary operation described above, if there is no effect, the numeric conversion type is determined to be no conversion.

[0095] Next, the quantization resolution in the quantization type is determined (S131b). In this embodiment, the quantization resolution is selected from a plurality of resolution types including N-bit fixed point and ternary. When fixed point precision is adopted as the quantization resolution, the number of bits of the fixed point precision is determined to the maximum number of bits that can be implemented depending on the configuration of the hardware that implements the quantization network. A method for determining whether to select fixed point or ternary as the quantization resolution will be described below.

[0096] When selecting three values ​​as the quantization precision, since a value can be expressed in two bits, two-bit fixed-point precision and three-bit fixed-point precision can be compared as being close to three values ​​in precision. Therefore, we calculate the redundancy when these are selected as the quantization precision. The redundancy R of a 2-bit tensor with the precision of all elements of the tensor set to 2-bit fixed-point precision is N2 and the redundancy R of a 3-bit tensor with 3-bit fixed-point precision for all elements of the tensor. N3 Calculate and if the numeric conversion type is pseudo ternary and R N2 <R N3 If this holds, it is determined that three values ​​are not suitable as the resolution of tensor quantization, and a fixed-point precision of 3 bits or more is selected as the quantization precision according to the hardware configuration.

[0097] On the other hand, R N2 ≧R N3 If true and the numeric conversion type is pseudo-ternary, select ternary as the tensor quantization precision, and R N2 ≧R N3 If the above holds and the numeric conversion type is logarithmic conversion or no conversion, select 2-bit fixed-point precision as the quantization precision of the tensor.

[0098] In this way, the type and fineness of quantization appropriate for each tensor can be determined.

[0099] (Variations, etc.) Although the network quantization method and the like according to the present disclosure have been described based on each embodiment, the present disclosure is not limited to these embodiments. As long as it does not deviate from the gist of the present disclosure, various modifications conceived by a person skilled in the art to each embodiment and other forms constructed by combining some of the components in each embodiment are also included within the scope of the present disclosure.

[0100] For example, although the functions are shared among the functional units in the network quantization device of each of the above embodiments, the sharing of the functions is not limited to the above embodiments. For example, the functional units according to each of the above embodiments may be integrated. In addition, in the second embodiment, the redundancy of the tensor after quantization is calculated in the parameter generating unit 120, but the redundancy of the tensor after quantization may also be calculated in the database constructing unit 16 in the same manner as the redundancy of the tensor before quantization. In this case, the redundancy of the tensor after quantization may be included in the statistical information database 18. Furthermore, the redundancy of the tensor before and after quantization may be calculated in a component other than the database constructing unit 16 of the network quantization device. Furthermore, the redundancy of the tensor before and after quantization may be calculated in a step other than the database construction step.

[0101] Furthermore, in the above-mentioned second embodiment, the resolution of quantization is selected from a plurality of resolution types including three values, but the plurality of resolution types do not have to include three values.

[0102] The following forms may also be included within the scope of one or more aspects of the present disclosure.

[0103] (1) Some of the components constituting the above-mentioned network quantization device may be a computer system composed of a microprocessor, a ROM, a RAM, a hard disk unit, a display unit, a keyboard, a mouse, etc. A computer program is stored in the RAM or the hard disk unit. The microprocessor operates according to the computer program to achieve its functions. Here, the computer program is composed of a combination of multiple instruction codes that indicate commands for a computer to achieve a predetermined function.

[0104] (2) Some of the components constituting the above-mentioned network quantization device may be composed of one system LSI (Large Scale Integration). The system LSI is an ultra-multifunctional LSI manufactured by integrating multiple components on one chip, and specifically, is a computer system including a microprocessor, ROM, RAM, etc. A computer program is stored in the RAM. The system LSI achieves its functions by the microprocessor operating in accordance with the computer program.

[0105] (3) Some of the components constituting the above network quantization device may be composed of an IC card or a standalone module that is detachable from each device. The IC card or the module is a computer system composed of a microprocessor, ROM, RAM, etc. The IC card or the module may include the above ultra-multifunction LSI. The microprocessor operates according to a computer program, causing the IC card or the module to achieve its functions. The IC card or the module may be tamper-resistant.

[0106] (4) Furthermore, some of the components constituting the above-mentioned network quantization device may be the computer program or the digital signal recorded on a computer-readable recording medium, such as a flexible disk, a hard disk, a CD-ROM, an MO, a DVD, a DVD-ROM, a DVD-RAM, a BD (Blu-ray (registered trademark) Disc), a semiconductor memory, etc. Also, some of the components constituting the above-mentioned network quantization device may be the digital signal recorded on such a recording medium.

[0107] In addition, some of the components constituting the above-mentioned network quantization device may transmit the computer program or the digital signal via a telecommunications line, a wireless or wired communication line, a network such as the Internet, data broadcasting, etc.

[0108] (5) The present disclosure may be realized as the above-described methods. It may also be realized as a computer program for implementing these methods by a computer, or as a digital signal comprising the computer program. Furthermore, the present disclosure may be realized as a non-transitory computer-readable recording medium, such as a CD-ROM, on which the computer program is recorded.

[0109] (6) The present disclosure may also provide a computer system having a microprocessor and a memory, the memory storing the computer program, and the microprocessor operating in accordance with the computer program.

[0110] (7) The program or the digital signal may also be implemented by another independent computer system by recording it on the recording medium and transferring it, or by transferring the program or the digital signal via the network, etc.

[0111] (8) The above-described embodiments and modifications may be combined with each other. [Industrial Applicability]

[0112] The present disclosure can be used as an image processing method and the like, as a method for implementing a neural network in a computer or the like. [Explanation of symbols]

[0113] 10, 110 Network quantizer 12. Test Dataset 14 Neural Networks 16 Database Construction Department 18 Statistical Information Database 20, 120 Parameter generation section 22 Quantization Parameter Set 24 Network Construction Department 26, 30 Quantization Network 28 Machine Learning Department 1000 Computers 1001 Input Device 1002 Output device 1003 CPU 1004 Internal Storage 1005 RAM 1007 Reading device 1008 Transmitting and receiving device 1009 Bus

Claims

1. A network quantization method executed by a computer for quantizing a neural network, comprising: a preparation step of preparing the neural network; A database construction step of constructing a statistical information database of tensors handled by the neural network obtained when a plurality of test data sets are input to the neural network; a parameter generating step of generating a quantization parameter set by quantizing the tensor values ​​based on the statistical information database and the neural network; and constructing a quantized network by quantizing the neural network using the quantization parameter set; The parameter generating step includes a quantization type determining step of determining a quantization type for each of a plurality of layers constituting the neural network. Network quantization methods.

2. In the quantization type determination step, the quantization type is selected from a plurality of numerical transformation types that perform different numerical transformations on the tensor, and the plurality of numerical transformation types include a logarithmic transformation and no transformation. The network quantization method of claim 1.

3. In the quantization type determination step, the quantization type is selected from a plurality of resolution types having different quantization resolutions, and the plurality of resolution types include an N-bit fixed point (N: an integer of 2 or more) and a ternary value. A network quantization method according to claim 1 or 2.

4. The quantization type is determined based on the redundancy of the tensors included in each of the plurality of layers. A network quantization method according to any one of claims 1 to 3.

5. The redundancy is determined based on the result of a tensor decomposition of the tensor. The network quantization method of claim 4.

6. The quantization type is determined to be a lower definition type as the redundancy increases. A network quantization method according to claim 4 or 5.

7. A network quantization device for quantizing a neural network, comprising: A database construction unit that constructs a statistical information database of tensors handled by the neural network obtained when a plurality of test data sets are input to the neural network; a parameter generating unit that generates a quantization parameter set by quantizing the tensor values ​​based on the statistical information database and the neural network; a network construction unit that constructs a quantization network by quantizing the neural network using the quantization parameter set; The parameter generating unit determines a quantization type for each of a plurality of layers that configure the neural network. Network quantizer.

Images