Storage medium, quantization method, and quantization apparatus

Abstract

Disclosed are a non-transitory computer-readable storage medium storing a quantization program, a quantization method, and a quantization device, the quantization program causing at least one computer to execute processing including: for all layers of a neural network, calculating a difference between a radius of a confidence domain threshold value and a quantization error of a first bit width, the first bit width being one level narrower than a second bit width; calculating each scaling coefficient of all layers based on the difference; updating the radius of the confidence domain by using a minimum value among the scaling coefficients; and quantizing parameters of the neural network by a third bit width set based on the confidence domain.

Description

Technical Field

[0001] The implementation methods discussed in this article involve storage media, quantization methods, and quantization devices. Background Technology

[0002] Quantization is known as a method for reducing the execution time of increasingly complex neural networks (NNs).

[0003] In quantization, computation and communication times are reduced by converting the data type of the weights used in the neural network (e.g., FP32) to a data type that includes smaller data sizes (INT8).

[0004] An optimization algorithm is known for automatically setting data types for each layer without degrading the recognition rate, and is used as a quantization technique for related technologies.

[0005] In such optimization algorithms of related technologies, the quantization error of the target vector is compared with a threshold determined by an algorithm based on the trust region method, and the bit width is temporarily calculated based on the comparison result. In the trust region method, to avoid incorrect bit width determination due to approximation errors, the threshold is scaled to be less than or equal to the trust region radius.

[0006] Training is performed iteratively multiple times by setting a temporary computation bit width. Afterward, the evaluation function is computed.

[0007] The bit width and trust region radius are updated based on whether the constraint that the loss function calculated using the temporarily calculated bit width is less than or equal to the loss function calculated under ideal conditions (ideal loss function) is met.

[0008] For example, if the calculated loss function is less than or equal to the ideal loss function, the bit width is updated using a temporarily calculated bit width value, and the trust region radius is increased by multiplying the current trust region radius by a constant "a". In this case, the constant "a" is a value greater than 1 ("a" > 1). On the other hand, if the calculated loss function is greater than the ideal loss function, the bit width value is maintained, the temporarily calculated bit width value is discarded, and the trust region radius is decreased by multiplying the current trust region radius by a constant "a". In this case, the constant "a" is a value less than 1 ("a" < 1).

[0009] In the quantization algorithm, these processing steps are repeated for all quantization target layers with a pre-specified number of iterations.

[0010] As related technologies, International Publication No. 2020 / 049681 and Japanese Patent Publication No. 2020-9048 have been disclosed. Summary of the Invention

[0011] [Technical Issues]

[0012] In such quantization techniques, the bit width can be converged deterministically by increasing the number of iterations. However, increasing the number of iterations increases the execution time. On the other hand, when the time period used for optimization is limited, the bit width may not be sufficiently quantized within that time period. Due to insufficient quantization, the training time of the neural network also increases.

[0013] One aspect is that the goal is to enable optimization of all layers in a short period of time.

[0014] [Problem Solution]

[0015] According to one aspect of the embodiments, a non-transitory computer-readable storage medium stores a quantization program that causes at least one computer to perform a process comprising: calculating, for all layers of a neural network, a difference between a quantization error of a trust region radius threshold and a first bit width, the first bit width being one level narrower than a second bit width; calculating each scaling factor for all layers based on the difference; updating the trust region radius by using the minimum value among the scaling factors; and quantizing the parameters of the neural network by using a third bit width based on a trust region setting.

[0016] [Beneficial effects of the invention]

[0017] According to the implementation method, all layers can be optimized in a short time. Attached Figure Description

[0018] Figure 1 This is a diagram illustrating an example of the hardware configuration of an information processing device used as an example of an implementation method;

[0019] Figure 2 This is a diagram illustrating an example of the functional configuration of an information processing apparatus used as an example of an implementation method;

[0020] Figure 3 This is a diagram showing an overview of the neural network;

[0021] Figure 4 This is a diagram illustrating an example of a quantized target vector in an information processing apparatus used as an example of an implementation method;

[0022] Figure 5 It is a graph used to describe the difference calculated by the difference calculation unit of the information processing apparatus used as an example of an implementation method;

[0023] Figure 6 It is a graph used to describe the difference calculated by the difference calculation unit of the information processing apparatus used as an example of an implementation method;

[0024] Figure 7This is a graph illustrating the relationship between quantization error and a threshold in an information processing apparatus used as an example of an implementation method;

[0025] Figure 8 This is a graph illustrating the relationship between quantization error and a threshold in an information processing apparatus used as an example of an implementation method;

[0026] Figure 9 It is a flowchart for describing the quantization process performed in an information processing apparatus used as an example of an implementation method;

[0027] Figure 10 It is a flowchart for describing the quantization process performed in an information processing apparatus used as an example of an implementation method;

[0028] Figure 11 This is a diagram illustrating an example of the result of quantization performed by an information processing device used as an example of an implementation method; and

[0029] Figure 12 This is a diagram illustrating an example of the result of quantization performed by an information processing device used as an example of an implementation method. Detailed Implementation

[0030] Embodiments of the quantization procedure, quantization method, and quantization apparatus will now be described with reference to the accompanying drawings. The embodiments presented below are merely examples and are not intended to exclude the application of various modifications and techniques not explicitly described in the embodiments. For example, various modifications can be made to this embodiment without departing from its spirit. The accompanying drawings do not represent only the components shown therein. Therefore, other functions, etc., may be included.

[0031] (A) Configuration

[0032] Figure 1 This is a diagram illustrating an example of the hardware configuration of an information processing device 1 used as an example of an implementation method.

[0033] Information processing device 1 is a computer that implements a quantized neural network. For example... Figure 1 As shown, the information processing device 1 includes a central processing unit (CPU) 10, a memory 11, and an accelerator 12. The CPU 10, memory 11, and accelerator 12 are coupled to each other via a communication bus 13 so that they can communicate with each other. The communication bus 13 enables data communication in the information processing device 1.

[0034] Memory 11 is a storage memory that includes read-only memory (ROM) and random-access memory (RAM). The ROM of memory 11 contains software programs related to quantization processing and data used in those programs. The software programs on memory 11 are appropriately read and executed by CPU 10. The RAM of memory 11 serves as either main storage or working memory. The RAM of memory 11 also stores parameters such as weights used in quantization and various thresholds.

[0035] Accelerator 12 performs computational processing, such as matrix computation, to be performed for computation in a neural network.

[0036] CPU 10 is a processing device (processor) that performs various control and calculation tasks. CPU 10 controls the entire information processing device 1 based on an installed program. CPU 10 executes a deep learning processing program (not shown) stored in memory 11, etc., and thus implements the deep learning processing unit 100. Figure 2 The function of ) will be described later.

[0037] The deep learning processing program may include a quantization program. The CPU 10 executes the quantization program (not shown) stored in memory 11, etc., and thus implements the quantization processing unit 101. Figure 2 The function of ) will be described later.

[0038] The CPU 10 of the information processing device 1 executes a deep learning processing program (quantization program) and thus serves as a deep learning processing unit 100 (quantization processing unit 101).

[0039] The program (quantization program) for implementing the functions of the deep learning processing unit 100 (quantization processing unit 101) is provided in the form of a computer-readable recording medium, such as a floppy disk, a compact disc (CD) such as a CD-ROM, a recordable CD (CD-R) or a rewritable CD (CD-RW), a digital versatile disc (DVD) such as a DVD-ROM, DVD-RAM, DVD-R, DVD+R, DVD-RW, DVD+RW, or high-definition (HD) DVD, a Blu-ray disc, a magnetic disk, an optical disk, or a magneto-optical disk. The computer (information processing device 1) reads the program from the recording medium, transfers the program to an internal or external storage device, stores the program in the internal or external storage device, and uses the program. For example, the program can be recorded on a storage device (recording medium) such as a magnetic disk, optical disk, or magneto-optical disk, and can be provided to the computer from the storage device via a communication channel.

[0040] When the deep learning processing unit 100 (quantization processing unit 101) performs its functions, the computer's microprocessor (CPU 10 in this embodiment) executes the program stored in the internal storage device (RAM or ROM of the memory 11 in this embodiment). At this time, the computer can read and execute the program recorded on the recording medium.

[0041] Figure 2 This is a diagram illustrating an example of the functional configuration of an information processing device 1 used as an example of an implementation method.

[0042] like Figure 2 As shown, the information processing device 1 has the function of a deep learning processing unit 100. The deep learning processing unit 100 performs deep learning in a neural network.

[0043] Neural networks can be hardware circuits or software-based virtual networks, in which layers, virtually constructed in a computer program by a CPU, are coupled to each other.

[0044] Figure 3 An overview of neural networks is shown. Figure 3 The neural network shown is a deep neural network that includes multiple hidden layers between the input and output layers. Hidden layers can be, for example, convolutional layers, pooling layers, fully connected layers, etc. The circles shown in each layer represent nodes that perform specific computations.

[0045] For example, a neural network receives input data, such as images or sounds, into its input layer and sequentially performs specific computations in hidden layers consisting of convolutional layers, pooling layers, etc. In this way, the neural network performs forward processing (forward propagation), where information obtained through computation propagates sequentially from the input side to the output side. After performing forward processing, backward processing (backward propagation) is performed to determine the parameters to be used in the forward processing, in order to reduce the value of the error function obtained from the output data from the output layer and correct the answer data. Based on the results of the backward propagation, update processing is performed to update variables such as weights. For example, gradient descent is used as an algorithm to determine the update width of the weights to be used in the computation in the backward propagation.

[0046] The deep learning processing unit 100 includes a quantization processing unit 101.

[0047] The quantization processing unit 101 quantizes the variables to be used in the neural network. In the information processing device 1, the quantization processing unit 101 achieves quantization by reducing the bit width of the weight vector in each layer. The information processing device 1 uses the trust region method.

[0048] like Figure 2 As shown, the quantization processing unit 101 has the functions of a quantization error calculation unit 106, a threshold generation unit 107, a bit width setting unit 102, a difference calculation unit 103, a scaling factor calculation unit 104, and a trust region radius update unit 105.

[0049] The quantization error calculation unit 106 calculates the quantization error of the vector to be used in the computation of the neural network. Since the quantization error can be calculated using various known methods, a description of them will be omitted.

[0050] Figure 4 This is a diagram illustrating an example of a quantized target vector in an information processing apparatus 1 used as an example of an implementation method.

[0051] Will give Figure 4 The example shown is an example of how the weight vector W1 of layer 1 and the weight vector W2 of layer 2 are quantized by the information processing device 1.

[0052] The threshold generation unit 107 generates a trust region radius threshold based on the set trust region radius and bit width. In the following text, the trust region radius threshold may be simply referred to as the threshold in some cases. For example, the threshold generation unit 107 calculates the loss and gradient using the set bit width and calculates the trust region radius threshold according to the following expression (1) used for derivation.

[0053]

[0054] In expression (1) above, ΔW represents the quantization error. L(w+Δw) represents the loss function of the quantized base model (e.g., INT16). L(w) represents the loss function of the base model (e.g., INT32). ideal This represents, for example, the loss function calculated using FP32 without quantization (the ideal loss function). margin This represents the loss function L(w+Δw) of the quantized model relative to the ideal loss function L. ideal The amount of the loss function margin, and the amount of this margin is set by the designer.

[0055] The loss function L(w+Δw) under quantization is Taylor expanded. Constraints are set such that even with quantization, the loss function is less than the loss function obtained by adjusting the margin L. margin Compared with the ideal value L ideal The quantity obtained by adding (using the loss function of FP32).

[0056] By rearranging the above expression (1) for the quantization error, we can derive the upper limit of the quantization error that satisfies the constraints, which is equal to the threshold.

[0057] Threshold generation unit 107 sets (scales) the threshold such that the L2 norm of the threshold (||Q) is... th ||2) Less than or equal to the trust region radius. In the following text, in some cases, the threshold can be determined by the reference symbol Q. th express.

[0058] Bit width setting unit 102 sets the bit width of the weight vector in each layer.

[0059] Bit width setting unit 102 calculates the loss function under ideal conditions (ideal loss function) L ideal .

[0060] The bit width setting unit 102 temporarily calculates the bit width. The bit width setting unit 102 temporarily calculates the smallest bit width among multiple bit width candidates whose quantization error is less than or equal to the calculated threshold. Assume that the information processing device 1 uses three types (three levels) of bit width [8, 16, 32] as bit width candidates.

[0061] The bit width setting unit 102 checks whether the temporarily calculated bit width meets the constraints. The bit width setting unit 102 determines the loss function (L) based on the temporarily calculated bit width. tmp Is it less than or equal to the ideal loss function (L)? ideal () is used as a constraint.

[0062] If the loss function (L) tmp ) less than or equal to the ideal loss function (L idealFor example, if the constraint conditions are met, the bit width setting unit 102 sets the temporarily calculated bit width as the bit width. If the constraint conditions are met, the bit width setting unit 102 increases the trust region radius by multiplying the current trust region radius by a constant (a value greater than 1).

[0063] On the other hand, if the loss function (L tmp ) greater than the ideal loss function (L ideal For example, if the constraints are not met, the bit width setting unit 102 maintains the bit width value, discards the temporarily calculated bit width value, and reduces the trust region radius by multiplying the current trust region radius by a constant (a value less than 1).

[0064] The difference calculation unit 103 measures the radius of the trust region (Q) for all quantized target vectors (all layers). th The difference between (Q) and the quantization error of the following bit width diff The bit width ratio is temporarily calculated by the bit width setting unit 102 to be one level narrower.

[0065] Figure 5 and Figure 6 Each of these is used to describe the difference (Q) calculated by the difference calculation unit 103 of the information processing apparatus 1, which is used as an example of this embodiment. diff (The image is shown.) Figure 5 This is a graph showing the quantization error of the weight vector W1 in layer 1. Figure 6 This is a graph showing the quantization error of the weight vector W2 in layer 2.

[0066] exist Figure 5 In the example shown, it is assumed that the temporary computation bit width is 16 bits. The threshold 1 of layer 1 is determined by the reference symbol Q. th,1 This indicates that the difference between layer 1 and layer 1 is represented by the reference symbol Q. diff,1 This can be represented as the threshold and difference of each layer, such that the threshold of layer k is determined by the reference symbol Q. th,k This indicates that the difference in layer k is represented by the reference symbol Q. diff,k express.

[0067] Difference calculation unit 103 for threshold 1 (Q) th,1 The difference Q between the quantization error of the provisionally calculated 16-bit width and the 8-bit width (see reference symbol P1) is one level narrower. diff,1 (See reference symbol P2) Perform measurements (calculations).

[0068] exist Figure 6 In the example shown, it is assumed that the temporary computation bit width is 32 bits. The threshold 2 of layer 2 is determined by the reference symbol Q. th,2 This indicates that the difference between layer 2 and layer 2 is represented by the reference symbol Q. diff,2 express.

[0069] Difference calculation unit 103 for threshold 2 (Q) th,2 The difference Q between the quantization error (see reference symbol P3) and the quantization error of a 16-bit width (one level narrower than the provisionally calculated 32-bit width) is... diff,2 (See reference symbol P4) Perform measurements (calculations).

[0070] Scaling factor calculation unit 104 calculates the scaling factor (Q) for all quantized target vectors (all layers). scale ).

[0071] Scaling factor (Q) scale ) is to calculate the difference (Q) by the difference calculation unit 103 in the multiplication performed by the trust region radius update unit 105, which will be described later. diff The value obtained by using it as a coefficient.

[0072] The scaling factor calculation unit 104 uses various thresholds (Q) th ) and the respective differences (Q) between the threshold and the quantization error of a bit width one level narrower than the temporarily calculated bit width. diff ), calculate each scaling factor Q for all quantized target vectors. scale .

[0073] The scaling factor calculation unit 104 calculates the scaling factor (Q) using the following expression (2). scale ).

[0074] Q scale，k =1+Q diff，k / Q th，k (2)

[0075] In the above expression (2), the scaling factor is expressed in a generalized manner, and k represents the layer number. Expression (2) represents the scaling factor that matches the upper limit (e.g., threshold) of the next trust region radius with the quantization error of a bit width one level narrower than the bit width temporarily calculated at this time.

[0076] Based on the above expression (2), the scaling factor of layer 1 is represented by the following expression (3).

[0077] Q scale，1 =1+Q diff，1 / Q th，1 (3)

[0078] Based on the above expression (2), the scaling factor of layer 2 is represented by the following expression (4).

[0079] Q scale，2 =1+Q diff，2 / Q th，2(4)

[0080] There is no difference (Q) between the threshold and the quantization error of a bit width one level narrower than the temporarily calculated bit width. diff In the case of ), the scaling factor was not obtained.

[0081] Scaling factor calculation unit 104 selects the minimum value among all scaling factors obtained for each layer as the minimum scaling factor Q. scale，min ).

[0082] If the next threshold becomes too large due to the scaling factor, it may be necessary to calculate bit width combinations that are difficult to satisfy constraints such as "bit width of all layers = 8 bits". Therefore, in the information processing device 1, the minimum scaling factor is selected to gradually change the bit width.

[0083] Minimum scaling factor Q scale，min ) is represented by the following expression (5).

[0084] Q scale，min =min(Q) scale，1 Q scale，2 (5)

[0085] Trust region radius update unit 105 updates the trust region radius.

[0086] Trust region radius update unit 105 updates the trust region radius using the following expression (6): Next trust region radius = Current trust region radius × max(constant "a", Q) scale，min (6)

[0087] The constant "a" is a specific constant (trust region radius update constant) used in the related technology method to increase or decrease the trust region radius. The trust region radius update unit 105 uses the trust region radius update constant "a" (hereinafter referred to as constant "a" in some cases) and the scaling factor (Q) scale,min The larger value in the ) is used to update the trust region radius.

[0088] Scaling factor Q scale,min Including the above difference (Q) diff Therefore, if the scaling factor Q is chosen... scale , min Then, the next calculation will use a different bit width. Therefore, in the case where "constant "a" > Q scale,min In the case of “”, even if a constant a is chosen, the bit width will be calculated differently next time.

[0089] Figure 7 and Figure 8 Each of these is a graph illustrating the relationship between quantization error and a threshold in an information processing device 1 used as an example of an implementation. Figure 7This is a graph showing the quantization error of the weight vector W1 in layer 1. Figure 8 This is a graph showing the quantization error of the weight vector W2 in layer 2.

[0090] exist Figure 7 and Figure 8 In each of these, the upper side shows the state before the trust region radius is updated, and the lower side shows the state after the trust region radius is updated. Figure 7 and Figure 8 The example shown presents Q scale,2 Chosen as Q scale,min For example, this is shown by using Q. scale,2 Here's an example of updating the trust region radius.

[0091] Figure 7 The following example illustrates a 16-bit bit width temporarily calculated in layer 1, but even when using Q... scale,2 After updating the trust region radius, the next bit width to be selected will also be 16 bits and will remain unchanged. If this is done, the quantization error of a bit width one level narrower will exceed the threshold Q. th,k If not, the bit width will not be updated.

[0092] On the other hand, Figure 8 In layer 2 shown, the temporary calculation bit width is 32 bits, but by using Q... scale,2 After updating the trust region radius, the upper limit (= threshold) of the next trust region radius matches the quantization error of the vector. Therefore, the next bit width to be selected is 16 bits, which indicates that a different bit width has been selected than the previous bit width.

[0093] (B) Operation

[0094] According to Figure 9 and Figure 10 The flowchart (steps S1 to S17) describes the quantization process performed in the information processing apparatus 1, which is used as an example of an implementation configured as described above. Figure 9 Steps S1 to S6 are shown, and Figure 10 Steps S7 to S17 are shown.

[0095] exist Figure 9 In step S1, the quantization processing unit 101 performs initialization processing. For example, the quantization processing unit 101 sets an initial value for the bit width and calculates the ideal loss function (target value).

[0096] exist Figure 9 In step S2, the threshold generation unit 107 calculates the threshold Q. th The threshold generation unit 107 calculates the threshold Q using the set bit width. thAt this time, the threshold generation unit 107 sets the threshold such that the threshold Q... th The L2 norm is less than or equal to the trust region radius.

[0097] exist Figure 9 In step S3, the threshold generation unit 107 checks the threshold Q. th Is the L2 norm less than or equal to the trust region radius? If the threshold Q th If the L2 norm is less than or equal to the trust region radius (see the "Yes" path in step S3), then the processing continues. Figure 9 Step S5 in the process.

[0098] On the other hand, if the threshold Q th If the L2 norm is greater than the trust region radius (see the "No" path in step S3), then the process proceeds to step S4. Figure 9 In step S4, the threshold generation unit 107 scales the threshold so that the threshold Q th The L2 norm is equal to the trust region radius. Then, the processing proceeds to step S5.

[0099] In step S5, the bit width setting unit 102 temporarily calculates the bit width. The bit width setting unit 102 temporarily calculates the minimum bit width among multiple bit width candidates whose quantization error is less than or equal to the calculated threshold.

[0100] exist Figure 9 In step S6, the bit width setting unit 102 checks whether the temporarily calculated bit width meets the constraint conditions. Figure 10 In step S7, the constraint condition is used to determine the loss function (L) based on the temporarily calculated bit width of the bit width by the bit width setting unit 102. tmp Is it less than or equal to the ideal loss function (L)? ideal ).

[0101] If the loss function (L) tmp ) less than or equal to the ideal loss function (L ideal For example, if the constraint condition is met (see the "Yes" path in step S7), then the bit width setting unit 102 will... Figure 10 In step S8, the temporarily calculated bit width is set as the bit width. Setting the bit width can be referred to as determining the bit width.

[0102] exist Figure 10 In step S9, the difference calculation unit 103 measures the difference at a threshold (Q) for all quantized target vectors (all layers). th The difference (Q) between the quantization error and the quantization error of a bit width that is one level narrower than the bit width temporarily calculated by the bit width setting unit 102. diff ).

[0103] exist Figure 10In step S10, the scaling factor calculation unit 104 calculates the scaling factor (Q) for all quantized target vectors (all layers) using the above expression (2). scale ).

[0104] exist Figure 10 In step S11, the scaling factor calculation unit 104 selects and outputs the smallest scaling factor (Q) among all calculated scaling factors. scale,min ).

[0105] exist Figure 10 In step S12, the trust region radius update unit 105 updates (increases) the trust region radius by using the above expression (6). Then, the process proceeds to step S15.

[0106] On the other hand, as a result of the check in step S7, if the loss function (L tmp ) greater than the ideal loss function (L ideal For example, if the constraints are not met (see the "No" path in step S7), the process proceeds to... Figure 10 Step S13 in the process.

[0107] In step S13, the bit width setting unit 102 maintains the bit width value and discards the temporarily calculated bit width value.

[0108] exist Figure 10 In step S14, the bit width setting unit 102 reduces the trust region radius by multiplying the current trust region radius by a constant “a” (“a” is a value less than 1).

[0109] The quantization processing unit 101 repeatedly executes steps S2 to S14 (bit width determination processing) until... Figure 10 The convergence condition is met in step S15. For example, the quantization processing unit 101 counts the number of times steps S2 to S14 are performed (determined number of times), and... Figure 10 In step S16, it is checked whether the number of determinations has reached the pre-specified number.

[0110] As a result of the check, if it is determined that the specified number of attempts has not been reached (see the "No" path in step S16), the process returns to... Figure 9 Step S2 in the process. On the other hand, if it is determined that the specified number of times has been reached (see the "Yes" path in step S16), then the quantization processing unit 101 in Figure 10 In step S17, the determined bit width is output. The quantization processing unit 101 can store the determined bit width for each layer in a storage area of ​​the memory 11, a storage device (not shown), etc. The quantization processing unit 101 performs quantization using the determined bit width for each layer. Then, the process ends.

[0111] (C) Effect

[0112] As described above, according to the information processing apparatus 1 used as an example of an implementation, the difference calculation unit 103 measures the difference at a threshold (Q) for all quantized target vectors (all layers). th The difference (Q) between the quantization error and the quantization error of a bit width that is one level narrower than the bit width temporarily calculated by the bit width setting unit 102. diff ).

[0113] Scaling factor calculation unit 104 calculates the scaling factor (Q) for all quantized target vectors (all layers). scale And select the minimum scaling factor (Q) among all the scaling factors obtained for each layer. scale,min ).

[0114] Trust region radius update unit 105 uses a constant "a" and a scaling factor (Q) scale,min The larger value in the range is used to update the trust region radius. Therefore, in the bit width determination process, as a combination of the bit widths of each layer after the trust region radius has been updated, a bit width that is different from the previous bit width and whose upper limit (= threshold) of the updated trust region radius matches the quantization error of the vector is selected.

[0115] Therefore, each time, the temporarily calculated bit width is determined to ensure that the constraints are met for different bit width combinations. For example, for the same bit width combination, the determination of whether the constraints are met is not performed repeatedly.

[0116] Therefore, bit width determination is not performed on repetitive bit width combinations, and it can be performed efficiently. For example, all layers can be optimized in a short time.

[0117] The bit width can be fully quantized within a finite optimization cycle. This reduces the number of algorithmic steps required to optimize the bit width, thus reducing computation time and cost. Compared to related techniques, the degree of quantization is increased. Consequently, the training time for neural networks can be reduced.

[0118] The reason for performing numerous iterations for bit-width convergence in quantization techniques is that there are instances where the constraint conditions are re-determined for a given bit-width combination. This repetitive work, as a result of reusing the already determined bit-width combination to determine the constraints, increases the quantization processing time.

[0119] Conversely, in the information processing device 1, the determination of repeated bit width combinations is not performed, and bit width determination can be performed efficiently.

[0120] Figure 11 and Figure 12Each of the figures is a diagram illustrating an example of the result of quantization performed by the information processing device 1, which is used as an example of an implementation method.

[0121] exist Figure 11 In the diagram, the horizontal axis represents the number of periods, and the vertical axis represents the validation accuracy. For example... Figure 11 As shown, even when the bit width is set to 8 bits for all layers by quantization performed by the information processing device 1 (in the case of training all layers by using QINT8), bit width adjustment (quantization) is performed to enable training without reducing the recognition rate (verification accuracy).

[0122] exist Figure 12 In this diagram, the horizontal axis represents a defined number of iterations, and the vertical axis represents a defined bit width. For example... Figure 12 As shown, through quantization performed by the information processing device 1, the bit width is set to (determined to) 8 bits for all layers during the optimization period (number of applications = 10 times).

[0123] (D) Other

[0124] The disclosed technology is not limited to the embodiments described above, and can be implemented with various modifications without departing from the spirit of these embodiments. Various configurations and processes described in these embodiments may be selectively adopted, omitted, or appropriately combined.

[0125] For example, in the above embodiments, an example of quantization by reducing the bit width of the weight vector in each layer has been described. However, this configuration is not limited to this. For example, this disclosure can be applied to the quantization of activation, weighted gradients, activation gradients, etc., and can be performed using various modifications.

[0126] The above disclosure enables those skilled in the art to implement or manufacture this embodiment.

[0127] (E) Notes

[0128] Further details regarding the above implementation methods will be disclosed in the appendix.

[0129] 1. A non-transitory computer-readable storage medium storing a quantization program, the quantization program causing at least one computer to perform a process, the process comprising:

[0130] For all layers of the neural network, calculate the difference between the quantization error of the first bit width and the trust region radius threshold, where the first bit width is one level narrower than the second bit width;

[0131] Calculate each scaling factor for all layers based on the difference;

[0132] The trust region radius is updated by using the minimum value among the scaling factors; and

[0133] The parameters of the neural network are quantized using a third bit width based on the trust region setting.

[0134] 2. The non-transitory computer-readable storage medium according to Appendix 1, wherein the process further includes:

[0135] The trust region radius is updated by multiplying the trust region radius by the minimum of the scaling factors.

[0136] 3. The non-transitory computer-readable storage medium according to Appendix 2, wherein the process further includes:

[0137] The trust region radius is updated by multiplying it by the larger of the following values: the minimum of the scaling factors and the constant used to update the trust region radius.

[0138] 4. A quantization method for causing a computer to perform a process, the process comprising:

[0139] For all layers of the neural network, calculate the difference between the quantization error of the first bit width and the trust region radius threshold, where the first bit width is one level narrower than the second bit width;

[0140] Calculate each scaling factor for all layers based on the difference;

[0141] The trust region radius is updated by using the minimum value among the scaling factors; and

[0142] The parameters of the neural network are quantized using a third bit width based on the trust region setting.

[0143] 5. The quantization method according to Appendix 4, wherein the processing further includes:

[0144] The trust region radius is updated by multiplying the trust region radius by the minimum of the scaling factors.

[0145] 6. The quantization method according to Appendix 5, wherein the processing further includes:

[0146] The trust region radius is updated by multiplying it by the larger of the following values: the minimum of the scaling factors and the constant used to update the trust region radius.

[0147] 7. A quantization device, comprising:

[0148] One or more memories; and

[0149] One or more processors coupled to the one or more memories, and the one or more processors are configured to:

[0150] For all layers of the neural network, calculate the difference between the quantization error of the first bit width and the trust region radius threshold, where the first bit width is one level narrower than the second bit width;

[0151] Calculate each scaling factor for all layers based on the difference;

[0152] The trust region radius is updated by using the minimum value among the scaling factors; and

[0153] The parameters of the neural network are quantized using a third bit width based on the trust region setting.

[0154] 8. The quantization apparatus according to Appendix 7, wherein the one or more processors are further configured to:

[0155] The trust region radius is updated by multiplying the trust region radius by the minimum of the scaling factors.

[0156] 9. The quantization apparatus according to Appendix 8, wherein the one or more processors are further configured to:

[0157] The trust region radius is updated by multiplying it by the larger of the following values: the minimum of the scaling factors and the constant used to update the trust region radius.

Claims

1. A non-transitory computer-readable storage medium storing a quantization program, the quantization program causing at least one computer to perform a process, the process comprising: For all layers of the neural network, calculate the difference between the quantization error of the first bit width and the trust region radius threshold, where the first bit width is one level narrower than the second bit width. The second bit width is the smallest bit width among multiple candidate bit widths that makes the quantization error less than or equal to the trust region radius threshold when the L2 norm of the trust region radius threshold is less than or equal to the trust region radius. The scaling factor for each of all layers is calculated based on the difference, where the scaling factor for layer k is determined by... Calculate, where k is the layer number. This represents the difference for layer k. This represents the trust region radius threshold of layer k; The trust region radius is updated by multiplying it by the larger of the following values: the minimum of the scaling factors and the constant used to update the trust region radius; and The parameters of the neural network are quantized using a third bit width based on the trust region setting. The third bit width is the bit width determined when the convergence condition is met.

2. A quantization method for causing a computer to perform processing, the processing comprising: For all layers of the neural network, calculate the difference between the quantization error of the first bit width and the trust region radius threshold, where the first bit width is one level narrower than the second bit width. The second bit width is the smallest bit width among multiple candidate bit widths that makes the quantization error less than or equal to the trust region radius threshold when the L2 norm of the trust region radius threshold is less than or equal to the trust region radius. The scaling factor for each of all layers is calculated based on the difference, where the scaling factor for layer k is determined by... Calculate, where k is the layer number. This represents the difference for layer k. This represents the trust region radius threshold of layer k; The trust region radius is updated by multiplying it by the larger of the following values: the minimum of the scaling factors and the constant used to update the trust region radius; and The parameters of the neural network are quantized using a third bit width based on the trust region setting. The third bit width is the bit width determined when the convergence condition is met.

3. A quantization device, comprising: One or more memory units; as well as One or more processors coupled to the one or more memories, and the one or more processors are configured to: For all layers of the neural network, calculate the difference between the quantization error of the first bit width and the trust region radius threshold, where the first bit width is one level narrower than the second bit width. The second bit width is the smallest bit width among multiple candidate bit widths that makes the quantization error less than or equal to the trust region radius threshold when the L2 norm of the trust region radius threshold is less than or equal to the trust region radius. The scaling factor for each of all layers is calculated based on the difference, where the scaling factor for layer k is determined by... Calculate, where k is the layer number. This represents the difference for layer k. This represents the trust region radius threshold of layer k; The trust region radius is updated by multiplying it by the larger of the following values: the minimum of the scaling factors and the constant used to update the trust region radius; and The parameters of the neural network are quantized using a third bit width based on the trust region setting. The third bit width is the bit width determined when the convergence condition is met.

Images