Model quantization method, electronic device, medium and program product

Abstract

The present application relates to the field of machine learning technology, and in particular to a model quantization method, electronic device, medium, and program product. The model includes a first operation, a second operation, and a third operation, wherein the third operation is an associated operation of the first operation and the second operation; and the method includes: obtaining a first quantization coefficient of the first operation, first quantization data of the first operation, a second quantization coefficient of the second operation, and second quantization data of the second operation; based on the size relationship between the first quantization coefficient and the second quantization coefficient, determining the first quantization coefficient or the second quantization coefficient as a common quantization coefficient; quantizing the first quantization data or the second quantization data based on the common quantization coefficient, and obtaining the operation result of the third operation based on the quantization result. In this way, the accuracy of the model can be improved and the speed at which the electronic device runs the model can be improved.

Description

Technical Field

[0001] The present application relates to the field of machine learning technology, and in particular to a model quantization method, electronic device, medium, and program product. Background Art

[0002] With the rapid development of artificial intelligence (AI) technology, neural networks (such as deep neural networks) have been widely used in computer vision, speech, natural language processing, reinforcement learning, and other fields in recent years. However, as neural network algorithms develop, their complexity and model structures become increasingly complex. Consequently, the computing resources and memory requirements of the devices deploying neural network models also increase.

[0003] Therefore, when a neural network model is deployed in an electronic device with limited computing and storage resources (e.g., mobile phones and other embedded electronic devices), it is generally necessary to quantize the neural network model to reduce the model memory size and speed up data processing. However, since quantizing the neural network model will quantize the weight matrix data and input data in the neural network model from a high-precision order of magnitude to a low-precision order of magnitude, there will be a loss of computational precision. Moreover, each operator in a neural network generally has multiple sub-operators, and these sub-operators are generally quantized independently. Since the input data and weight matrix data that determine the quantization coefficients of each sub-operator are different, the quantization coefficients of some sub-operators may be inconsistent. When the quantization coefficients of multiple sub-operators are inconsistent, there will be a greater loss of precision. Summary of the Invention

[0004] The purpose of this application is to provide a model quantization method, electronic device, and medium.

[0005] The first aspect of the present application provides a model quantization method, which is applied to an electronic device, wherein the model is a GRU model or an LSTM model, and the model includes a first operation, a second operation, and a third operation, wherein the third operation is at least an associated operation of the first operation and the second operation; and the method includes: obtaining a first quantization coefficient of the first operation, first quantization data of the first operation, a second quantization coefficient of the second operation, and second quantization data of the second operation; determining a common quantization coefficient based on the size relationship between the first quantization coefficient and the second quantization coefficient; quantizing the first quantization data or the second quantization data based on the common quantization coefficient, and obtaining an operation result of the third operation based on the quantization result.

[0006] By using the method provided in the embodiments of the present application, the computational accuracy of the third operation can be improved. In some embodiments, the first operation can be the operation corresponding to the first operator described below, the second operation can be the operation corresponding to the second operator described below, and the third operation can be the operation corresponding to the third operator described below.

[0007] In a possible implementation of the first aspect, the determining of the common quantization coefficient based on the magnitude relationship between the first quantization coefficient and the second quantization coefficient includes taking the larger one of the first quantization coefficient and the second quantization coefficient as the common quantization coefficient.

[0008] In an embodiment of the present application, the common quantization coefficient is one of the first quantization coefficient and the second quantization coefficient, so that the electronic device only needs to quantize the first quantized data or the second quantized data, which can reduce the number of times the electronic device dequantizes the quantized data into floating-point data and then requantizes the floating-point data into quantized data during the calculation model process, thereby improving the speed at which the electronic device runs the model.

[0009] In a possible implementation of the first aspect above, the quantization of the first quantized data or the second quantized data based on the common quantization coefficient includes: when the common quantization coefficient is the second quantization coefficient, quantizing the first quantized data of the first operation into third quantized data based on the second quantization coefficient; when the common quantization coefficient is the first quantization coefficient, quantizing the second quantized data of the second operation into fourth quantized data based on the first quantization coefficient.

[0010] In a possible implementation of the first aspect above, the quantizing of the first quantized data of the first operation into third quantized data based on the second quantization coefficient includes: dequantizing the first quantized data into corresponding first floating-point data according to the first quantization coefficient, and quantizing the first floating-point data into third quantized data according to the second quantization coefficient.

[0011] In a possible implementation of the first aspect above, the above-mentioned quantization of the second quantized data of the second operation into fourth quantized data based on the first quantization coefficient includes: dequantizing the second quantized data into corresponding second floating-point data according to the second quantization coefficient, and quantizing the second floating-point data into fourth quantized data according to the first quantization coefficient.

[0012] In a possible implementation of the above-mentioned first aspect, the above-mentioned operation result of the third operation based on the quantization result includes: when the common quantization coefficient is the second quantization coefficient, the operation result of the third operation is obtained according to the third quantization data, the second quantization data, and the second quantization coefficient; when the common quantization coefficient is the first quantization coefficient, the operation result of the third operation is obtained according to the first quantization data, the fourth quantization data, and the first quantization coefficient.

[0013] In a possible implementation of the first aspect above, the above-mentioned third operation is an associated operation of the first operation and the second operation, including: the first quantized data is the output data of the first operation, the second quantized data is the output data of the second operation, and the input data of the third operation includes the first quantized data and the second quantized data.

[0014] In a second aspect, the present application provides a model quantization device, the device comprising:

[0015] A common quantization coefficient determination module is used to obtain the first quantization coefficient of the first operation, the first quantization data of the first operation, the second quantization coefficient of the second operation, and the second quantization data of the second operation, and determine the common quantization coefficient based on the size relationship between the first quantization coefficient and the second quantization coefficient; a data quantization module is used to quantize the first quantization data or the second quantization data based on the common quantization coefficient to obtain a quantization result; and an operation module is used to obtain an operation result of the third operation based on the quantization result.

[0016] In a third aspect, the present application provides an electronic device comprising: a memory for storing instructions executed by one or more processors of the electronic device, and a processor, which is one of the one or more processors of the electronic device, for executing the instructions to enable the electronic device to implement any one of the model quantization methods provided by the above-mentioned first aspect and various possible implementations of the above-mentioned first aspect.

[0017] In a fourth aspect, an embodiment of the present application provides a readable medium having instructions stored thereon, which, when executed by an electronic device, enables the electronic device to implement any one of the model quantization methods provided by the first aspect and various possible implementations of the first aspect.

[0018] In a fifth aspect, an embodiment of the present application provides a computer program product, which includes instructions. When the instructions are executed by an electronic device, the electronic device implements any model quantization method provided by the above-mentioned first aspect and various possible implementations of the above-mentioned first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1A According to some embodiments of the present application, a schematic diagram of an application of a GRU model is shown;

[0020] Figure 1B According to some embodiments of the present application, a structural diagram of a GRU model is shown;

[0021] Figure 2 According to some embodiments of the present application, a schematic structural diagram of an LSTM model is shown;

[0022] Figure 3According to some embodiments of the present application, a schematic diagram of a model quantization scenario is shown;

[0023] Figure 4 According to some embodiments of the present application, a flowchart of a GRU model quantization method is shown;

[0024] Figure 5 According to some embodiments of the present application, a flowchart of a LSTM model quantization method is shown;

[0025] Figure 6 According to some embodiments of the present application, a schematic structural diagram of a model quantization device is shown;

[0026] Figure 7 According to some embodiments of the present application, a structural block diagram of an electronic device 10 is shown;

[0027] Figure 8 According to some embodiments of the present application, a structural block diagram of a SOC 1100 is shown. DETAILED DESCRIPTION

[0028] The illustrative embodiments of the present application include, but are not limited to, a model quantization method, apparatus, electronic device, medium, and computer program product. The embodiments of the present application will be further described in detail below with reference to the accompanying drawings.

[0029] Since this application involves the contents of neural network models and model quantization, in order to more clearly illustrate the solutions of the embodiments of this application, some contents involved in the embodiments of this application are explained below.

[0030] (1) Neural network model

[0031] Neural network models are complex network systems formed by extensively interconnected processing units (called neurons). They are at the core of artificial intelligence and a branch of it. Neural network models have a wide range of applications, including data mining, data classification, computer vision, natural language processing (NLP), biometric recognition, search engines, medical diagnosis, securities market analysis, DNA sequencing, speech and handwriting recognition, strategic gaming, and robotics. These include, but are not limited to, convolutional neural network models, recurrent neural network models, and deep neural network models.

[0032] (2) Convolutional Neural Network (CNN)

[0033] A convolutional neural network is a multi-layered neural network, where each layer consists of multiple two-dimensional planes, each of which is composed of multiple independent neurons. The neurons in each plane share weights, which reduces the number of parameters in the neural network. Currently, in convolutional neural networks, the processor typically performs a convolution operation by converting the convolution of input signal features with weights into a matrix multiplication between a feature map matrix and a weight coefficient matrix.

[0034] (3) Recurrent Neural Network (RNN) Model

[0035] A recursive neural network is a type of network that takes a sequence of data as input (for example, a piece of speech data) and performs recursion in the direction of the sequence's evolution, with all nodes (recurrent units) connected in a chain-like fashion. RNN models can be applied to fields such as speech recognition, language modeling, and machine translation, as well as various time series forecasting applications.

[0036] The RNN model includes at least one of the following: a bidirectional recurrent neural network (Bi-RNN) model, a long short-term memory network (LSTM) model, and a gated recurrent unit (GRU) model.

[0037] (5) Gated Recurrent Unit (GRU) model

[0038] It is a type of Recurrent Neural Network (RNN) model. Like the Long-Short Term Memory (LSTM) model, it was proposed to solve problems such as long-term memory and gradients in backpropagation.

[0039] Figure 1A FIG. 1 shows an application diagram of a GRU model 100. Figure 1A As shown, the input data of the GRU model 100 is the speech to be recognized, and the output data of the GRU model 100 is the text corresponding to the speech to be recognized.

[0040] In some other embodiments, the GRU model 100 can also be used for text translation in different languages. For example, the input data of the GRU model 100 can be Chinese text, and the output data of the GRU model 100 can be English text corresponding to the Chinese text. The GRU model 100 can also be used for image classification. For example, the input data of the GRU model 100 can be multiple frames of images, and the output data of the GRU model 100 can be the image type corresponding to each frame of image. It can be understood that the GRU model 100 is mainly used to process and predict sequence data. According to actual applications, this application does not specifically limit the content of the recognition of the GRU model 100.

[0041] Figure 1B FIG. 1 shows a schematic diagram of the structure of a GRU model 100. Figure 1B As shown, the GRU model 100 includes n GRU networks, namely GRU1, GRU2, ..., GRUt-1, GRUt..., GRUn. GRU1 represents the GRU network at the first time step, GRU2 represents the GRU network at the second time step, GRUt-1 represents the GRU network at the t-1th time step, GRUt represents the GRU network at the tth time step, and GRUn represents the GRU network at the nth time step. The time step is the time step length, which is used to describe the time series of the input data.

[0042] For example, Figure 1B As shown, { 、 、…、 、 ...、 } is the speech data to be recognized, where The speech input data for the GRU network (GRU1) at the first time step, The speech input data for the GRU network (GRU2) at the second time step, is the input data of the GRU network (GRUt) at the tth time step, ..., The speech input data of the GRU network (GRUn) at the nth time step. The output data of the GRU model 100 { 、 、…、 、 ...、 } can be the text corresponding to the speech to be recognized, where is the output data of the GRU network (GRU1) at the first time step, is the output data of the GRU network (GRU2) at the second time step, is the output data of the GRU network (GRUt) at the tth time step, ..., is the output data of the GRU network (GRUn) at the nth time step.

[0043] It is understood that the input data or output data of the GRU network at each time step can be a matrix, tensor, vector, etc. For the convenience of description, in the following scheme introduction, the input data or output data of the GRU network is taken as a matrix as an example to illustrate the data processing involving the input or output of each layer of the neural network. It is not difficult to see that since the GRU model 100 is mainly used to process and predict sequence data, the GRU network at the current time step needs to combine the output data of the previous time step to process the input data of the current time step and generate the output data of the current time step.

[0044] For example, during the operation of the GRU network at the t-th time step of the GRU model 100, the input data input to the GRU network (GRUt) at the t-th time step can be Output data of the GRU network (GRUt-1) at the t-1th time step , weight coefficient matrix and error coefficient matrix, etc., to determine the output data of the GRU network (GRUt) at the tth time step Specifically, the output data of the GRU network (GRUt) at the tth time step is It can be calculated by the following formula:

[0045] (1)

[0046] (2)

[0047] (3)

[0048] (4)

[0049] In formula (1) to formula (4), 、 、 、 、 、 、 Respectively The weight coefficient matrix, 、 、 Respectively The error coefficient matrix, represents the input data at the tth time step, Represents the output data at the t-1th time step. Represents the sigmoid function. The calculation formula of the sigmoid function can be expressed as , the calculation formula of tanh function is f . represents the gate value of the update gate at the t-th time step, represents the gate value of the reset gate at the t-th time step, Represents the intermediate output result of the t-th time step, " means that the data with the same row and column numbers in the two matrices are multiplied accordingly," ” indicates matrix multiplication.

[0050] (6) Long Short-Term Memory Networks (LSTM) Model

[0051] It is a special RNN model. In the LSTM model 200, the states and outputs of the hidden neural network layers can be communicated between them. That is, the output of each hidden neural network layer is related to the input of the input layer, the output of the previous neural network layer, and the state of the previous neural network layer. The hidden neural network layer can filter the input data through a gate structure, that is, forgetting the unimportant and selecting the important ones. Correspondingly, the gate structure includes a forget gate, a select gate, and an output gate. Therefore, compared with ordinary RNNs, LSTM can perform better on longer sequences.

[0052] Figure 2 The schematic diagram of the structure of an LSTM model 200 is shown. The LSTM model 200 includes an input layer 110, hidden layers 120-1 to 120-t (hereinafter referred to as neural network layers 120-1 to 120-t), and an output layer 130. The input data of the input layer 110 include x1, x2, x3, ..., x t The data output by the output layer 130 includes h1, h2, h3, ..., h t The states c and outputs h of the neural network layers 120 can be transmitted between the neural network layers 120, such as states c1 and c2 and outputs h1 and h2.

[0053] For example, in the operation process of the neural network layer 120-t of the LSTM model 200, the input data of the input layer 110 can be used to calculate the input data of the input layer 110. Data output by the output layer 130 , the state of neural network layer 120-t-1 , weight coefficient matrix and error coefficient matrix, etc., to determine the data output by the output layer 130 Specifically, the data output by the output layer 130 is It can be calculated by the following formula:

[0054] (5)

[0055] (6)

[0056] (7)

[0057] (8)

[0058] (9)

[0059] (10)

[0060] In formula (5) to formula (10), 、 、 、 、 、 、 、 denote the weight coefficient matrices of the neural network layer 120-t, 、 、 、 denote the error coefficient matrix of the neural network layer 120-t, Represents the input data of the input layer 110 , represents the data output by the output layer 130. Represents the sigmoid function. The calculation formula of the sigmoid function can be expressed as . Represents the tanh function, the calculation formula of the tanh function is f . represents the gate value of the update gate at the t-th time step, represents the gate value of the reset gate at the t-th time step, represents the state of the neural network layer 120-t-1, Represents matrix multiplication. Represents the dot product operation of a matrix.

[0061] (7) Data quantization (i.e., converting floating-point data into quantized data)

[0062] It can refer to converting a large data type (e.g., a 32-bit floating point number) into a smaller data type (e.g., 8-bit symmetrical quantized data or asymmetrical quantized data). For example, the data quantization formula for converting a floating point number into quantized data is as follows:

[0063] The quantitative data Q can be calculated using the following formula:

[0064] (11)

[0065] (12)

[0066] in, Represents the maximum value of a floating-point number. Represents the minimum value of a floating-point number. Indicates the maximum value of the quantized data. Indicates the minimum value of quantized data. F represents a floating-point number. S represents the floating-point-to-quantization range conversion factor (i.e., the quantization factor).

[0067] (8) Data dequantization (i.e., converting quantized data into floating-point data)

[0068] It can refer to converting a smaller data type (e.g., 8-bit symmetrical quantized data or asymmetrical quantized data) into a larger data type (e.g., 32-bit floating-point number). For example, the dequantization formula for converting quantized data into floating-point numbers is as follows:

[0069] The floating point number F can be calculated using the following formula:

[0070] (13)

[0071] Where Q represents quantized data, and S represents the floating-point to quantized range conversion coefficient (i.e., quantization coefficient).

[0072] (9) Operator

[0073] Neural network models are composed of computational units, which we call operators (Op). In neural network model operations, operators correspond to the computational logic within the neural network layer. Operators in neural network models can be categorized by operation type: linear operators and nonlinear operators.

[0074] Nonlinear operator: a nonlinear function, also known as a nonlinear mapping, an operator that does not satisfy linear conditions.

[0075] Linear operator: a linear function, an operator that satisfies linear conditions.

[0076] Taking the t-th GRU network of GRU model 100 as an example, for example, in formula (1), can be an operator, and is a linear operator, is also a linear operator, is also a linear operator. is a nonlinear operator, is a nonlinear operator, is a linear operator.

[0077] (10) Model quantification

[0078] Model quantization refers to converting the weight matrix data, input data, and output data of each node in the neural network model from a high-precision quantization level to a low-precision quantization level for calculation, such as converting 32-bit single-precision floating-point numbers (FP32) into 8-bit integer data (INT8).

[0079] Taking the model quantization of the GRU model 100 as an example, the GRU model 100 quantizes the input data to be input into the GRU model 100, the weight matrix data involved in the GRU model 100, and the data output by the GRU model 100. For example, before the GRU model 100 is operated, the input data, weight matrix data, and output data of the GRU model 100 are converted from 32-bit single-precision floating point numbers (FP32) to 8-bit integer data (INT8).

[0080] As previously mentioned, quantized neural network models can experience computational precision loss. For example, because the weight matrix data and input data in the GRU model 100 are converted from a high-precision order of magnitude to a low-precision quantization level, computational precision loss may occur. Furthermore, each operator in a current neural network typically has multiple sub-operators, and these multiple sub-operators are typically independently quantized. Because the input data and weight matrix data used to determine the quantization coefficients of each sub-operator have different quantization coefficients, some sub-operators may have inconsistent quantization coefficients. When the quantization coefficients of multiple sub-operators being computed are inconsistent, greater precision loss occurs, resulting in lower accuracy in the data ultimately output by the model.

[0081] Specifically, for example, in the operation of formula (1), when calculating the operator When the value of Quantized data and sub-operators The quantization coefficients corresponding to the quantized data are inconsistent, so the sub-operator Quantized data and sub-operators The quantized data is added to the matrix, and the main operator is calculated. The quantitative data is inaccurate, resulting in low accuracy of the quantitative model.

[0082] For example, in the non-quantized model, the sub-operator in formula (1) The floating point data is 0.2. The floating point data is 0.5. Floating point data and sub-operators The floating-point data is added to the matrix, and the operator is calculated. The floating point data is 0.7.

[0083] In the quantized model, the sub-operator The quantized data is 20, the sub-operator The quantization coefficient is 0.01, the sub-operator The quantized data is 100, the sub-operator The quantization coefficient is 0.005. Quantized data and sub-operators The quantized data is added to the matrix, and the operator is calculated. The quantitative data is 120.

[0084] It is not difficult to see that whether it is based on the sub-operator The quantization coefficient 0.01 is used as the operator The quantized coefficients of the operator The quantized data is dequantized and the floating point data obtained is 1.2. The quantization coefficient of 0.005 is used as the operator The quantized coefficients of the operator The quantized data is dequantized and the floating point data obtained is 0.6. The floating point data of 0.7 has a deviation. Therefore, the calculated operator The calculation accuracy of quantized data is low, which leads to the problem of precision loss in the quantized GRU model.

[0085] To solve the above problem, the present application provides a model quantization method. Before the quantized data of the first operator and the quantized data of the second operator perform the operation corresponding to the third operator, the first quantized data of the first operator and the first quantized data of the second operator can be dequantized according to the quantization coefficient of the first operator and the quantization coefficient of the second operator to obtain the floating-point data of the first operator and the floating-point data of the second operator.

[0086] Then, based on the comparison of the quantization coefficients of the first operator and the second operator, the larger quantization coefficient of the quantization coefficients of the first operator and the second operator is determined as the common quantization coefficient of the first operator and the second operator, and the floating-point data of the first operator and the floating-point data of the second operator are quantized to obtain the second quantization data of the first operator and the second quantization data of the second operator. Then, based on the second quantization data of the first operator and the second quantization data of the second operator, the operation of the third operator is performed to generate the quantization data of the third operator. Among them, the third operator is the corresponding operation with an associated relationship between the first operator and the second operator. For example, in the above formula (1), the first operator is , the second operator is The third operator is the addition of the first and second operators. .

[0087] It can be understood that the third operator is a corresponding operation associated with the first operator and the second operator, which means that the operation corresponding to the third operator uses the operation result corresponding to the first operator and the operation result corresponding to the second operator as input data, or that the operation corresponding to the first operator and the operation corresponding to the second operator are sub-operations of the operation corresponding to the third operator. In addition, in some embodiments, when the output data of the first operation and the second operation are input data of the third operation, or the first operation and the second operation are sub-operations of the third operation, the third operation can be referred to as an associated operation of the first and second operations.

[0088] It can be understood that the quantization coefficient of the first operator mentioned above refers to the product of the quantization coefficient of the weight coefficient matrix of the first operator and the quantization coefficient of the input data matrix, and the quantization coefficient of the second operator mentioned above refers to the product of the quantization coefficient of the weight coefficient matrix of the second operator and the quantization coefficient of the output data matrix.

[0089] Based on the above scheme, the calculation result of the third operator can be made more accurate. When the entire model adopts the above scheme to adjust the quantization coefficient of each operator, the quantization accuracy of the neural network model can be effectively improved. In addition, since the data processed in the GRU model or LSTM type is related to time, that is, the first operator and the second operator need to be requantized at each time step, the method provided in the embodiment of the present application only needs to requantize the data of one of the first operator and the second operator, which can reduce the number of times the quantized data is dequantized into floating-point numbers and then the floating-point numbers are quantized into quantized data, thereby improving the speed at which electronic devices run the GRU model or LSTM model.

[0090] It is not difficult to understand that the model quantization method of the present application can be applied to the neural network model that has been quantized. By optimizing the neural network model that has been quantized, the accuracy loss of the quantized neural network model can be avoided and the quantization accuracy of the quantized neural network model can be improved.

[0091] For example, in the GRU model that has been initially quantized above, when calculating the operator When the value of (ie the first operator) quantization coefficient 0.01 and the sub-operator (i.e. the second operator) has a quantization coefficient of 0.005, and the sub-operator (ie, the first operator) first quantized data 20 and operator The quantized data 100 (i.e. the second operator) is dequantized to generate the operator The first quantized data corresponds to the floating point data 0.2 (ie 20×0.01) and the operator The floating-point data corresponding to the first quantized data is 0.5 (ie, 100×0.005).

[0092] By comparing the operator The quantization coefficient 0.01 and the operator The quantization coefficient is 0.005, which can be expressed as AND operator The larger quantization coefficient among the quantization coefficients is used as the common quantization coefficient, and the operator is AND operator Quantize the floating-point data to generate operators The second quantized data 20 and the operator The second quantized data 50. According to the operator The second quantized data 20 and the operator The second quantized data 50 generates the main operator The quantized data is 70. It is not difficult to see that the public quantization coefficient 0.01 is used as the main operator The quantized coefficients of the main operator The quantized data 70 is dequantized to generate the main operator The floating point data is 0.7. The main operator of the non-quantized model above The floating point data is consistent with 0.7. This proves that the model quantization method provided by this application can improve the calculation accuracy of the quantization model and further improve the accuracy of the output data of the quantization model.

[0093] Below, combined with Figures 1 to Figure 7 The technical solution of model quantization of this application is described in detail.

[0094] Figure 3 According to an embodiment of the present application, a model quantized scene graph is shown. Figure 3 As shown, the scenario includes a first electronic device 10, a second electronic device 20, and a user A.

[0095] like Figure 3 As shown, the first electronic device 10 can collect a speech to be recognized from user A, where the speech to be recognized can be "Xiao A, how's the weather today?" The first electronic device 10 can run the quantized GRU model 100 and input the speech to be recognized as input data into the quantized GRU model 100. The output data of the quantized GRU model 100 is the text corresponding to the speech to be recognized.

[0096] like Figure 3As shown, the first electronic device 10 can receive quantization coefficients, model coefficients, etc. of the GRU model 100 sent by the second electronic device 20. The quantization coefficients and model coefficients can be used to quantize the GRU model 100. The quantization coefficients can be quantization coefficients of a weight coefficient matrix, quantization coefficients of an input data matrix, etc. The model coefficients can be a weight coefficient matrix, an error coefficient matrix, etc.

[0097] Specifically, for example, the second electronic device 20 may send the floating-point data of the weight coefficient matrix, the floating-point data of the error coefficient matrix, the quantized coefficients of the weight coefficient matrix, the quantized coefficients of the input data matrix, etc. in the GRU model 100 to the first electronic device 10. The second electronic device 20 may calculate the quantized data of the weight coefficient matrix and the quantized data of the error coefficient matrix according to the floating-point data of the weight coefficient matrix, the floating-point data of the error coefficient matrix, the quantized coefficients of the weight coefficient matrix, etc. in the GRU model 100 using the above formula (12).

[0098] In some embodiments, when the first electronic device 10 quantizes the GRU model 100, to improve the accuracy of the quantized GRU model 100 running on the first electronic device 10, the first electronic device 10 not only converts the weight matrix data and input data input to the GRU model 100 from high-precision floating-point data to corresponding low-precision quantized data, but also compares the quantization coefficient of a first operator with the quantization coefficient of a second operator in the GRU model 100, uses the larger quantization coefficient of the first operator and the quantization coefficient of the second operator in the GRU model 100 as a common quantization coefficient, and re-determines the quantization data of the first operator or the second operator in the GRU model 100 based on the common quantization coefficient, thereby ensuring that the quantization coefficient corresponding to the quantization data of the first operator in the GRU model 100 is consistent with the quantization coefficient corresponding to the quantization data of the second operator in the GRU model 100 (i.e., both are common quantization coefficients). As a result, the quantization data of the third operator in the GRU model 100 calculated by the first electronic device 100 based on the re-determined quantization data of the first and second operators in the GRU model 100 is more accurate, thereby improving the quantization accuracy of the GRU model 100.

[0099] Therefore, it is not difficult to understand that the first electronic device 10 runs the GRU model 100 with higher quantization accuracy provided by this application. The GRU model 100 with higher quantization accuracy takes the data of the speech to be recognized (for example, "Xiao A, how is the weather today") as input data, performs operations on the data of the speech to be recognized, and the text corresponding to the generated speech to be recognized is more accurate.

[0100] The first electronic device 10 in this application may include, but is not limited to, a smart mobile phone, a television, a tablet computer, a wristband, a head-mounted display (HMD), an augmented reality (AR) device, a mixed reality (MR) device, a cellular phone, a smartphone, a personal digital assistant (PDA), a tablet computer, an in-vehicle electronic device, a laptop computer, a personal computer (PC), a monitoring device, a robot, an in-vehicle terminal, an autonomous vehicle, etc. Of course, in the following embodiments, there is no limitation on the specific form of the first electronic device 10.

[0101] The second electronic device 20 of the present application may include, but is not limited to, a laptop computer, a handheld computer, a notebook computer, a desktop computer, an ultra-mobile personal computer (UMPC), a server, and the like. As an example, the server may be a cloud server, which may be a hardware server or embedded in a virtualized environment. For example, the server may be a virtual machine executing on a hardware server that includes one or more other virtual machines. Furthermore, the server may be a standalone server, which possesses all the software and hardware resources of the entire server and can independently allocate and perform various services, such as generating quantization coefficients and model coefficients for the neural network model of the present application.

[0102] It is not difficult to understand that the neural network model mentioned in the technical solution of this application can be a recurrent neural network model, such as a GRU model, an LSTM model, a Bi-RNN model, etc. It can also be a convolutional neural network model, a deep learning model, etc. It is understood that, depending on the actual application, this application does not specifically limit the neural network model applicable to model quantization.

[0103] The following takes the GRU model 100 as an example of a neural network model to describe in detail the specific process of quantizing the GRU model 100 of the present application.

[0104] Combine Figure 3 The scene, Figure 4 FIG. 1 shows a flow chart of a GRU model 100 quantization method. Figure 4 As shown, the execution subject of the method may be the first electronic device 10, and the method includes the following steps:

[0105] S401: Obtain the first quantized data of the first operator, the first quantized data of the second operator, the quantized coefficient of the first operator, and the quantized coefficient of the second operator in the GRU model 100, wherein the third operator in the GRU model 100 is the corresponding operation with an associated relationship between the first operator and the second operator.

[0106] In some embodiments, the first electronic device 10 obtains the first quantized data of the first operator, the first quantized data of the second operator, the quantized coefficient of the first operator, and the quantized coefficient of the second operator in the GRU model 100, wherein the third operator is the associated operation relationship between the first operator and the second operator.

[0107] In some embodiments, the first operator and / or the second operator may be a linear operator or a nonlinear operator. The third operator may be a linear operator or a nonlinear operator. It is understood that, depending on the actual application, this application does not specifically limit the operator types of the first operator, the second operator, and the third operator.

[0108] Specifically, for example, in the operation of the t-th GRU network of the GRU model 100, the first operator may be the operator of formula (1): , the second operator can be the operator of formula (1) , the third operator can be the operator of formula (1) The first operator can also be the operator of formula (1) , the second operator can also be the operator of formula (1) , the corresponding third operator can be the operator of formula (1) .

[0109] Similarly, in the operation of the t-th GRU network of the GRU model 100, the first operator can be the operator of formula (2) , the second operator can be the operator of formula (2) , the third operator can be the operator of formula (2) The first operator can also be the operator of formula (2) , the second operator can also be the operator of formula (2) , the corresponding third operator can be the operator of formula (2) .

[0110] Similarly, in the operation of the t-th GRU network of the GRU model 100, the first operator can be the operator of formula (3) , the second operator can be the operator of formula (3) , the third operator can be the operator of formula (3) The first operator can also be the operator of formula (3) , the second operator can also be the operator of formula (3) , the corresponding third operator can be the operator of formula (3) .

[0111] Similarly, in the operation of the t-th GRU network of the GRU model 100, the first operator can be the operator of formula (4) , the second operator can be the operator of formula (4) , the third operator can be the operator of formula (4) .

[0112] In some embodiments, the first electronic device 10 may obtain floating-point data of the weight coefficient matrix, floating-point data of the error coefficient matrix, quantized coefficients of the weight coefficient matrix, quantized coefficients of the input data matrix, etc. in the GRU model 100 from the second electronic device 20. The first electronic device 10 may also obtain floating-point data of the input data matrix. The first electronic device 10 may calculate the quantized data of the weight coefficient matrix, the quantized data of the error coefficient matrix, the quantized coefficients of the input data matrix, and the floating-point data of the input data matrix using the above formula (12). For example, the quantized data of the input data matrix is equal to the floating-point data of the input data matrix divided by the quantized coefficients of the input data matrix.

[0113] In some other embodiments, the second electronic device 20 can calculate the quantized data of the weight coefficient matrix and the quantized data of the error coefficient matrix using the above formula (12) based on the floating-point data of the weight coefficient matrix, the floating-point data of the error coefficient matrix, and the quantized coefficients of the weight coefficient matrix. Then, the first electronic device 10 can directly obtain the quantized data of the weight coefficient matrix and the quantized data of the error coefficient matrix in the GRU model 100 from the second electronic device 20.

[0114] In some embodiments, the first electronic device 10 can calculate the first quantized data of the first operator and the first quantized data of the second operator based on the quantized data of the weight coefficient matrix of the GRU model 100, the quantized data of the error coefficient matrix, and the quantized data of the input data matrix.

[0115] For example, in the operation of the t-th GRU network of the GRU model 100, the first operator may be the operator of formula (1) , the second operator can be the operator of formula (1) Specifically, the first electronic device 10 can use the weight coefficient matrix Quantized data and the input data matrix of the tth time step The quantitative data is obtained by transforming the input data matrix of the t-th time step into Quantized data and weight coefficient matrix The quantized data is multiplied by matrix to calculate the first operator The first quantitative data.

[0116] Specifically, the first electronic device 10 can also use the weight coefficient matrix Quantized data, output data matrix of the t-1 time step Quantized data and error coefficient matrix The quantitative data is obtained by converting the output data matrix of the t-1 time step into Quantized data and weight coefficient matrix The quantized data is matrix multiplied, and then the output data of the t-1 time step is Quantized data and weight coefficient matrix The result of matrix multiplication of the quantized data and the error coefficient matrix The quantized data is summed up by matrix, and the second operator is finally calculated. The first quantitative data.

[0117] In some embodiments, the first electronic device 10 may calculate the quantization coefficients of the first operator and the quantization coefficients of the second operator based on the quantization coefficients of the weight coefficient matrix of the GRU model 100 and the quantization coefficients of the input data matrix.

[0118] For example, in the operation of the t-th GRU network of the GRU model 100, the first operator may be the operator of formula (1) , the second operator can be the operator of formula (1) Specifically, the first electronic device 10 can use the weight coefficient matrix The quantization coefficients and the input data matrix of the tth time step The quantization coefficients are obtained by transforming the input data matrix of the t-th time step into Quantization coefficient and weight coefficient matrix The quantized coefficients are multiplied by the matrix to calculate the first operator Specifically, the first electronic device 10 can also calculate the quantization coefficient of Quantization coefficients, output data matrix of the t-1th time step The quantization coefficients are obtained by transforming the output data matrix of the t-1th time step into Quantization coefficient and weight coefficient matrix The quantized coefficients are multiplied by the matrix to calculate the second operator quantization coefficients.

[0119] In some other embodiments, when the first operator and / or the second operator is the main operator of other operators, the first quantized data of the first operator and / or the first quantized data of the second operator may also be quantized data calculated according to the model quantization method provided in this application. For example, in the operation of the t-th GRU network of the GRU model 100, the first operator may also be the operator of formula (1): , the second operator can also be the operator of formula (1) , where the second operator Can be an operator Sum Operator The quantized data of the second operator can be the operator Quantitative data and operators The quantized data matrix of is summed up. Among them, the operator The quantization coefficients and operators corresponding to the quantized data The quantization coefficients corresponding to the quantized data are the same.

[0120] S402: Determine whether the quantization coefficient of the first operator in the GRU model 100 is greater than the quantization coefficient of the second operator. If the quantization coefficient of the first operator in the GRU model 100 is greater than the quantization coefficient of the second operator, execute step S403. If the quantization coefficient of the first operator is not greater than the quantization coefficient of the second operator, execute step S404.

[0121] In some embodiments, the first electronic device 10 can determine whether the quantization coefficient of the first operator in the GRU model 100 is greater than the quantization coefficient of the second operator, and select the larger quantization coefficient of the first operator or the quantization coefficient of the second operator as the common quantization coefficient of the first operator and the second operator, so that when the quantization data of the first operator and the quantization data of the second operator perform the third operator operation, the quantization coefficients corresponding to the quantization data of the first operator and the quantization data of the second operator are the same, thereby improving the operation accuracy of the quantized GRU model 100.

[0122] S403: When the quantization coefficient of the first operator in the GRU model 100 is greater than the quantization coefficient of the second operator, the quantization coefficient of the first operator in the GRU model 100 is used as the common quantization coefficient of the first operator and the second operator to determine the precision coefficient of the second operator.

[0123] To improve the operational accuracy of model quantization, before performing a third operator operation on the quantized data of the first operator and the quantized data of the second operator, the first electronic device 10 may inversely quantize the first quantized data of the first operator and the first quantized data of the second operator to obtain floating-point data of the first operator and floating-point data of the second operator. Then, based on the determined common quantization coefficients of the first and second operators, the first electronic device 10 may quantize the floating-point data of the first operator and the floating-point data of the second operator to obtain second quantized data of the first operator and second quantized data of the second operator. The first electronic device 10 then performs a third operator operation based on the second quantized data of the first operator and the second quantized data of the second operator to generate quantized data of the third operator.

[0124] It is not difficult to understand that if the quantization coefficient of the first operator is used as the common quantization coefficient for the first and second operators, and the quantization coefficient used when dequantizing the first quantized data of the first operator is the same as the quantization coefficient used when quantizing the floating-point data of the first operator, then the first quantized data of the first operator is the same as the second quantized data of the first operator. Therefore, the first electronic device 10 does not need to perform additional dequantization and quantization operations on the first quantized data of the first operator. In addition, the first electronic device 10 can determine the second quantized data of the second operator by determining the precision coefficient of the second operator based on the precision coefficient of the second operator.

[0125] Specifically, the precision coefficient of the second operator can be used to make the quantization coefficient used in the second quantized data generated by the second operator consistent with the quantization coefficient of the first quantized data of the first operator. The first electronic device 10 can calculate the precision coefficient of the second operator based on the quantization coefficient of the first operator and the quantization coefficient of the second operator. Specifically, the precision coefficient of the second operator It can be calculated by the following formula (14):

[0126] = (14)

[0127] In formula (14), represents the quantized coefficient of the first operator, Represents the quantized coefficient of the second operator.

[0128] S405: Determine the second quantized data of the second operator according to the precision coefficient of the second operator in the GRU model 100 and the first quantized data of the second operator.

[0129] In some embodiments, the first electronic device 10 can determine the second quantized data of the second operator based on the precision coefficient of the second operator in the GRU model 100 and the first quantized data of the second operator, wherein the quantized coefficient corresponding to the second quantized data of the second operator is the same as the quantized coefficient corresponding to the first data of the first operator. Specifically, the second quantized data of the second operator It can be calculated by the following formula (15):

[0130] = * (15)

[0131] In formula (15), represents the first quantized data of the second operator, Indicates the precision coefficient of the second operator.

[0132] S406: Determine the quantized data of the third operator in the GRU model 100 according to the first quantized data of the first operator and the second quantized data of the second operator in the GRU model 100.

[0133] In some embodiments, the third operator in the GRU model 100 is an associated operation between the first operator and the second operator. The first electronic device 10 can determine the quantization data of the third operator based on the first quantization data of the first operator and the second quantization data of the second operator in the GRU model 100, wherein the quantization coefficient corresponding to the first quantization data of the first operator and the quantization coefficient corresponding to the second quantization data of the second operator are the same.

[0134] For example, in the operation of the t-th GRU network of the GRU model 100, the first operator may be the operator of formula (1) , the second operator can be the operator of formula (1) , the third operator can be the operator Specifically, the third operator The quantized data can be the first operator The first quantized data and the second operator The second quantized data is obtained by performing matrix addition calculation.

[0135] S404: When the quantization coefficient of the first operator in the GRU model 100 is not greater than the quantization coefficient of the second operator, determine whether the quantization coefficient of the first operator in the GRU model 100 is equal to the quantization coefficient of the second operator.

[0136] If the quantization coefficient of the first operator in the GRU model 100 is not greater than the quantization coefficient of the second operator, and the quantization coefficient of the first operator in the GRU model 100 is equal to the quantization coefficient of the second operator, step S407 is performed. If the quantization coefficient of the first operator in the GRU model 100 is not greater than the quantization coefficient of the second operator, and the quantization coefficient of the first operator in the GRU model 100 is not equal to the quantization coefficient of the second operator, step S408 is performed.

[0137] S407: When the quantization coefficient of the first operator in the GRU model 100 is not greater than the quantization coefficient of the second operator, and the quantization coefficient of the first operator in the GRU model 100 is equal to the quantization coefficient of the second operator, the quantization data of the third operator in the GRU model 100 is determined based on the first quantization data of the first operator in the GRU model 100 and the first quantization data of the second operator.

[0138] S408: When the quantization coefficient of the first operator is not greater than the quantization coefficient of the second operator and the quantization coefficient of the first operator is not equal to the quantization coefficient of the second operator, the quantization coefficient of the second operator in the GRU model 100 is used as the common quantization coefficient of the first operator and the second operator to determine the precision coefficient of the first operator.

[0139] In order to improve the operational accuracy of model quantization, when the first operator and the second operator perform the corresponding operation of the third operator, the first electronic device 10 can obtain the floating-point data of the first operator and the floating-point data of the second operator by dequantizing the first quantized data of the first operator and the first quantized data of the second operator, and then quantize the floating-point data of the first operator and the floating-point data of the second operator according to the determined common quantization coefficient of the first operator and the second operator to obtain the second quantized data of the first operator and the second quantized data of the second operator. It is not difficult to understand that if the quantization coefficient of the second operator is used as the common quantization coefficient of the first operator and the second operator, the quantization coefficient used when dequantizing the first quantized data of the second operator is the same as the quantization coefficient used when quantizing the floating-point data of the second operator, then the first quantized data of the second operator is the same as the second quantized data of the second operator. Therefore, the first electronic device 10 does not need to perform additional dequantization and quantization operations on the first quantized data of the second operator.

[0140] In some embodiments, the precision coefficient of the first operator is used to make the quantization coefficient corresponding to the second quantization data of the first operator generated consistent with the quantization coefficient corresponding to the first quantization data of the second operator. The first electronic device 10 can calculate the precision coefficient of the first operator based on the quantization coefficient of the first operator and the quantization coefficient of the second operator. Specifically, the precision coefficient of the first operator It can be calculated by the following formula (16):

[0141] = (16)

[0142] In formula (16), represents the quantized coefficient of the first operator, Represents the quantized coefficient of the second operator.

[0143] S409: Determine the second quantization data of the first operator in the GRU model 100 according to the precision coefficient of the first operator in the GRU model 100 and the first quantization data of the first operator.

[0144] In some embodiments, the first electronic device 10 can determine the second quantized data of the first operator based on the precision coefficient of the first operator in the GRU model 100 and the first quantized data of the first operator, wherein the quantized coefficient corresponding to the second quantized data of the first operator is the same as the quantized coefficient corresponding to the first quantized data of the second operator. Specifically, the second quantized data of the first operator It can be calculated by the following formula (17):

[0145] = * (17)

[0146] In formula (17), represents the first quantized data of the first operator, Indicates the precision coefficient of the first operator.

[0147] S410: Determine the quantized data of the third operator in the GRU model 100 based on the first quantized data of the second operator in the GRU model 100 and the second quantized data of the first operator.

[0148] In some embodiments, the third operator in the GRU model 100 is an associated operation between the first operator and the second operator. The first electronic device 10 can determine the quantization data of the third operator based on the first quantization data of the second operator and the second quantization data of the first operator in the GRU model 100, wherein the quantization coefficient corresponding to the first quantization data of the second operator is the same as the quantization coefficient corresponding to the second quantization data of the first operator.

[0149] For example, in the operation of the t-th GRU network of the GRU model 100, the first operator may be the operator of formula (1) , the second operator can be the operator of formula (1) , the third operator can be the operator Specifically, the third operator The quantized data can be the first operator The second quantized data and the second operator The first quantized data is obtained by performing matrix addition calculation.

[0150] It is not difficult to see from the above description that in order to improve the operational accuracy of model quantization, the first electronic device 10 can compare the quantization coefficient of the first operator with the quantization coefficient of the second operator, and use the larger quantization coefficient of the first operator and the quantization coefficient of the second operator as the common quantization coefficient. According to the common quantization coefficient, the quantization data of the first operator or the second operator is re-determined, so that the quantization coefficient corresponding to the quantization data of the first operator is consistent with the quantization coefficient corresponding to the quantization data of the second operator (that is, both are common quantization coefficients), so that the first electronic device 10 can calculate the quantization data of the third operator more accurately based on the re-determined quantization data of the first operator and the second operator, thereby improving the quantization accuracy of the GRU model 100.

[0151] It is not difficult to see that Figure 4 The quantization process of the GRU model 100 is specifically described. In some other embodiments, the model quantization method of the present application is also applicable to the LSTM model 200. The following takes the LSTM model 200 as an example of the neural network model to describe in detail the specific quantization process of the LSTM model 200 of the present application.

[0152] Combine Figure 3 The scene, Figure 5 Schematic diagram of a LSTM model 200 quantization method. Figure 5 As shown, the execution subject of the method may be the first electronic device 10, and the method includes the following steps:

[0153] S501: Obtain the first quantized data of the first operator, the first quantized data of the second operator, the quantized coefficient of the first operator, and the quantized coefficient of the second operator in the LSTM model 200, wherein the third operator in the LSTM model 200 is the corresponding operation with an associated relationship between the first operator and the second operator.

[0154] In some embodiments, the first electronic device 10 obtains the first quantized data of the first operator, the first quantized data of the second operator, the quantized coefficient of the first operator, and the quantized coefficient of the second operator in the LSTM model 200.

[0155] Specifically, for example, in the operation process of the neural network layer 120-t of the LSTM model 200, the first operator may be the operator of formula (5): , the second operator can be the operator of formula (5) , the third operator can be the operator of formula (5) The first operator can also be the operator of formula (5) , the second operator can also be the operator of formula (5) , the corresponding third operator can be the operator of formula (5) .

[0156] Similarly, in the operation process of the neural network layer 120-t of the LSTM model 200, the first operator can be the operator of formula (6) , the second operator can be the operator of formula (6) , the third operator can be the operator of formula (6) The first operator can also be the operator of formula (6) , the second operator can also be the operator of formula (6) , the corresponding third operator can be the operator of formula (6) .

[0157] Similarly, in the operation process of the neural network layer 120-t of the LSTM model 200, the first operator can be the operator of formula (7) , the second operator can be the operator of formula (7) , the third operator can be the operator of formula (7) The first operator can also be the operator of formula (7) , the second operator can also be the operator of formula (7) , the corresponding third operator can be the operator of formula (7) .

[0158] Similarly, in the operation process of the neural network layer 120-t of the LSTM model 200, the first operator can be the operator of formula (8) , the second operator can be the operator of formula (8) , the third operator can be the operator of formula (8) The first operator can also be the operator of formula (8) , the second operator can also be the operator of formula (8) , the corresponding third operator can be the operator of formula (8) .

[0159] Similarly, in the operation process of the neural network layer 120-t of the LSTM model 200, the first operator can be the operator of formula (9) , the second operator can be the operator of formula (9) , the third operator can be the operator of formula (9) .

[0160] The determination process of the first quantized data of the first operator, the first quantized data of the second operator, the quantized coefficient of the first operator, and the quantized coefficient of the second operator in the LSTM model 200 is specifically referred to. Figure 4 Step S401 is not described here in detail.

[0161] S502: Determine whether the quantization coefficient of the first operator in the LSTM model 200 is greater than the quantization coefficient of the second operator. If the quantization coefficient of the first operator in the LSTM model 200 is greater than the quantization coefficient of the second operator, execute step S503. If the quantization coefficient of the first operator is not greater than the quantization coefficient of the second operator, execute step S504.

[0162] In some embodiments, the first electronic device 10 can determine whether the quantization coefficient of the first operator in the LSTM model 200 is greater than the quantization coefficient of the second operator, and select the larger quantization coefficient of the first operator or the quantization coefficient of the second operator as the common quantization coefficient of the first operator and the second operator, so that when the quantization data of the first operator and the quantization data of the second operator perform the third operator operation, the quantization coefficients corresponding to the quantization data of the first operator and the quantization data of the second operator are the same, thereby improving the calculation accuracy of the quantization model.

[0163] S503: When the quantization coefficient of the first operator in the LSTM model 200 is greater than the quantization coefficient of the second operator, the quantization coefficient of the first operator in the LSTM model 200 is used as the common quantization coefficient of the first operator and the second operator to determine the precision coefficient of the second operator. Figure 4 Step S403 is not described here in detail.

[0164] S505: Determine the second quantized data of the second operator according to the precision coefficient of the second operator in the LSTM model 200 and the first quantized data of the second operator. Figure 4 Step S405 is not described in detail here.

[0165] S506: Determine the quantized data of the third operator in the LSTM model 200 based on the first quantized data of the first operator and the second quantized data of the second operator in the LSTM model 200.

[0166] In some embodiments, the third operator in the LSTM model 200 is an associated operation between the first operator and the second operator. The first electronic device 10 can determine the quantized data of the third operator based on the first quantized data of the first operator and the second quantized data of the second operator in the LSTM model 200, wherein the quantization coefficient corresponding to the first quantized data of the first operator and the quantization coefficient corresponding to the second quantized data of the second operator are the same.

[0167] For example, in the operation process of the neural network layer 120-t of the LSTM model 200, the first operator may be the operator of formula (5) , the second operator can be the operator of formula (5) , the third operator can be the operator of formula (5) Specifically, the third operator The quantized data can be the first operator The first quantized data and the second operator The second quantized data is obtained by performing matrix addition calculation.

[0168] S504: When the quantization coefficient of the first operator in the LSTM model 200 is not greater than the quantization coefficient of the second operator, determine whether the quantization coefficient of the first operator in the LSTM model 200 is equal to the quantization coefficient of the second operator.

[0169] If the quantization coefficient of the first operator in the LSTM model 200 is not greater than the quantization coefficient of the second operator, and the quantization coefficient of the first operator in the LSTM model 200 is equal to the quantization coefficient of the second operator, step S507 is performed. If the quantization coefficient of the first operator in the LSTM model 200 is not greater than the quantization coefficient of the second operator, and the quantization coefficient of the first operator in the LSTM model 200 is not equal to the quantization coefficient of the second operator, step S508 is performed.

[0170] S507: When the quantization coefficient of the first operator in the LSTM model 200 is not greater than the quantization coefficient of the second operator, and the quantization coefficient of the first operator in the LSTM model 200 is equal to the quantization coefficient of the second operator, determine the quantization data of the third operator in the LSTM model 200 based on the first quantization data of the first operator in the LSTM model 200 and the first quantization data of the second operator.

[0171] S508: When the quantization coefficient of the first operator is not greater than the quantization coefficient of the second operator and the quantization coefficient of the first operator is not equal to the quantization coefficient of the second operator, the quantization coefficient of the second operator in the LSTM model 200 is used as the common quantization coefficient of the first operator and the second operator to determine the precision coefficient of the first operator. Figure 4 Step S408 is not described in detail here.

[0172] S509: Determine the second quantized data of the first operator in the LSTM model 200 according to the precision coefficient of the first operator in the LSTM model 200 and the first quantized data of the first operator. Figure 4 Step S409 is not described in detail here.

[0173] S510: Determine the quantized data of the third operator in the LSTM model 200 based on the first quantized data of the second operator in the LSTM model 200 and the second quantized data of the first operator.

[0174] In some embodiments, the third operator in the LSTM model 200 is an associated operation between the first operator and the second operator. The first electronic device 10 can determine the quantized data of the third operator based on the first quantized data of the second operator and the second quantized data of the first operator in the LSTM model 200, wherein the quantization coefficient corresponding to the first quantized data of the second operator is the same as the quantization coefficient corresponding to the second quantized data of the first operator.

[0175] For example, in the operation process of the neural network layer 120-t of the LSTM model 200, the first operator may be the operator of formula (7) , the second operator can be the operator of formula (7) , the third operator can be the operator of formula (7) Specifically, the third operator The quantized data can be the first operator The second quantitative data and the second calculation The first quantized data is obtained by performing matrix addition calculation.

[0176] It is not difficult to see from the above description that in order to improve the quantization operation accuracy of the LSTM model 200, the first electronic device 10 can compare the quantization coefficient of the first operator in the LSTM model 200 with the quantization coefficient of the second operator, and use the larger quantization coefficient of the quantization coefficient of the first operator and the quantization coefficient of the second operator as the common quantization coefficient. According to the common quantization coefficient, the quantization data of the first operator or the second operator in the LSTM model 200 is re-determined, so that the quantization coefficient corresponding to the quantization data of the first operator is consistent with the quantization coefficient corresponding to the quantization data of the second operator (that is, both are common quantization coefficients), so that the first electronic device 10 can calculate the quantization data of the third operator more accurately based on the re-determined quantization data of the first operator and the second operator, thereby improving the quantization accuracy of the LSTM model 200.

[0177] The embodiments of the present application also provide a model quantization device for implementing the model quantization method in the aforementioned embodiments.

[0178] Specifically, Figure 6 According to some embodiments of the present application, a schematic structural diagram of a model quantization device 300 is shown.

[0179] like Figure 6 As shown, the model quantization device 300 includes:

[0180] The common quantization coefficient determination module 301 is configured to obtain a first quantization coefficient and first quantization data for a first operation (e.g., the operation corresponding to the aforementioned first operator), a second quantization coefficient and second quantization data for a second operation (e.g., the operation corresponding to the aforementioned second operator), and determine a common quantization coefficient based on the magnitude relationship between the first quantization coefficient and the second quantization coefficient. For example, in some embodiments, the common quantization coefficient determination module 301 may be configured to perform operations related to steps 401 to 404, step 408, steps 501 to 504, and step 508.

[0181] The data quantization module 302 is configured to re-quantize the first quantized data and / or the second quantized data based on the common quantization coefficient. For example, in some embodiments, the data quantization module 302 may be configured to perform operations related to the aforementioned steps 405, 409, 505, and 509.

[0182] The operation module 303 is configured to obtain a result of a third operation (e.g., an operation corresponding to the aforementioned third operator) based on data obtained by requantizing the first quantized data and / or the second quantized data, where the third operation is a correlation operation between the first operation and the second operation. For example, in some embodiments, the operation module 303 may be configured to perform operations related to steps 406, 410, 506, and 510.

[0183] Figure 7 FIG2 is a block diagram of a first electronic device 10 according to an embodiment of the present application. In one embodiment, the first electronic device 10 may include one or more processors 1004, a system control logic 1008 connected to at least one of the processors 1004, a system memory 1012 connected to the system control logic 1008, a non-volatile memory (NVM) 1016 connected to the system control logic 1008, and a network interface 1020 connected to the system control logic 1008.

[0184] In some embodiments, processor 1004 may include one or more single-core or multi-core processors. In some embodiments, processor 1004 may include any combination of general-purpose processors and specialized processors (e.g., graphics processors, application processors, baseband processors, etc.). In embodiments where the first electronic device 10 utilizes an eNB (Evolved Node B) or RAN (Radio Access Network) controller, processor 1004 may be configured to execute various embodiments.

[0185] In some embodiments, system control logic 1008 may include any suitable interface controller to provide any suitable interface to at least one of processors 1004 and / or any suitable device or component in communication with system control logic 1008 .

[0186] In some embodiments, the system control logic 1008 may include one or more memory controllers to provide an interface to the system memory 1012. The system memory 1012 may be used to load and store data and / or instructions. In some embodiments, the memory 1012 of the system 1000 may include any suitable volatile memory, such as a suitable dynamic random access memory (DRAM).

[0187] NVM 1016 may include one or more tangible, non-transitory computer-readable media for storing data and / or instructions. In some embodiments, NVM 1016 may include any suitable non-volatile memory such as flash memory and / or any suitable non-volatile storage device, such as at least one of an HDD (Hard Disk Drive), a CD (Compact Disc) drive, and a DVD (Digital Versatile Disc) drive.

[0188] The NVM 1016 may include a portion of storage resources on the device on which the first electronic device 10 is installed, or it may be accessible to the device but not necessarily part of the device. For example, the NVM 1016 may be accessed over a network via the network interface 1020 .

[0189] In particular, the system memory 1012 and the NVM 1016 may respectively include a temporary copy and a permanent copy of the instructions 1024. The instructions 1024 may include instructions that, when executed by at least one of the processors 1004, cause the first electronic device 10 to implement the following: Figure 7 In some embodiments, instructions 1024 , hardware, firmware, and / or software components thereof may additionally or alternatively reside in system control logic 1008 , network interface 1020 , and / or processor 1004 .

[0190] The network interface 1020 may include a transceiver for providing a radio interface for the first electronic device 10, thereby communicating with any other suitable devices (such as a front-end module, an antenna, etc.) via one or more networks. In some embodiments, the network interface 1020 may be integrated with other components of the first electronic device 10. For example, the network interface 1020 may be integrated with at least one of the processor 1004, the system memory 1012, the NVM 1016, and a firmware device (not shown) having instructions. When at least one of the processors 1004 executes the instructions, the first electronic device 10 implements the following. Figure 4 or Figure 5 The method shown.

[0191] The network interface 1020 may further include any suitable hardware and / or firmware to provide a multiple-input multiple-output radio interface. For example, the network interface 1020 may be a network adapter, a wireless network adapter, a telephone modem, and / or a wireless modem.

[0192] In one embodiment, at least one of the processors 1004 may be packaged together with logic for one or more controllers of the system control logic 1008 to form a system-in-package (SiP). In one embodiment, at least one of the processors 1004 may be integrated on the same die with logic for one or more controllers of the system control logic 1008 to form a system-on-chip (SoC).

[0193] The first electronic device 10 may further include an input / output (I / O) device 1032 .

[0194] According to the embodiments of the present application, Figure 8 FIG1 shows a block diagram of a SoC (System on Chip) 1100. The SoC 1100 is provided in the first electronic device 10. Figure 8 In FIG, similar components have the same reference numerals. In addition, the dashed boxes are optional features of more advanced SoCs. Figure 8 In the embodiment, the SoC 1100 includes an interconnect unit 1150 coupled to the application processor 1110 , a system agent unit 1170 , a bus controller unit 1180 , an integrated memory controller unit 1240 , and a direct memory access (DMA) unit 1160 .

[0195] It will be understood that, as used herein, the term "module" may refer to or include, or be part of, an application specific integrated circuit (ASIC), an electronic circuit, a processor (shared, dedicated, or group) and / or memory that executes one or more software or firmware programs, a combinational logic circuit, and / or other appropriate hardware components that provide the described functionality.

[0196] It is understood that in each embodiment of the present application, the processor can be a microprocessor, a digital signal processor, a microcontroller, etc., and / or any combination thereof. According to another aspect, the processor can be a single-core processor, a multi-core processor, etc., and / or any combination thereof.

[0197] The various embodiments of the mechanisms disclosed in this application can be implemented in hardware, software, firmware, or a combination of these implementation methods. The embodiments of this application can be implemented as a computer program or program code executed on a programmable system, which includes at least one processor, a storage system (including volatile and non-volatile memory and / or storage elements), at least one input device, and at least one output device.

[0198] Program code can be applied to input instructions to perform the functions described herein and generate output information. The output information can be applied to one or more output devices in a known manner. For the purposes of this application, a processing system includes any system having a processor, such as, for example, a digital signal processor (DSP), a microcontroller, an application-specific integrated circuit (ASIC), or a microprocessor.

[0199] Program code can be implemented with a high-level programming language or an object-oriented programming language to communicate with the processing system. Where necessary, program code can also be implemented in assembly language or machine language. In fact, the mechanism described in this application is not limited to the scope of any particular programming language. In either case, the language can be a compiled language or an interpreted language.

[0200] In some cases, the disclosed embodiments may be implemented in hardware, firmware, software, or any combination thereof. The disclosed embodiments may also be implemented as instructions carried by or stored on one or more transitory or non-transitory machine-readable (e.g., computer-readable) storage media, which can be read and executed by one or more processors. For example, the instructions may be distributed over a network or via other computer-readable media. Thus, a machine-readable medium may include any mechanism for storing or transmitting information in a form readable by a machine (e.g., a computer), including but not limited to floppy disks, optical disks, optical discs, read-only memories (CD-ROMs), magneto-optical disks, read-only memories (ROMs), random access memories (RAMs), erasable programmable read-only memories (EPROMs), electrically erasable programmable read-only memories (EEPROMs), magnetic or optical cards, flash memory, or tangible machine-readable storage for transmitting information via the Internet using electrical, optical, acoustical, or other propagated signals, such as carrier waves, infrared signals, digital signals, etc. Thus, a machine-readable medium includes any type of machine-readable medium suitable for storing or transmitting electronic instructions or information in a form readable by a machine (e.g., a computer).

[0201] In the accompanying drawings, some structural or method features may be shown in a particular arrangement and / or order. However, it should be understood that such a particular arrangement and / or order may not be required. Rather, in some embodiments, these features may be arranged in a manner and / or order different from that shown in the illustrative drawings. In addition, the inclusion of a structural or method feature in a particular figure does not imply that such feature is required in all embodiments, and in some embodiments, such features may not be included or may be combined with other features.

[0202] It should be noted that the units / modules mentioned in the various device embodiments of the present application are all logical units / modules. Physically, a logical unit / module can be a physical unit / module, or a part of a physical unit / module, or can be implemented as a combination of multiple physical units / modules. The physical implementation of these logical units / modules themselves is not the most important. The combination of functions implemented by these logical units / modules is the key to solving the technical problems raised by this application. In addition, in order to highlight the innovative part of this application, the above-mentioned device embodiments of this application do not introduce units / modules that are not closely related to solving the technical problems raised by this application. This does not mean that other units / modules do not exist in the above-mentioned device embodiments.

[0203] It should be noted that in the examples and description of this patent, relational terms such as first and second, etc. are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device that includes a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "including a" does not exclude the presence of other identical elements in the process, method, article or device that includes the element.

[0204] Although the present application has been shown and described with reference to certain preferred embodiments thereof, it will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the application.

Claims

1. A speech recognition model quantization method, applied to electronic equipment, characterized in that: The model is a GRU model or an LSTM model, and the model includes a first operation, a second operation, and a third operation, wherein the third operation is at least a correlation operation of the first operation and the second operation; And the method includes: Obtaining a first quantization coefficient of the first operation, first quantization data of the first operation, a second quantization coefficient of the second operation, and second quantization data of the second operation; Using the larger one of the first quantization coefficient and the second quantization coefficient as a common quantization coefficient; When the common quantization coefficient is the second quantization coefficient, quantizing the first floating-point data corresponding to the first quantized data of the first operation into third quantized data based on the second quantization coefficient, and obtaining a calculation result of the third operation according to the third quantized data, the second quantized data, and the second quantization coefficient; When the common quantization coefficient is the first quantization coefficient, the second floating-point data corresponding to the second quantized data of the second operation is quantized into fourth quantized data based on the first quantization coefficient, and the operation result of the third operation is obtained according to the first quantization data, the fourth quantization data, and the first quantization coefficient.

2. The method according to claim 1, characterized in that The step of quantizing the first floating-point data corresponding to the first quantized data of the first operation into third quantized data based on the second quantization coefficient includes: The first quantized data is dequantized into corresponding first floating-point data according to the first quantization coefficient, and the first floating-point data is quantized into the third quantized data according to the second quantization coefficient.

3. The method according to claim 1, characterized in that The step of quantizing the second floating-point data corresponding to the second quantized data of the second operation into fourth quantized data based on the first quantization coefficient includes: The second quantized data is dequantized into corresponding second floating-point data according to the second quantization coefficient, and the second floating-point data is quantized into the fourth quantized data according to the first quantization coefficient.

4. The method according to claim 1, wherein The third operation is an associated operation of the first operation and the second operation, including: The first quantized data is output data of the first operation, the second quantized data is output data of the second operation, and input data of the third operation includes the first quantized data and the second quantized data.

5. An electronic device, characterized in that: include: a memory for storing instructions to be executed by one or more processors of the electronic device, and The processor is one of the one or more processors of the electronic device, and is used to execute the instructions to enable the electronic device to implement the speech recognition model quantization method according to any one of claims 1 to 4.

6. A readable medium, characterized in that The readable medium stores instructions, which, when executed by an electronic device, enable the electronic device to implement the speech recognition model quantization method according to any one of claims 1 to 4.

7. A computer program product, characterized in that The computer program product includes instructions, and when the instructions are executed by an electronic device, the electronic device implements the speech recognition model quantization method according to any one of claims 1 to 4.

Images