Method for operating a neural network model, readable medium, and electronic device

Abstract

The present application relates to the field of artificial intelligence, and specifically to a method for operating a neural network model, a readable medium, and an electronic device. The method of the present application, when the electronic device detects that the type of the neural network model deployed thereon is different from the type corresponding to the preset operation logic, converts the first type of data applicable to the neural network model into a data type that can be processed by the preset operation logic, so that the electronic device can use the preset operation logic to process the data of the neural network model, and in the conversion process, by adjusting the scaling factor of the preset operation logic, eliminates the data errors existing in the conversion process, and ensures the accuracy of the data operation after the conversion.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence, and specifically to an operating method of a neural network model, a readable medium, and an electronic device. Background Art

[0002] With the rapid development of artificial intelligence (AI), neural network models are increasingly widely used in the field of artificial intelligence. Since fixed-point neural network models are small in size and have faster data operations, when deploying AI models on electronic devices, each operator of the neural network model is usually fixed-point quantized to obtain a fixed-point neural network model, which is then run by the central processing unit (CPU) or neural-network processing unit (NPU) of the electronic device. The quantization of each operator of the neural network model includes symmetric quantization and asymmetric quantization, and the corresponding CPU or NPU also needs to preset the operation logic of the symmetric quantized operator or the asymmetric quantized operator for the corresponding quantization model.

[0003] However, doing so will increase R&D costs on the one hand, and on the other hand may result in a symmetric quantized neural network model running on an electronic device with a preset asymmetric quantized operator's operation logic, or an asymmetric quantized neural network model running on an electronic device with a preset symmetric quantized operator's operation logic, resulting in errors in the calculation results.

[0004] Therefore, a neural network model operation method is urgently needed to solve the above problems. Summary of the invention

[0005] In view of this, the embodiments of the present application provide a method for operating a neural network model, a readable medium, and an electronic device. The method for operating a neural network model of the present application can reduce R&D costs on the one hand, and on the other hand, when the neural network model to be operated does not conform to the operation logic preset by the electronic device, the neural network model can also be successfully operated on the electronic device, thereby increasing the range of electronic devices to which the neural network model can be applied, while also ensuring the accuracy of the operation results.

[0006] The following is an introduction to the operation method of the neural network model of the present application.

[0007] In a first aspect, an embodiment of the present application provides a method for operating a neural network model, which is applied to an electronic device, and the method includes: detecting a neural network model to be operated, and determining that the type of the neural network model to be operated belongs to a first type of neural network model; determining that the first type of neural network model is different from a preset second type corresponding to a preset operation logic of the electronic device, using a first parameter of the first neural network model to convert a first type of data corresponding to the first type of neural network model into a second type of data, the second type of data corresponds to the preset second type corresponding to the preset operation logic of the electronic device, and using the preset operation logic of the electronic device to perform operation processing on the second type of data to realize the function of the first neural network model.

[0008] The first type of neural network model may be a symmetric quantized neural network model or an asymmetric quantized neural network model. Similarly, the preset operation logic may be a symmetric quantized logic or an asymmetric quantized logic.

[0009] When an electronic device detects that the type of the neural network model deployed on it is different from the type corresponding to the preset operation logic, it converts the first type of data applicable to the neural network model into a data type that can be processed by the preset operation logic (that is, the second type of data mentioned above), so that the electronic device can use the preset operation logic to process the data of the neural network model, thereby increasing the range of electronic devices that can be used with the neural network model and reducing research and development costs.

[0010] In combination with the first aspect, in a possible implementation of the first aspect, a neural network model to be run is detected, and it is determined that the type of the neural network model to be run belongs to a first type of neural network model, including: determining the data type of data in the neural network model to be run, and if the data type of the data in the neural network model to be run is a preset data type, determining that the neural network model to be run belongs to the first type of neural network model.

[0011] That is, by determining whether the data type in the neural network model to be run matches the preset data type, it is determined whether the type of the neural network to be run matches the preset second type corresponding to the preset operation logic of the electronic device. The data type in the neural network model to be run can refer to the data type of the neural network model to be run, and can also refer to the numerical type of the data in the neural network model to be run. The data type of the neural network model to be run includes but is not limited to zero point and scaling factor. The numerical type of the data of the neural network model to be run includes but is not limited to INT type data and UINT type data.

[0012] Among them, the preset data type is related to the first type of neural network model. If the first type of neural network model is an asymmetric quantized neural network model, then the preset data type is a UINT type data type and / or the preset data type includes zero points. If the data type of the neural network model to be run at this time is also UINT type data and / or includes zero points, it indicates that the neural network model to be run belongs to the first type of neural network model.

[0013] If the first type of neural network model is a symmetric quantized neural network model, then the preset data type is the INT data type. If the data type of the neural network model to be run is also INT data, it indicates that the neural network model to be run belongs to the first type of neural network model.

[0014] In combination with the first aspect and the possible implementation methods of the first aspect mentioned above, in a possible implementation method of the first aspect, the first type of data corresponding to the first type of neural network model is converted into second type data using the first parameter of the neural network model to be run, including: determining a third parameter based on the first parameter of the neural network model to be run and the second parameter of the preset operation logic, wherein the third parameter includes the second parameter and the first parameter, and obtaining the second type data based on the third parameter and the first type data.

[0015] It can be understood that since the type of the neural network model to be run may be a symmetric quantized neural network model or an asymmetric quantized neural network, correspondingly, the first parameter of the neural network model to be run will also change with the type of the neural network model to be run. In some implementations, when the neural network model to be run is an asymmetric quantized neural network model, the first parameter includes a first zero point, a first scaling factor such as ZP and ascale in formula (3) below. When the neural network model to be run is a symmetric quantized neural network model, the first parameter includes a first scaling factor such as scale in formula (2) below. Similarly, for the preset operation logic, if the preset operation logic is a symmetric quantized logic, the second parameter includes a second scaling factor, and if the preset operation logic is an asymmetric quantized logic, the second parameter includes a second scaling factor and a zero point, which will not be repeated here.

[0016] Therefore, in combination with the first aspect and the possible implementation methods of the first aspect mentioned above, in a possible implementation method of the first aspect, the first parameter includes at least a first scaling coefficient of the neural network model to be run, and the second parameter includes at least a second scaling coefficient of a preset operation logic. According to the first parameter of the neural network model to be run and the second parameter of the preset operation logic, determining the third parameter includes: adjusting the second scaling coefficient according to the first scaling coefficient, the adjusted second scaling coefficient and the first scaling coefficient being the reciprocal of each other; determining the third parameter according to the first scaling coefficient and the adjusted second scaling coefficient.

[0017] That is, since the error in the process of converting the first type of data into the second type of data is actually caused by the product of the first scaling factor and the second scaling factor (for example, as shown in formula (15) below), in some implementations, the error in the above data conversion process can be eliminated by adjusting the product of the first scaling factor and the second scaling factor to 1.

[0018] It can be understood that, due to the different neural network models to be run, the process of converting the first type of data into the second type of data is also different. The details are as follows:

[0019] In combination with the first aspect and the possible implementation of the first aspect, in a possible implementation of the first aspect, the second type of data is obtained according to the third parameter and the first type of data, including: when the neural network model to be run is an asymmetric quantized neural network model, the first parameter also includes the zero point of the neural network model to be run, and the floating-point data corresponding to the first type of data is determined according to the first scaling factor, the zero point and the first type of data; the second type of data is determined according to the floating-point data and the adjusted second scaling factor. For example, assuming that the neural network model to be run is an asymmetric quantized neural network model, as shown in the following formula (15), the first scaling factor is data_ascale, the second scaling factor is sym_scale, the zero point is data_ZP, the first type of data is Xq, and the floating-point data corresponding to the first type of data is float_data. That is, the above-mentioned first type of data conversion process is actually to convert the first type of data corresponding to the neural network model to be run into the corresponding floating-point data, and then convert the floating-point data into fixed-point data (that is, the second type of data) based on the adjusted second scaling factor of the preset operation logic, and finally call the preset operation logic to process the second type of data.

[0020] In combination with the first aspect and the possible implementation of the first aspect, in a possible implementation of the first aspect, the second type of data is obtained according to the third parameter and the first type of data, including: when the neural network model to be run is a symmetric quantized neural network model, the floating-point data corresponding to the first type of data is determined according to the first scaling factor and the first type of data; the second type of data is determined according to the floating-point data and the adjusted second scaling factor. From the following formulas (1) to (3), it can be seen that the difference between the symmetric quantized neural network model and the asymmetric quantized neural network model is that the symmetric quantized neural network model has no zero point, while the asymmetric quantized neural network model has a zero point. Based on this difference, when converting the first type of data into the corresponding floating-point data, the conversion process corresponding to the symmetric quantized neural network model naturally does not require the participation of the zero point, that is, according to the first scaling factor and the first type of data, the floating-point data corresponding to the first type of data can be obtained.

[0021] In combination with the first aspect and the possible implementation methods of the first aspect, in a possible implementation method of the first aspect, the neural network model to be run runs on a first processor of the electronic device, and the first processor includes a central processing unit CPU or an image processor NPU.

[0022] In combination with the first aspect and the possible implementation manner of the first aspect, in a possible implementation manner of the first aspect, the preset operation logic of the electronic device includes: asymmetric operation logic or symmetric operation logic.

[0023] In combination with the first aspect and the possible implementation methods of the first aspect, in a possible implementation method of the first aspect, the first type of data includes input data of the neural network model to be run.

[0024] In combination with the first aspect and the possible implementation of the first aspect, the function of the neural network model to be run includes an image classification function. That is, the function of the neural network model to be run can be as follows: Figure 5 or Figure 7 It is understood that in other implementations, the neural network model to be run may also have other functions, such as image recognition, audio recognition, etc., which are not limited in this application.

[0025] In a second aspect, an embodiment of the present application provides a neural network model operation device, the device comprising:

[0026] A detection module, a data conversion module and an operation module, wherein the detection module is used to detect the neural network model to be run and determine that the type of the neural network model to be run belongs to the first type of neural network model.

[0027] In some implementations, the detection module determines that the neural network model to be run belongs to the first type of neural network model when the data type of the data in the neural network model to be run is consistent with a preset data type.

[0028] A data conversion module is used to convert the first type of data corresponding to the first type of neural network model into the second type of data using the first parameter of the first neural network model when it is detected that the preset type corresponding to the preset operation logic of the electronic device is different from the preset type corresponding to the first type of neural network model. The second type of data corresponds to the preset type corresponding to the preset operation logic of the electronic device. In some implementations, the first type of data includes input data of the neural network model to be run. For example, assuming that the input data of the neural network model to be run is a picture, the first type of data is a pixel value matrix corresponding to the picture. The preset operation logic includes an asymmetric operation logic or a symmetric operation logic.

[0029] In some implementations, the data conversion module is used to determine a third parameter based on a first parameter of the neural network model to be run and a second parameter of a preset operation logic, wherein the third parameter includes the second parameter and the first parameter.

[0030] In some implementations, the first parameter includes at least a first scaling factor of the neural network model to be run, and the second parameter includes at least a second scaling factor of a preset operation logic. In some implementations, the data conversion module can adjust the second scaling factor according to the first scaling factor, and the adjusted second scaling factor is the reciprocal of the first scaling factor; and determine the third parameter according to the first scaling factor and the adjusted second scaling factor.

[0031] Since the types of neural network models to be run are different, in some implementations, the data conversion module needs to determine the method for converting the second type of data according to the type of the neural network model to be run.

[0032] Specifically, when the neural network model to be run is an asymmetric quantized neural network model, the first parameter also includes the zero point of the neural network model to be run. At this time, the data conversion module determines the floating-point data corresponding to the first type of data based on the first scaling factor, the zero point and the first type of data; and then determines the second type of data based on the floating-point data and the adjusted second scaling factor.

[0033] When the neural network model to be run is a symmetric quantized neural network model, the data conversion module determines the floating-point data corresponding to the first type of data based on the first scaling factor and the first type of data; and then determines the second type of data based on the floating-point data and the adjusted second scaling factor.

[0034] The operation module is used to control the electronic device, and uses its preset operation logic to perform operation processing on the second type of data to realize the function of the first neural network model. In some implementations, the function of the first neural network model includes an image classification function.

[0035] In some implementations, the electronic device includes a first processor, which may be a central processing unit or an image processor, and the neural network model to be executed runs on the central processing unit or the image processor of the electronic device.

[0036] In a third aspect, an embodiment of the present application provides a readable medium, which contains instructions. When the instructions are executed by a processor of an electronic device, the electronic device implements an operating method of any neural network model provided by the first aspect and various possible implementations of the first aspect.

[0037] In a fourth aspect, an embodiment of the present application provides an electronic device, comprising: a memory for storing instructions executed by one or more processors of the electronic device; and a processor, which is one of the processors of the electronic device, for executing instructions so that the electronic device implements the operating method of any one of the neural network models provided by the first aspect and various possible implementations of the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1A According to some embodiments of the present application, a schematic diagram of 8-bit symmetric quantization is shown;

[0039] Figure 1B According to some embodiments of the present application, a schematic diagram of 8-bit asymmetric quantization is shown;

[0040] Figure 2 According to some embodiments of the present application, a schematic diagram of a scenario in which a symmetric quantized neural network model is deployed in an electronic device 100 is shown;

[0041] Figure 3 According to some embodiments of the present application, a scene diagram of an electronic device 100 running a symmetric quantized neural network model is shown;

[0042] Figure 4 According to some embodiments of the present application, a schematic diagram of the structure of a neural network model 10 is shown;

[0043] Figure 5 According to some embodiments of the present application, a schematic diagram of a process in which an electronic device classifies an image 20 using an asymmetric quantized neural network model 10 is shown;

[0044] Figure 6According to some embodiments of the present application, a flowchart of a method for operating a neural network model is shown;

[0045] Figure 7 According to some embodiments of the present application, a schematic diagram of a process in which an electronic device 100 calls the operation logic of a symmetric quantization operator to classify an image 20 is shown;

[0046] Figure 8 According to some embodiments of the present application, a schematic diagram of the structure of an apparatus 800 corresponding to the method flow of the present application is shown;

[0047] Fig. 9 According to some embodiments of the present application, a schematic structural diagram of an electronic device 100 is shown. DETAILED DESCRIPTION

[0048] Illustrative embodiments of the present application include, but are not limited to, methods of operating a neural network model, readable media, and electronic devices.

[0049] To facilitate understanding, the terms involved in the embodiments of the present application are first introduced.

[0050] Quantization process: the process of converting each operator of the neural network model from floating-point numbers to fixed-point numbers.

[0051] The quantization process is divided into symmetric quantization process and asymmetric quantization process.

[0052] (1) Symmetric quantization

[0053] Symmetric quantization is to convert floating-point numbers into fixed-point numbers whose value range is symmetric with respect to "0", which is the corresponding quantization process when ZP is 0. Assume that the floating-point number to be quantized is X f , the quantitative goal is to convert X f If n-bit symmetric quantization is performed, the range of the fixed-point number after quantization is [-2 n-1 , 2 n-1 -1], and X f The quantized fixed-point number is X p , the symmetric quantization process can be expressed as the following formula (1).

[0054]

[0055] In formula (3), abs() indicates the absolute value, X q is a quantized fixed-point number, n represents the number of quantized bits, where Indicates the scaling factor scale for symmetric quantization.

[0056] Figure 1A According to some embodiments of the present application, a method for performing a floating point number X is shown.f Schematic diagram of 8-bit symmetric quantization. Figure 1A The quantitative goal is to convert X f Perform 8-bit symmetric quantization, that is, X f Convert to INT8 type (value range is [-128, 127]), assuming X f The maximum absolute value of abs(X f )=X fmax , then for X f The process of 8-bit symmetric quantization is to convert the interval [-X fmax , X fmax ] is mapped to the interval [-128, 127].

[0057] It is understood that in some embodiments, the fixed-point number can be other INT types, for example, it can be expressed as INTn, where n can be 4, 8, 16, 32, etc., or other integers, and the value range of the signed data of INTn type is [-2 n-1 , 2 n -1 -1].

[0058] (2) Asymmetric quantization

[0059] Asymmetric quantization is to convert floating point numbers into values ​​in the range [0, 2 n -1], where n is the number of bits of asymmetric quantization. Assume that the floating point number to be quantized is X f , the quantitative goal is to convert X f Perform n-bit asymmetric quantization, that is, the range of the fixed-point number after quantization is [0, 2 n -1], the asymmetric quantization process can be expressed as the following formula (3).

[0060]

[0061]

[0062] In formula (3), ZP represents the fixed-point number corresponding to 0 in the floating-point number after quantization, max() is the maximum value function, min() is the minimum value function, and X q is a quantized fixed-point number, n represents the number of quantized bits, where Represents the scaling factor ascale for asymmetric quantization.

[0063] Figure 1B According to some embodiments of the present application, a method for performing a floating point number X is shown. f Schematic diagram of 8-bit asymmetric quantization. Figure 1B The quantitative goal is to convert Xf Perform 8-bit asymmetric quantization, that is, X f Convert to UINT8 type (value range is [0, 255]), assuming X f The maximum value of max(X f ), the minimum value is min(X f ), then for X f The process of 8-bit asymmetric quantization is to convert the interval [min(X f ), max(X f )] is mapped to the interval [0, 255].

[0064] It can be understood that in some embodiments, the UINT type can be represented as UINTn, where n can be 4, 8, 16, 32, etc., or other integers, and the value range of the unsigned data of the UINTn type is [0, 2 n -1].

[0065] (3) Quantization of Neural Network Model

[0066] Neural network model quantization is the process of converting the input data, output data, and constant data in each operator of the neural network model from floating-point numbers of large data types (for example, 32-bit floating-point numbers) to fixed-point numbers of smaller data types (for example, 4 / 8 / 16-bit fixed-point numbers). The number of bits of the fixed-point number usually matches the number of bits of the fixed-point number supported by the computing unit that runs the neural network model, such as the NPU, to increase the speed at which the NPU runs the neural network model.

[0067] The quantization process of the neural network model is to determine the quantization parameters of the floating-point input data, floating-point output data and floating-point constant data of each operator according to the value range of the floating-point input data and the floating-point output data of each operator, and the type of the fixed-point number quantized (for example, UINT8), (for example, the quantization parameters of asymmetric quantization include asymmetric quantization scaling coefficients and asymmetric quantization zero points, and the quantization parameters of symmetric quantization include symmetric quantization scaling coefficients). That is, the quantization parameters of the quantized operator include the quantization parameters of the operator's input data, the quantization parameters of the output data, the quantized constant data and the quantization parameters of the constant data.

[0068] In the electronic device used to run the quantized neural network model, the operation logic of the quantized operator is preset. The operation logic of the quantized operator takes fixed-point numbers as input and outputs fixed-point numbers, and the parameters in the operation logic include the quantization parameters of the input data, the quantization parameters of the output data, the quantized constant data, and the quantization parameters of the constant data. When the electronic device runs a quantized operator, it calls the preset operation logic according to the quantization parameters of the operator, and the function of the operator can be realized through fixed-point operation.

[0069] Moreover, usually the operation logic preset in the electronic device is consistent with the quantization method adopted by the neural network model. For example, if the neural network model adopts a symmetric quantization method, the operation logic preset in the electronic device is also the corresponding symmetric quantization operation logic. If the neural network model adopts an asymmetric quantization method, the operation logic preset in the electronic device is also an asymmetric quantization operation logic, so as to improve the operating efficiency of the neural network model.

[0070] The following takes the convolution operator as an example to illustrate the operation logic of operators using symmetric quantization and asymmetric quantization.

[0071] Assuming that B_f is the floating-point input matrix of the convolution operator, C_f is the floating-point convolution kernel, and D_f is the floating-point convolution result, the relationship between B_f, C_f, and D_f can be expressed as formula (6).

[0072] D_f=B_f*C_f (6)

[0073] In formula (6), “*” is the convolution operation symbol.

[0074] According to formula (3), the operation logic of the asymmetric quantization operator can be obtained as follows:

[0075] B_f=(B_q-ZP_B) / B_ascale,

[0076] C_f=(C_q-ZP_C) / C_ascale,

[0077] D_f=(D_q-ZP_D) / D_ascale,

[0078] Among them, B_qp is the fixed-point number matrix of asymmetric quantization corresponding to B_f, B_ascale is the asymmetric quantization scaling coefficient for quantizing B_f to B_q, and ZP_B is the asymmetric quantization zero point for asymmetric quantization of B_f to B_q; C_q is the fixed-point number convolution kernel of asymmetric quantization corresponding to C_f, C_ascale is the asymmetric quantization scaling coefficient for quantizing C_f to C_q, and ZP_C is the asymmetric quantization zero point for asymmetric quantization of C_f to C_q; D_q is the fixed-point number matrix of asymmetric quantization corresponding to D_f, D_ascale is the symmetric quantization scaling coefficient for asymmetric quantization of D_f to D_q, and ZP_D is the asymmetric quantization zero point for asymmetric quantization of D_f to D_q.

[0079] Then formula (6) can be expressed as the following formula (7):

[0080] (D_q-ZP_D) / D_ascale=(B_q-ZP_B) / B_ascale*(C_q-ZP_C) / C_ascale (7)

[0082] By transforming formula (7), we can obtain the operation logic of the asymmetric quantized convolution operator as shown in the following formula (8).

[0083] D_q=(B_q-ZP_B) / B_ascale*(C_q-ZP_C) / C_ascale×D_ascale+ZP_D (8)

[0084] That is to say, the electronic device is preset with the operation logic shown in formula (8), the input of the operation logic includes asymmetric quantized fixed-point input data B_q, asymmetric quantization zero point ZP_B of the input data B_q, asymmetric quantization scaling coefficient B_ascale, asymmetric quantization convolution kernel C_q, asymmetric quantization zero point ZP_C of the convolution kernel C_q, asymmetric quantization scaling coefficient C_ascale, asymmetric quantization zero point ZP_D of the fixed-point output data D_q, asymmetric quantization scaling coefficient D_ascale, and the output data is the asymmetric quantization convolution result D_q.

[0085] Similarly, the operation logic of the symmetric quantized convolution operator can be expressed as the following formula (9), wherein the specific derivation process can refer to the operation logic of the symmetric quantized convolution operator, which will not be repeated here.

[0086] D_q=(B_q) / B_scale*(C_q) / C_scale×D_scale (9)

[0087] In formula (9), B_q is the symmetrically quantized input data, C_q is the symmetrically quantized convolution kernel, D_q is the fixed-point convolution result, B_scale is the symmetrical quantization scaling factor of B_q, D_scale is the symmetrical quantization parameter of the output data D_q, and C_scale is the symmetrical quantization scaling factor of the convolution kernel.

[0088] As mentioned above, in order to improve the operating efficiency of the neural network model, the NPU or CPU of the electronic device will generally have preset operation logic corresponding to the quantization method of the neural network model. For example, if the neural network model adopts a symmetric quantization method, the operation logic preset in the electronic device is also the corresponding symmetric quantization operation logic. If the neural network model adopts an asymmetric quantization method, the operation logic preset in the electronic device is also an asymmetric quantization operation logic. However, this will undoubtedly increase R&D costs.

[0089] Moreover, this research and development logic may result in a situation where the quantization method of the neural network model is inconsistent with the preset operation logic of the electronic device. For example, a symmetrically quantized neural network model runs on an electronic device with preset asymmetrically quantized operation logic, or an asymmetrically quantized neural network model runs on an electronic device with preset symmetrically quantized operation logic, which may lead to operation errors when the electronic device runs the neural network model.

[0090] For example, Figure 2 Taking the application scenario shown as an example, the NPU of the electronic device 100 can only run a symmetrically quantized neural network model, that is, the data required by the preset operation logic on the electronic device 100 are all second-type data, for example, the scaling factor sym_scale is a symmetrically quantized scaling factor, and the zero point sym_ZP is 0, while the neural network model to be run is an asymmetrically quantized neural network model, that is, the input data, quantization parameters, etc. of the neural network model to be run are all second-type data, for example, the scaling factor data_ascale of the neural network model to be run is an asymmetrically quantized scaling factor, and data_ZP is not equal to 0.

[0091] In order to solve the above problems, the present application provides a method for operating a neural network model. In this method, when the electronic device 100 detects that the quantization type of the deployed neural network model is different from the operation logic of the preset neural network model, the first type of data corresponding to the quantization type of the deployed neural network model is converted into the second type of data corresponding to the quantization type applicable to the operation logic preset by the electronic device 100. For example Figure 3 As shown, the electronic device 100 converts the first type of data of the deployed neural network model into the second type of data suitable for processing by the preset operation logic, and then processes the second type of data using the preset operation logic.

[0092] At the same time, since there are errors caused by data conversion in the above data conversion process, the method of the present application also eliminates the errors in the above data conversion process by adjusting the scaling factor of the preset operation logic to ensure the accuracy of the converted data operation.

[0093] Among them, the first type of data can be the fixed-point input data and output data of the neural network model, as well as the fixed-point weights and bias data involved in each layer (such as convolutional layer, network layer, fully connected layer) in the neural network model, etc., and this application does not impose any restrictions on this.

[0094] For example, taking the input data of the UNIT 8 type asymmetric quantized neural network model as the first type of data as an example, the process of converting the first type of data into the second type of data is introduced.

[0095] Xq_max represents the maximum value 255 of the fixed-point number corresponding to the input data of the asymmetric quantization model, and Xq_min represents the minimum value 0 of the fixed-point number corresponding to the input data of the asymmetric quantization model.

[0096] Since the floating-point data corresponding to the first type of data and the second type of data are consistent, both are floating-point data corresponding to the quantized fixed-point data, according to the above formula (5), the floating-point data of the input data corresponding to Xq_max and Xq_min can be deduced as follows:

[0097] float_data=(Xq_data_ZP)×data_ascale (10)

[0098] float_data_min=(Xq_min-ZP)×data_ascale (11)

[0099] float_data_max=(Xq_max-ZP)×data_ascale (12)

[0100] Among them, the minimum value of the floating-point data float_data before quantization is float_data_min, the maximum value is float_data_max, and data_ascale is the scaling factor of asymmetric quantization.

[0101] As mentioned above, the scaling factor of symmetric quantization in NPU is sym_scale, then:

[0102]

[0103] quantized_data=float_data×sym_scale (14)

[0104] Among them, n represents the number of quantized bits, float_data_min represents the minimum value of the floating-point data, float_data_max represents the maximum value of the floating-point data, quantized_data is a fixed-point number quantized by the preset symmetric quantization operation logic of the NPU, abs() represents the absolute value, and max represents the maximum value.

[0105] As mentioned above, since the neural network model is an asymmetric quantized neural network model, the first type of data of the neural network model is also data that conforms to the asymmetric quantized neural network model. Therefore, when the electronic device 100 processes the first type of data of the asymmetric neural network model, it is necessary to convert the first type of data into the second type of data suitable for the preset symmetric quantized operation logic processing. According to formula (14) and the above formula (10), the specific conversion process can be expressed as follows:

[0106] sym_data=float_data×sym_scale

[0107] =round((Xq-data_ZP)×data_ascale×sym_scale) (15)

[0108] Among them, Xq represents the first type of data of asymmetric quantization (fixed-point input data corresponding to the asymmetric quantization neural network model), data_ascale represents the scaling factor of asymmetric quantization, sym_scale represents the scaling factor of symmetric quantization preset on the electronic device 100, sym_data represents the second type of data obtained after the fixed-point input data of the asymmetric neural network model is converted (i.e., the converted fixed-point input data), and round represents the rounding operation.

[0109] Since in the above formula (15), only data_scale and sym_scale are floating-point values, and each sym_data is obtained after rounding, there is an error caused by rounding between the second type of data input to the electronic device 100 for calculation and the first type of data of the asymmetric quantization model, which in turn causes an error in the result of running the entire asymmetric quantization neural network model on the electronic device 100.

[0110] In some implementations, based on the above formula (15), the sym_scale of the electronic device 100 may be set to 1 / data_ascale, so that the above formula (15) becomes:

[0111] sym_data = X-data_ZP (16)

[0112] This eliminates the errors caused by the two floating-point numbers asym_scale and sym_scale.

[0113] It can be understood that in other implementations, if other formulas are used to derive

[0114]

[0115]

[0116] Then according to formula (15), we can get:

[0117] sym_data=float_data / sym_scale

[0118] =round((Xp-data_ZP) / data_ascale×sym_scale) (19)

[0119] At this time, the error caused by the two floating point numbers asym_scale and sym_scale can be eliminated by setting the sym_scale of the electronic device 100 to data_ascale, and the present application does not impose any limitation on this.

[0120] Then, the electronic device 100 is used to process the converted first type of data to realize the function of the asymmetric neural network model.

[0121] It can be understood that under UINT 8, the value range of X is [0, 255], and the value range of data_ZP is [0, 255]. Therefore, according to the above formula (16), the value range of sym_data is [-255, 255]. Obviously, the original 8-bit value range corresponding to UINT 8 is obviously unable to represent sym_data. Therefore, it is necessary to increase the bit position to expand its representable value range.

[0122] For example, the original 8 bits of UINT 8 can be increased by one bit to 9 bits, which can be called INT9. According to the relationship between the number of bits n and the range of values ​​that can be represented [-2 n-1 , 2 n -1], we can see that the value range that INT9 can represent is: [-256, 255], which can represent the above sym_data.

[0123] It can be understood that, since in a computer system, the data type is generally an integer multiple of 8, in some implementations of the present application, 16 bits may also be used to represent the above sym_data, and the present application does not impose any limitation on this.

[0124] In the above method, the electronic device 100 converts the first type of data suitable for the asymmetric quantization neural network model into the second type of data suitable for the symmetric quantization operation logic, and eliminates the error in the above data conversion process by adjusting the symmetric scaling coefficient in the NPU of the electronic device 100, so that the electronic device 100 can call the preset symmetric quantization operation logic to process the second type of data after the conversion to realize the relevant functions of the asymmetric quantization neural network model. There is no need to further develop the corresponding asymmetric quantization neural network model operation logic for the asymmetric quantization neural network model, which reduces the research and development cost and increases the type of neural network models that the electronic device 100 can run.

[0125] It can be understood that the operation logic of the preset symmetric quantization operator can be preset in the NPU by the developer of the NPU of the electronic device 100, or provided by the developer of the NPU to the developer of the electronic device 100, and preset in the memory of the electronic device 100 by the developer of the electronic device 100. This application does not impose any restrictions on this.

[0126] To further illustrate, assume that the input data of the convolution operator of an asymmetric quantized neural network model quantized to UINT8 (representing a numerical range of [0, 255]) is the first type of data matrix B_f, and the value range of each element is a floating point number [-1, 1], and the floating point convolution kernel Then we can get that the value range of each element in the floating point convolution result D_f is [-3, 3].

[0127] For B_f, according to formulas (3), (4), and (5), it can be obtained that the asymmetric quantization scaling coefficient of the input data B_ascale=0.0078 and the asymmetric quantization zero point of the input data B_ZP=127.

[0128] Similarly, we can also obtain the asymmetric quantization scaling coefficients C_ascale=0.023 and C_ZP=42 corresponding to the floating-point convolution kernel C_f, and the asymmetric quantization scaling coefficients D_ascale=0.023 and D_ZP=127 corresponding to the floating-point convolution result D_f.

[0129] At the same time, the first type of data B_q=[0, 255] corresponding to the above floating point type can be obtained according to formula (3), D_q=[0, 255].

[0130] Then, using the above formula (16), we can obtain the second type of data corresponding to the first type of data B_q matrix, the second type of data corresponding to the convolution kernel C_q, and the second type of data corresponding to the convolution result D_q, that is,

[0131] B_sym_data=B_q-B_ZP=[0, 255]-(127)=[-127, 128]

[0132] C_sym_data=C_q-C_ZP=[0, 255]-42=[-42, 213]

[0133] D_sym_data=D_q-D_ZP=[0, 255]-(125)=[-125, 130]

[0134] Assuming that the operation logic of symmetric quantization preset by the NPU of the electronic device 100 is as shown in the above formula (9), the sym_scale of the electronic device 100, that is, the above-mentioned B_scale, can be set to 1 / B_ascale, C_scale is set to 1 / C_ascale, and D_scale is set to 1 / D_ascale, that is, the following formula (20) is obtained:

[0135] D_q=(B_q×B_ascale)*(C_q×C_ascale) / D_ascale (20)

[0136] Then substitute the above B_sym_data, C_sym_data, and D_sym_data into formula (20) to obtain:

[0137] D_sym_data=(B_sym_data×B_ascale)*(C_sym_data×C_ascale) / D_ascale (twenty one)

[0139] Then D_q is equal to D_sym_data+ZP.

[0140] Through the above method, the first type of data suitable for the asymmetric quantized neural network model can be converted into the second type of data suitable for processing by the electronic device 100 with preset symmetric operation logic, thereby realizing the relevant functions of the asymmetric quantized neural network model.

[0141] The technical solution of the embodiment of the present application is introduced below in conjunction with a specific neural network model.

[0142] Figure 4 According to some embodiments of the present application, a schematic diagram of the structure of a neural network model 10 is shown. Figure 4As shown, the neural network model 10 is a symmetric quantized neural network model, including an input layer 11, a convolution operator 12, a fully connected operator 13, a Softmax operator 14 and an output layer 15, which is used to classify the input image data. Among them, the input layer 11 is used to preprocess the input image and convert the input image data into the first type of data suitable for the asymmetric quantized neural network model, such as UINT8 type data; the convolution operator 12 is used to perform convolution operation on the asymmetric quantized input data to obtain the asymmetric quantized feature matrix corresponding to the input image; the fully connected operator 13 is used to perform a fully connected operation on the asymmetric quantized feature matrix to obtain the score of the input image data belonging to each preset category; the Softmax operator 14 is used to obtain the probability that the input image data belongs to each preset category according to the score that the input image data belongs to each preset category; the output layer 15 is used to determine the category of the input image data according to the probability that the input image data belongs to each preset category, for example, the category with the highest probability that the input image data belongs to each preset category is used as the category of the input image data.

[0143] Figure 5 According to an embodiment of the present application, the electronic device 100 is shown as follows Figure 4 The illustrated diagram is a schematic diagram of the process of the neural network model 10 classifying the image 20.

[0144] like Figure 5 As shown, the electronic device 100 first uses the input layer 11 to preprocess the image 20 to obtain the UINT8 image matrix H, wherein the asymmetric quantization scaling coefficient of the input layer 11 is ascale_1=0.0498, and ZP_1=127.

[0145] Secondly, the image matrix H is respectively combined with the convolution kernel K using the convolution operator 12. i (i=1,2,3) perform convolution operation to obtain three feature matrices A i (i=1, 2, 3), the asymmetric quantization scaling coefficients of the input data of the convolution operator 12 are ascale_2=0.0078, ZP_2=127, the symmetric quantization scaling coefficients of the output data are ascale_3=0.0023, ZP_3=42, and the asymmetric quantization constant data includes the convolution kernel K i The asymmetric quantization scaling coefficients for (i=1, 2, 3) are ascale_4=0.023, ZP_4=127.

[0146] Then use the fully connected operator 13 to pair the feature matrix A i (i=1, 2, 3) to perform full connection operations, for example, the feature matrix A i(i=1, 2, 3) and the weight matrix W are inner-producted to obtain the score of the image 20 as a preset category (rabbit / dog / cat). The asymmetric quantization scaling factor of the input data of the fully connected operator 13 is ascale_4=0.0417, and the asymmetric quantization scaling factor of the output data is ascale_5=1.1×10 -5 , the asymmetric quantization constant data includes a weight matrix W (assuming that the asymmetric scaling coefficient of the weight matrix W is the same as the asymmetric scaling coefficient of the input data).

[0147] Then, the Softmax operator 14 is used to obtain the probability that the image 20 belongs to the preset category from the lookup table LUT according to the score of the image 20 belonging to the preset category.

[0148] Finally, the output layer 15 is used to determine the category of the image 20 according to the probability that the image 20 belongs to each preset category. For example, the probabilities that the image 20 belongs to each preset category are compared, and the highest probability among the probabilities that the image 20 belongs to the preset category is determined as the category of the image 20 (dog).

[0149] Combine the following Figure 4 The structure of the neural network model 10 shown and Figure 5 The process of classifying the image 20 by the neural network model 10 shown introduces the technical solution of the embodiment of the present application.

[0150] Figure 6 According to some embodiments of the present application, a flowchart of a method for operating a neural network model is shown. The execution subject of the method is an electronic device 100, such as Figure 6 As shown, the method includes:

[0151] 601: An asymmetric quantized neural network model is detected. In some embodiments, the electronic device 100 can determine whether the neural network is a symmetric quantized neural network model based on the data type of the data in the neural network model to be run. Specifically, when the electronic device 100 detects that the data type of the data in the neural network model to be run is UINT (UINT includes but is not limited to UINT4, UINT8, UINT16, UINT32, etc.), for example, when the electronic device 100 detects that the data type of the data in the neural network model to be run is UINT8, it determines that the neural network model to be run is an asymmetric quantized neural network model.

[0152] In some embodiments, the electronic device 100 may also determine whether the neural network model to be run is a symmetrically quantized neural network model based on the quantization parameters of each operator of the neural network model to be run. For example, the electronic device 100 may determine that the neural network model is a symmetrically quantized neural network model when it is detected that the quantization parameters of the neural network model to be run include a scaling factor and a zero point.

[0153] It can be understood that in other embodiments, the electronic device 100 can also determine whether the neural network model to be run is a symmetric quantized neural network model by other means, and when it is detected that the neural network model to be run is a symmetric quantized neural network model, it triggers the operation method of the neural network model provided in the embodiment of the present application. The present application does not limit this.

[0154] 602: Convert the first type of data corresponding to the asymmetric quantized neural network model into the second type of data suitable for processing by the symmetric quantized electronic device. The first type of data includes but is not limited to the input data corresponding to the asymmetric neural network model, the asymmetric quantization scaling coefficient and zero point corresponding to the asymmetric neural network model, and the asymmetric quantization scaling coefficient and zero point corresponding to the convolution kernel corresponding to each operator.

[0155] For example, Figure 4 Take the convolution operator 12 of the neural network model 10 as an example: the first type of data includes the asymmetric quantization scaling coefficients ascale_2=0.0078 and ZP_1=127 of the input data of the operator. Therefore, the electronic device 100 can use the above formula (16) to convert the first type of data of the convolution operator 12 to obtain the second type of data. For the specific conversion process, refer to the above description of the conversion of the first type of data B_q, the convolution kernel C_q, and the convolution result D_q into the second type of data.

[0156] Among them, the above conversion of the first type of data of the convolution operator into the second type of data is only an example. For the first type of data in other layers (such as input layer 11) or operators (including but not limited to pooling operators, activation operators, sorting operators, normalization operators, etc.), a similar method can be used to convert it into the second type of data, which will not be elaborated here.

[0157] 603: Call the preset operation logic of the symmetric quantization electronic device to process the second type of data to implement the function of the asymmetric quantization neural network model. That is, after the electronic device 100 converts the first type of data corresponding to the asymmetric neural network model into the second type of data, it can call the operation logic of the corresponding symmetric quantization operator through the NPU to implement the function of the symmetric quantization neural network model. For details, please refer to the process shown in the above formulas (20) to (21), which will not be repeated here.

[0158] Figure 7 FIG. 1 shows a process in which the electronic device 100 processes the second type of data after the conversion. Figure 7 As shown, after the electronic device 100 converts the asymmetric data of each operator of the neural network model 10 into symmetric data, each operator can be represented as Figure 7Shown are an input layer 11', a convolution operator 12', a fully connected operator 13', a Softmax operator 14', and an output layer 15'.

[0159] The NPU of the electronic device 100 can first adjust the preset symmetric quantization scaling coefficient based on the asymmetric quantization scaling coefficient of the asymmetric quantization neural network model, eliminate the data conversion error introduced by the two scaling coefficients in the above formula (15), and then quantize the image 20 into the image matrix H′ based on formula (16).

[0160] Secondly, the NPU of the electronic device 100 calls the operation logic of the symmetric quantized convolution operator, such as the operation logic shown in the above formula (9), when the convolution kernel is K i ′(i=1, 2, 3) (i.e. C_P), the image matrix H′ is respectively i 'Convolve to get the fixed-point feature matrix A i ′(i=1, 2, 3).

[0161] Then the NPU obtains the asymmetric quantized asymmetric input data B_q, the asymmetric quantization scaling coefficient B_ascale of the input data, the asymmetric quantization zero point B_ZP of the input data, the asymmetric quantization convolution kernel C_q, the asymmetric quantization scaling coefficient C_ascale of the convolution kernel, the asymmetric quantization zero point C_ZP of the convolution kernel, the asymmetric quantization scaling coefficient D_ascale of the output data, and the asymmetric quantization zero point D_ZP of the output data, and then obtains D_q according to the above formulas (20) and (21).

[0162] Then, the NPU of the electronic device 100 calls the operation logic of the symmetric quantized fully connected operator to convert the feature matrix A i ′ is fully connected with the weight matrix W′, for example, an inner product is performed to obtain the scores in1′, in2′ and in3′ of the image 20 belonging to each preset category; then the operation logic of the symmetric quantized Softmax operator is called, that is, [in1′, in2′, in3′] is used as the lookup table index to find the probability of the image 20 belonging to each preset category from the lookup table LUT′; finally, the operation logic of the symmetric quantized output layer is called, that is, the preset category corresponding to the maximum probability among the probabilities that the image 20 belongs to each preset category is taken as the category of the image 20, for example, the category of the image 20 is determined to be dog.

[0163] It can be understood that steps 601 to 603 in the above method can be completed entirely by the CPU of the electronic device 100 , or entirely by the NPU of the electronic device 100 , or by the CPU and NPU of the electronic device 100 working together, which is not limited here.

[0164] Through the method provided in the embodiment of the present application, the electronic device 100 converts the first type of data suitable for the asymmetric quantization neural network model into the second type of data suitable for the symmetric quantization operation logic, and eliminates the error in the above-mentioned data conversion process by adjusting the symmetric scaling coefficient in the NPU of the electronic device 100, so that the electronic device 100 can call the preset symmetric quantization operation logic to process the second type of data after the above-mentioned conversion to realize the relevant functions of the asymmetric quantization neural network model. There is no need to further develop the corresponding asymmetric quantization neural network model operation logic for the asymmetric quantization neural network model, thereby reducing the research and development cost and increasing the type of neural network models that the electronic device 100 can run.

[0165] Figure 8 A schematic structural diagram of an apparatus 800 corresponding to the flowchart of the method of the present application is shown.

[0166] like Figure 8 As shown, the device includes a detection module 801, a data conversion module 802 and a calculation module 803.

[0167] Among them, the detection module 801 is used to detect the neural network model to be run, and determine that the type of the neural network model to be run belongs to the first type of neural network model.

[0168] In some implementations, the detection module 801 determines that the neural network model to be run belongs to the first type of neural network model when the data type of the data in the neural network model to be run is consistent with a preset data type.

[0169] The data conversion module 802 is used to convert the first type of data corresponding to the first type of neural network model into the second type of data using the first parameter of the first neural network model when it is detected that the preset type corresponding to the preset operation logic of the electronic device is different from the first type of neural network model. The second type of data corresponds to the preset type corresponding to the preset operation logic of the electronic device. In some implementations, the first type of data includes input data of the neural network model to be run. For example, assuming that the input data of the neural network model to be run is a picture, the first type of data is the pixel value matrix corresponding to the picture. The preset operation logic includes asymmetric operation logic or symmetric operation logic.

[0170] In some implementations, the data conversion module 802 is used to determine a third parameter based on the first parameter of the neural network model to be run and the second parameter of the preset operation logic, wherein the third parameter includes the second parameter and the first parameter.

[0171] In some implementations, the first parameter includes at least a first scaling factor of the neural network model to be run, and the second parameter includes at least a second scaling factor of the preset operation logic. In some implementations, the data conversion module 802 can adjust the second scaling factor according to the first scaling factor, and the adjusted second scaling factor is the reciprocal of the first scaling factor; and determine the third parameter according to the first scaling factor and the adjusted second scaling factor.

[0172] Since the types of neural network models to be run are different, in some implementations, the data conversion module 802 needs to determine the method for converting the second type of data according to the type of the neural network model to be run.

[0173] Specifically, when the neural network model to be run is an asymmetric quantized neural network model, the first parameter also includes the zero point of the neural network model to be run. At this time, the data conversion module 802 determines the floating-point data corresponding to the first type of data based on the first scaling factor, the zero point and the first type of data; and then determines the second type of data based on the floating-point data and the adjusted second scaling factor.

[0174] When the neural network model to be run is a symmetric quantized neural network model, the data conversion module 802 determines the floating-point data corresponding to the first type of data based on the first scaling factor and the first type of data; and then determines the second type of data based on the floating-point data and the adjusted second scaling factor.

[0175] The operation module 803 is used to control the electronic device to perform operation processing on the second type of data using its preset operation logic to implement the function of the first neural network model. In some implementations, the function of the first neural network model includes an image classification function.

[0176] In some implementations, the electronic device includes a first processor, which may be a central processing unit or an image processor, and the neural network model to be run runs on the central processing unit or the image processor of the electronic device.

[0177] Fig. 9 According to some embodiments of the present application, a schematic diagram of the structure of an electronic device 100 is shown. Fig. 9As shown, the electronic device 100 includes one or more processors 101A, a system memory 102, a non-volatile memory (NVM) 103, a communication interface 104, an input / output (I / O) device 105, and a system control logic 106 for coupling the processor 101A, the system memory 102, the non-volatile memory 103, the communication interface 104 and the input / output (I / O) device 105. Among them:

[0178] The processor 101A may include one or more processing units, for example, a central processing unit CPU (Central Processing Unit), an image processor GPU (Graphics Processing Unit), a digital signal processor DSP (Digital Signal Processor), a microprocessor MCU (Micro-programmed Control Unit), an AI (Artificial Intelligence) processor or a programmable logic device FPGA (Field Programmable Gate Array) The processing module or processing circuit may include one or more single-core or multi-core processors.

[0179] The neural network processor 101B can be used to call the operation logic of the preset asymmetric quantized operator to realize the reasoning of the neural network model. The neural network processor 101B can be an independent processor or integrated into the processor 101A. In some embodiments, the NPU can be used to run the instructions corresponding to the operation method of the neural network model provided in the embodiment of the present application.

[0180] The system memory 102 is a volatile memory, such as a random-access memory (RAM), a double data rate synchronous dynamic random access memory (DDR SDRAM), etc. The system memory is used to temporarily store data and / or instructions. For example, in some embodiments, the system memory 102 can be used to store the relevant instructions of the neural network model 10, asymmetric / symmetric quantization parameters, asymmetric / symmetric quantization constant data, etc., and can also be used to store the operation logic of a preset asymmetric quantization operator.

[0181] The non-volatile memory 103 may include one or more tangible, non-temporary computer-readable media for storing data and / or instructions. In some embodiments, the non-volatile memory 103 may include any suitable non-volatile memory such as flash memory and / or any suitable non-volatile storage device, such as a hard disk drive (HDD), a compact disc (CD), a digital versatile disc (DVD), a solid-state drive (SSD), etc. In some embodiments, the non-volatile memory 103 may also be a removable storage medium, such as a secure digital (SD) memory card, etc. In other embodiments, the non-volatile memory 103 may be used to store the relevant instructions, asymmetric / symmetric quantization parameters, asymmetric / symmetric quantization constant data, etc. of the above-mentioned neural network model 10, and may also be used to store the operation logic of a preset asymmetric quantization operator.

[0182] In particular, the system memory 102 and the non-volatile memory 103 may respectively include: a temporary copy and a permanent copy of the instruction 107. The instruction 107 may include: when executed by at least one of the processor 101A and / or the neural network processor 101B, the electronic device 100 implements the operation method of the neural network model provided in various embodiments of the present application.

[0183] The communication interface 104 may include a transceiver for providing a wired or wireless communication interface for the electronic device 100, and then communicating with any other suitable device through one or more networks. In some embodiments, the communication interface 104 may be integrated into other components of the electronic device 100, for example, the communication interface 104 may be integrated into the processor 101A. In some embodiments, the electronic device 100 may communicate with other devices through the communication interface 104, for example, the electronic device 100 may obtain a neural network model to be run from other electronic devices through the communication interface 104.

[0184] The input / output (I / O) device 105 may include input devices such as a keyboard, a mouse, etc., and output devices such as a display, etc. A user may interact with the electronic device 100 via the input / output (I / O) device 105 .

[0185] System control logic 106 may include any suitable interface controller to provide any suitable interface with other modules of electronic device 100. For example, in some embodiments, system control logic 106 may include one or more memory controllers to provide interfaces to system memory 102 and non-volatile memory 103.

[0186] In some embodiments, at least one of the processors 101A may be packaged together with the logic of one or more controllers for the system control logic 106 to form a system in package (SiP). In other embodiments, at least one of the processors 101A may also be integrated with the logic of one or more controllers for the system control logic 106 on the same chip to form a system-on-chip (SoC).

[0187] It can be understood that the electronic device 100 can be any electronic device that can run a neural network model, including but not limited to mobile phones, wearable devices (such as smart watches, etc.), tablet computers, desktops, laptops, handheld computers, notebook computers, ultra-mobile personal computers (ultra-mobile personal computers, UMPC), netbooks, as well as cellular phones, personal digital assistants (personal digital assistants, PDA), augmented reality (augmented reality, AR) / virtual reality (virtual reality, VR) devices, etc., and the embodiments of the present application are not limited thereto.

[0188] Understandably, Fig. 9 The structure of the electronic device 100 shown is only an example. In other embodiments, the electronic device 100 may include more or fewer components than shown, or combine some components, or separate some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0189] The various embodiments of the mechanism disclosed in the present application can be implemented in hardware, software, firmware or a combination of these implementation methods. The embodiments of the present application can be implemented as a computer program or program code executed on a programmable system, which includes at least one processor, a storage system (including volatile and non-volatile memory and / or storage elements), at least one input device and at least one output device.

[0190] Program code can be applied to input instructions to perform the functions described in this application and generate output information. The output information can be applied to one or more output devices in a known manner. For the purposes of this application, a processing system includes any system having a processor such as, for example, a digital signal processor (DSP), a microcontroller, an application specific integrated circuit (ASIC), or a microprocessor.

[0191] Program code can be implemented with high-level programming language or object-oriented programming language to communicate with the processing system. When necessary, program code can also be implemented with assembly language or machine language. In fact, the mechanism described in this application is not limited to the scope of any specific programming language. In either case, the language can be a compiled language or an interpreted language.

[0192] In some cases, the disclosed embodiments may be implemented in hardware, firmware, software, or any combination thereof. The disclosed embodiments may also be implemented as instructions carried or stored on one or more temporary or non-temporary machine-readable (e.g., computer-readable) storage media, which may be read and executed by one or more processors. For example, instructions may be distributed over a network or through other computer-readable media. Therefore, machine-readable media may include any mechanism for storing or transmitting information in a machine (e.g., computer) readable form, including but not limited to, floppy disks, optical disks, optical disks, read-only memories (CD-ROMs), magneto-optical disks, read-only memories (ROM), random access memories (RAM), erasable programmable read-only memories (EPROM), electrically erasable programmable read-only memories (EEPROM), magnetic cards or optical cards, flash memory, or tangible machine-readable memories for transmitting information (e.g., carrier waves, infrared signal digital signals, etc.) using the Internet in electrical, optical, acoustic, or other forms of propagation signals. Accordingly, machine-readable media include any type of machine-readable media suitable for storing or transmitting electronic instructions or information in a form readable by a machine (eg, a computer).

[0193] In the accompanying drawings, some structural or method features may be shown in a specific arrangement and / or order. However, it should be understood that such a specific arrangement and / or order may not be required. Instead, in some embodiments, these features may be arranged in a manner and / or order different from that shown in the illustrative drawings. In addition, the inclusion of structural or method features in a particular figure does not mean that such features are required in all embodiments, and in some embodiments, these features may not be included or may be combined with other features.

[0194] It should be noted that the units / modules mentioned in the various device embodiments of the present application are all logical units / modules. Physically, a logical unit / module can be a physical unit / module, or a part of a physical unit / module, or can be implemented as a combination of multiple physical units / modules. The physical implementation method of these logical units / modules themselves is not the most important. The combination of functions implemented by these logical units / modules is the key to solving the technical problems proposed by the present application. In addition, in order to highlight the innovative part of the present application, the above-mentioned device embodiments of the present application do not introduce units / modules that are not closely related to solving the technical problems proposed by the present application, which does not mean that there are no other units / modules in the above-mentioned device embodiments.

[0195] It should be noted that in the examples and description of this patent, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of more restrictions, an element defined by the sentence "comprises one" does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0196] Although the present application has been illustrated and described with reference to certain preferred embodiments thereof, it will be apparent to those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the present application.

Claims

1. A method for operating a neural network model, applied to an electronic device, characterized in that: include: Detecting a neural network model to be run, and determining that the type of the neural network model to be run belongs to a first type of neural network model; determining that the first type of neural network model is different from a preset second type corresponding to a preset operation logic of the electronic device, converting a first type of data corresponding to the first type of neural network model into a second type of data using a first parameter of the first type of neural network model, the second type of data corresponding to the preset second type corresponding to the preset operation logic of the electronic device, and Using the preset operation logic of the electronic device to perform operation processing on the second type of data to realize the function of the first type of neural network model; The function of the neural network model to be run includes an image classification function; The converting the first type of data corresponding to the first type of neural network model into the second type of data by using the first parameter of the neural network model to be run includes: Determine a third parameter according to the first parameter of the neural network model to be run and the second parameter of the preset operation logic, wherein the third parameter includes the second parameter and the first parameter, Obtaining the second type of data according to the third parameter and the first type of data; The first parameter includes at least a first scaling factor of the neural network model to be run, the second parameter includes at least a second scaling factor of the preset operation logic, and the determining the third parameter according to the first parameter of the neural network model to be run and the second parameter of the preset operation logic includes: According to the first scaling factor, adjusting the second scaling factor, wherein the adjusted second scaling factor and the first scaling factor are reciprocals of each other; The third parameter is determined according to the first scaling factor and the adjusted second scaling factor.

2. The method according to claim 1, characterized in that The detecting the neural network model to be run and determining that the type of the neural network model to be run belongs to the first type of neural network model includes: Determine the data type of the data in the neural network model to be run, If the data type of the data in the neural network model to be run is a preset data type, it is determined that the neural network model to be run belongs to a first type of neural network model.

3. The method according to claim 2, characterized in that The obtaining the second type of data according to the third parameter and the first type of data includes: When the neural network model to be run is an asymmetric quantized neural network model, the first parameter also includes a zero point of the neural network model to be run, and the floating-point data corresponding to the first type of data is determined according to the first scaling factor, the zero point and the first type of data; The second type of data is determined according to the floating point data and the adjusted second scaling factor.

4. The method according to claim 1, characterized in that The obtaining the second type of data according to the third parameter and the first type of data includes: When the neural network model to be run is a symmetric quantized neural network model, determining floating-point data corresponding to the first type of data according to the first scaling factor and the first type of data; The second type of data is determined according to the floating point data and the adjusted second scaling factor.

5. The method according to any one of claims 1 to 4, characterized in that The neural network model to be run runs on a first processor of the electronic device, and the first processor includes a central processing unit CPU or a graphics processor NPU.

6. The method according to any one of claims 1 to 4, characterized in that The preset operation logic of the electronic device includes: asymmetric operation logic or symmetric operation logic.

7. The method according to any one of claims 1 to 4, characterized in that The first type of data includes input data of the neural network model to be run.

8. A readable medium, characterized in that The readable medium contains instructions, which, when executed by a processor of an electronic device, enable the electronic device to implement any one of the methods of claims 1 to 7.

9. An electronic device, characterized in that: include: a memory for storing instructions to be executed by one or more processors of the electronic device; and a processor, which is one of the processors of the electronic device, configured to execute the instructions so that the electronic device implements the method according to any one of claims 1 to 7.

Images