Full integer type neural network model inference method and device, equipment and storage medium

Abstract

The application discloses a full-integer neural network model reasoning method and device, equipment and a storage medium, which are applied to an embedded device supporting integer calculation, relate to the field of neural networks, and include the following steps: quantizing a trained neural network model deployed in the embedded device by using a preset method to obtain weight coefficients of the neural network model; classifying operators based on a preset rule to obtain several types of operators; performing corresponding calculation on the several types of operators to obtain intermediate data corresponding to the various types of operators; the data types of the intermediate data are all target integer data; and performing full-integer neural network model calculation reasoning based on the intermediate data and the weight coefficients of the neural network model. In this way, the neural network model can be deployed on the embedded device supporting only hardware calculation.

Description

Technical Field

[0001] The present invention relates to the field of neural networks, and in particular to a full-integer neural network model inference method, device, equipment and storage medium. Background Art

[0002] With the continuous advancement of deep learning technologies, neural network models have been widely applied in many industries and scenarios. Due to the large number of parameters and computational complexity of neural network models, deployment on resource-constrained embedded devices is challenging, often requiring lightweight methods such as model quantization. Furthermore, some embedded devices only support integer data computation. Existing model inference methods use quantized integer data for computationally intensive operations such as matrix multiplication, while many other operations often require floating-point computation. For example, the Add operator in a common residual network often uses floating-point computation because adding two int8 values ​​can result in data overflow. Furthermore, when adding scalar values, quantized int8 values ​​cannot be directly added to the scalar value due to scaling, so floating-point computation is often required. Therefore, deploying neural network models on embedded devices that only support hardware computation has become a pressing issue. Summary of the Invention

[0003] In view of this, the purpose of the present invention is to provide a full-integer neural network model inference method, apparatus, device, and storage medium. Through this method, a neural network model can be deployed on embedded devices that only support hardware computing. The specific scheme is as follows:

[0004] In a first aspect, the present application discloses a full integer neural network model inference method, which is applied to an embedded device that supports integer computing, including:

[0005] quantizing the trained neural network model deployed in the embedded device using a preset method to obtain a weight coefficient of the neural network model;

[0006] Classify operators based on preset rules to obtain several types of operators;

[0007] Perform corresponding calculations on the several types of operators to obtain intermediate data corresponding to the various types of operators; the data type of each intermediate data is target integer data;

[0008] The computational reasoning of the full-integer neural network model is performed based on the intermediate data and the weight coefficients of the neural network model.

[0009] Optionally, the quantizing the trained neural network model deployed in the embedded device by using a preset method includes:

[0010] The trained neural network model deployed in the embedded device is quantized using the int data type.

[0011] Optionally, classifying operators based on preset rules includes:

[0012] The operators are categorized based on how each type of operator is calculated.

[0013] Optionally, the operators are classified based on preset rules to obtain several types of operators, including:

[0014] Based on the calculation method of each type of operator, the operators are divided into a first type of operator that performs quantization and inverse quantization calculations, a second type of operator that is calculated by a shift method, and a third type of operator that is calculated by a table lookup method.

[0015] Optionally, performing corresponding calculations on the several types of operators to obtain intermediate data corresponding to the various types of operators includes:

[0016] When performing calculations on any first-type operator, if the current input is the output of another first-type operator, the corresponding convolution calculation or matrix multiplication calculation is directly performed by the any first-type operator to obtain the intermediate data corresponding to the any first-type operator; if the current input is the output of the second-type operator or the third-type operator, a quantization operation is performed on the any first-type operator, and then a corresponding convolution calculation or matrix multiplication calculation is performed to obtain the intermediate data corresponding to the any first-type operator; if the current output is the input of another first-type operator, an inverse quantization operation is performed on the any first-type operator and the data component in the preset intermediate representation constant is multiplied to obtain the intermediate data corresponding to the any first-type operator;

[0017] When calculating the second type operator, performing corresponding calculations based on different shift operations to obtain the intermediate data of the second type operator;

[0018] When calculating the third type operator, the intermediate data of the third type operator is obtained based on the hash table.

[0019] Optionally, the intermediate representation constant is a constant constructed based on the data component and the shift component, wherein the shift component represents the number of bits to be right-shifted during integer calculations, and the data component is a value obtained with base 2 and the shift component as the exponent.

[0020] Optionally, the process of performing corresponding calculations on the several types of operators to obtain intermediate data corresponding to the various types of operators further includes:

[0021] A pre-created data overflow detection module is used to detect whether overflow occurs when the intermediate data is currently calculated. If overflow occurs, the shift component is reduced.

[0022] In a second aspect, the present application discloses a full integer neural network model inference device, which is applied to an embedded device supporting integer calculations, comprising:

[0023] A weight coefficient acquisition module, configured to quantize the trained neural network model deployed in the embedded device using a preset method to obtain a weight coefficient of the neural network model;

[0024] An operator acquisition module is used to classify operators based on preset rules to obtain several types of operators;

[0025] An intermediate data acquisition module is used to perform corresponding calculations on the several types of operators to obtain intermediate data corresponding to the various types of operators; the data type of each intermediate data is target integer data;

[0026] A model reasoning module is used to perform computational reasoning of a full-integer neural network model based on the intermediate data and the weight coefficients of the neural network model.

[0027] In a third aspect, the present application discloses an electronic device, comprising:

[0028] Memory, used to store computer programs;

[0029] A processor is used to execute the computer program to implement the aforementioned full integer neural network model inference method.

[0030] In a fourth aspect, the present application discloses a computer-readable storage medium for storing a computer program, which, when executed by a processor, implements the aforementioned full-integer neural network model inference method.

[0031] As can be seen from the above, when inferring a full-integer neural network model, the present application first uses a preset method to quantize the trained neural network model deployed in the embedded device to obtain the weight coefficient of the neural network model; classifies the operators based on preset rules to obtain several types of operators; performs corresponding calculations on the several types of operators to obtain intermediate data corresponding to the various types of operators; the data type of each intermediate data is target integer data; and performs computational reasoning of the full-integer neural network model based on the intermediate data and the weight coefficient of the neural network model. It can be seen that by providing a full-integer neural network model reasoning method, the present application can deploy a neural network model on an embedded device that only supports hardware calculations, and is applicable to common neural network models such as convolution, and has high practical value and innovative value. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are merely embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0033] Figure 1 This application discloses a flow chart of a full-integer neural network model inference method;

[0034] Figure 2 A specific flow chart of the full-integer neural network model inference method is disclosed for this application;

[0035] Figure 3 This application discloses a schematic diagram of the structure of a full-integer neural network model inference device;

[0036] Figure 4 This is a structural diagram of an electronic device disclosed in this application. DETAILED DESCRIPTION

[0037] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0038] In existing model inference methods, operators with large computational complexity, such as matrix multiplication, are calculated using quantized integer data, and for more operators, floating-point calculations are often required. For example, the Add operator in the common residual network often uses floating-point calculations because adding two int8 data may cause data overflow. In addition, for the addition of scalar data, since the quantized int8 data has been scaled, it cannot be directly added to the scalar data, and floating-point calculations are often required. To solve the above problems, the present application discloses a full-integer neural network model inference method that can deploy neural network models on embedded devices that only support hardware calculations.

[0039] See also Figure 1 As shown, an embodiment of the present invention discloses a full integer neural network model inference method, which is applied to an embedded device supporting integer calculations, including:

[0040] Step S11: quantize the trained neural network model deployed in the embedded device using a preset method to obtain a weight coefficient of the neural network model.

[0041] In this embodiment, the trained neural network model deployed in the embedded device is first quantized using the int data type. Specifically, a suitable method is used to quantize the trained model. For ease of description, it is assumed that the model is quantized using the int8 data type. However, quantization in this patent is not limited to the int8 data type. The weight coefficients of the neural network model are obtained.

[0042] Step S12: Classify the operators based on preset rules to obtain several types of operators.

[0043] In this embodiment, operators are classified based on their calculation methods. The operators are divided into a first type of operator that performs quantization and inverse quantization calculations, a second type of operator that performs calculations using a shifting method, and a third type of operator that performs calculations using a table lookup method. Specifically, operators are divided into the following three categories based on their corresponding calculation methods: operators such as convolution and matrix multiplication that can perform quantization and inverse quantization calculations are represented by OPS_A; operators such as element-wise, relu, relu6, clip, pooling, hswish, hsigmoid, and scalar calculations that are calculated using a shifting method are represented by OPS_B; and operators such as sigmoid, tanh, sin, and cos that are calculated using a table lookup method are represented by OPS_C. This results in several types of operators.

[0044] Step S13: performing corresponding calculations on the several types of operators to obtain intermediate data corresponding to the various types of operators; the data type of each intermediate data is target integer data.

[0045] In this embodiment, based on the above embodiments, it can be seen that several types of operators are obtained, among which, when calculating on any first-type operator, if the current input is the output of another first-type operator, then the corresponding convolution calculation or matrix multiplication calculation is directly performed by the any first-type operator to obtain the intermediate data corresponding to the any first-type operator; if the current input is the output of the second-type operator or the third-type operator, then the any first-type operator is quantized, and then the corresponding convolution calculation or matrix multiplication calculation is performed to obtain the intermediate data corresponding to the any first-type operator; if the current output is the input of another first-type operator, then the any first-type operator is inversely quantized and multiplied by the data component in the preset intermediate representation constant to obtain the intermediate data corresponding to the any first-type operator; when calculating on the second-type operator, the corresponding calculation is performed based on different shift operations to obtain the intermediate data of the second-type operator; when calculating on the third-type operator, the intermediate data of the third-type operator is obtained based on the hash table. In this way, the intermediate data corresponding to various types of operators are obtained; wherein, the data type of each intermediate data is the target integer data. It should be pointed out that the intermediate representation constant is a constant constructed based on the data component and the shift component, wherein the shift component represents the number of bits to be right-shifted during integer calculations, and the data component is a value obtained with 2 as the base and the shift component as the exponent. That is, it is necessary to set the intermediate representation constant Alpha, wherein Alpha is composed of Alpha_data and Alpha_shift, and the specific relationship is as follows: Alpha_data = Pow(2, Alpha_shift)

[0046] Alpha is used to multiply floating-point data by Alpha_data and convert it into int32 integer data. In subsequent calculations, the Alpha_shift bit is shifted right accordingly to meet the data transformation requirements. Generally, Alpha_shift can be set to an integer between 10 and 15. And in the process of performing corresponding calculations on the several types of operators to obtain the intermediate data corresponding to the various types of operators, it also includes: using a pre-created data overflow detection module to detect whether overflow occurs when calculating the intermediate data. If overflow occurs, the shift component is reduced. That is, the corresponding data overflow detection module is designed during the reasoning process. If an overflow of the int32 data type occurs, a smaller Alpha_shift needs to be selected.

[0047] Step S14: performing computational inference of a full-integer neural network model based on the intermediate data and the weight coefficients of the neural network model.

[0048] In this embodiment, the weight coefficients of the neural network model obtained through quantization and the intermediate data are used to calculate each layer of the neural network. Similarly, computational reasoning of a full integer neural network model can be performed. It should be noted that this application primarily addresses the problem of deploying neural network models on embedded devices that only support integer calculations. In other words, it primarily addresses the problem of embedded devices that only support integer calculations.

[0049] As can be seen from the above, when inferring a full-integer neural network model, the present application first uses a preset method to quantize the trained neural network model deployed in the embedded device to obtain the weight coefficient of the neural network model; classifies the operators based on preset rules to obtain several types of operators; performs corresponding calculations on the several types of operators to obtain intermediate data corresponding to the various types of operators; the data type of each intermediate data is target integer data; and performs computational reasoning of the full-integer neural network model based on the intermediate data and the weight coefficient of the neural network model. It can be seen that by providing a full-integer neural network model reasoning method, the present application can deploy a neural network model on an embedded device that only supports hardware calculations, and is applicable to common neural network models such as convolution, and has high practical value and innovative value.

[0050] Based on the above embodiments, it can be seen that the present application can perform corresponding calculations on the various types of operators to obtain intermediate data corresponding to the various types of operators. Next, the process of obtaining the intermediate data corresponding to the various types of operators will be described in detail. Figure 2 As shown, the embodiment of the present invention discloses a specific full-integer neural network model inference method, which is applied to an embedded device supporting integer calculations, including:

[0051] Step S21: When performing calculation on any first-type operator, if the current input is the output of another first-type operator, the corresponding convolution calculation or matrix multiplication calculation is directly performed by the any first-type operator to obtain the intermediate data corresponding to the any first-type operator; if the current input is the output of a second-type operator or a third-type operator, a quantization operation is performed on the any first-type operator, and then a corresponding convolution calculation or matrix multiplication calculation is performed to obtain the intermediate data corresponding to the any first-type operator; if the current output is the input of another first-type operator, an inverse quantization operation is performed on the any first-type operator and multiplied by the data component in the preset intermediate representation constant to obtain the intermediate data corresponding to the any first-type operator.

[0052] In this embodiment, it can be seen from the previous embodiment that the present application classifies the operators to obtain a first type of operator (which can be represented by OPS_A) for quantization and inverse quantization calculations, a second type of operator (which can be represented by OPS_B) calculated by shifting, and a third type of operator (which can be represented by OPS_C) calculated by table lookup. For the OPS_A type operator. If the input is the output of another OPS_A operator, that is, the int8 data type, the corresponding convolution or matrix multiplication calculation can be performed directly to obtain an intermediate result of the int32 data type. If the input is the output of an OPS_B or OPS_C type operator, quantization is required, and the quantization is performed simultaneously with the right shift of the Alpha_shift bit to obtain the quantized int8 data, and then the corresponding convolution or matrix multiplication calculation is performed to obtain an intermediate result of the int32 data type. In addition, if the output of the operator is the input of another OPS_A operator, the above calculation result is simultaneously subjected to the inverse quantization of the operator and the quantization of the next OPS_A operator to obtain an output of the int8 data type. Otherwise, the above calculation result is simultaneously subjected to inverse quantization of the operator and multiplied by Alpha_data to obtain int32 intermediate data, that is, integer intermediate data.

[0053] Step S22: When calculating the second type operator, perform corresponding calculations based on different shift operations to obtain intermediate data of the second type operator.

[0054] In this embodiment, the input of the OPS_B type operator is generally int32 data obtained by multiplying by Alpha_data. For element-wise operations, int32 data is used for calculation and right-shifted by the number of Alpha_shift bits according to the specific operator type. For example, the dot multiplication operator needs to be right-shifted by one Alpha_shift bit, while the dot addition operator does not need to be right-shifted. For the relu, relu6, and clip operators, the corresponding clip threshold needs to be left-shifted by the number of Alpha_shift bits before the clip calculation is performed. For the pooling operator, max_pooling can be calculated directly. The division in average_pooling is converted to multiplication by a decimal, which is then left-shifted by a certain number of bits, shift_pool, and converted to int32 data. Then, the pooling calculation is performed, and the calculation needs to be right-shifted by the number of shift_pool bits. For scalar operators, the scalar needs to be left-shifted by the number of Alpha_shift bits before the calculation is performed. For the hswish and hsigmoid operators, they are actually composed of scalar operators and are calculated using the scalar operator calculation method. In the above way, the output of all OPS_B type operators is still the int32 data type multiplied by Alpha_data, which preserves the data precision.

[0055] Step S23: When calculating the third type operator, obtain the intermediate data of the third type operator based on the hash table.

[0056] In this embodiment, the input to an OPS_C operator is typically int32 data obtained by multiplying it by Alpha_data. First, a hash table is created, where the key is the int32 data and the value is also the int32 data obtained by multiplying it by Alpha_data. Therefore, when constructing this hash table, the key is right-shifted by the number of bits in the Alpha_shift column and the value is multiplied by Alpha_data.

[0057] As can be seen from the above, this application provides a full-integer neural network model inference method. First, the trained model is quantized. Second, the operator types are divided into three types according to the calculation method. For each operator, a different integer calculation method is used. Through the above method, the neural network model can be deployed on embedded devices that only support hardware calculations. It is suitable for common neural network models such as convolution, and has high practical value and innovative value.

[0058] See also Figure 3 As shown, an embodiment of the present invention discloses a full integer neural network model inference device, which is applied to an embedded device supporting integer calculations, including:

[0059] A weight coefficient acquisition module 11 is used to quantize the trained neural network model deployed in the embedded device using a preset method to obtain the weight coefficient of the neural network model;

[0060] An operator acquisition module 12 is used to classify operators based on preset rules to obtain several types of operators;

[0061] The intermediate data acquisition module 13 is used to perform corresponding calculations on the several types of operators to obtain intermediate data corresponding to the various types of operators; the data type of each intermediate data is target integer data;

[0062] The model inference module 14 is used to perform computational inference of the full-integer neural network model based on the intermediate data and the weight coefficients of the neural network model.

[0063] As can be seen from the above, when inferring a full-integer neural network model, the present application first uses a preset method to quantize the trained neural network model deployed in the embedded device to obtain the weight coefficient of the neural network model; classifies the operators based on preset rules to obtain several types of operators; performs corresponding calculations on the several types of operators to obtain intermediate data corresponding to the various types of operators; the data type of each intermediate data is target integer data; and performs computational reasoning of the full-integer neural network model based on the intermediate data and the weight coefficient of the neural network model. It can be seen that by providing a full-integer neural network model reasoning method, the present application can deploy a neural network model on an embedded device that only supports hardware calculations, and is applicable to common neural network models such as convolution, and has high practical value and innovative value.

[0064] In some specific embodiments, the weight coefficient acquisition module 11 may specifically include:

[0065] A quantization unit is used to quantize the trained neural network model deployed in the embedded device using an int data type.

[0066] In some specific embodiments, the operator acquisition module 12 may specifically include:

[0067] The classification unit is configured to classify the operators based on the calculation method of each type of operator.

[0068] In some specific embodiments, the operator acquisition module 12 may specifically include:

[0069] The operator acquisition unit is used to classify the operators into a first type of operators for quantization and inverse quantization calculations, a second type of operators for calculations by shifting, and a third type of operators for calculations by table lookup based on the calculation method of each type of operators.

[0070] In some specific embodiments, the intermediate data acquisition module 13 may specifically include:

[0071] a first intermediate data acquisition unit, configured to, when performing calculations on any first-type operator, directly perform a corresponding convolution operation or matrix multiplication operation on any first-type operator to obtain the intermediate data corresponding to any first-type operator if the current input is the output of another first-type operator; perform a quantization operation on any first-type operator and then perform a corresponding convolution operation or matrix multiplication operation on any first-type operator to obtain the intermediate data corresponding to any first-type operator if the current input is the output of any second-type operator or any third-type operator; and perform an inverse quantization operation on any first-type operator and multiply the data component of the data by a preset intermediate representation constant to obtain the intermediate data corresponding to any first-type operator if the current output is the input of another first-type operator.

[0072] a first intermediate data acquisition unit, configured to, when performing calculations on the second-type operator, perform corresponding calculations based on different shift operations to obtain the intermediate data of the second-type operator;

[0073] The first intermediate data acquisition unit is configured to acquire the intermediate data of the third type operator based on a hash table when performing calculations on the third type operator.

[0074] In some specific embodiments, the intermediate data acquisition module 13 may further include:

[0075] The detection unit is used to use a pre-created data overflow detection module to detect whether overflow occurs when the intermediate data is currently calculated, and if overflow occurs, reduce the shift component.

[0076] Furthermore, the embodiment of the present application also discloses an electronic device, Figure 4 This is a structural diagram of an electronic device 20 according to an exemplary embodiment. The content in the diagram should not be considered as any limitation to the scope of application of the present application.

[0077] Figure 4This is a schematic diagram of the structure of an electronic device 20 provided in an embodiment of the present application. The electronic device 20 may specifically include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. The memory 22 is used to store a computer program, which is loaded and executed by the processor 21 to implement the relevant steps in the full-integer neural network model inference method disclosed in any of the aforementioned embodiments. In addition, the electronic device 20 in this embodiment may specifically be an electronic computer.

[0078] In this embodiment, the power supply 23 is used to provide operating voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and the external device. The communication protocol it follows is any communication protocol that can be applied to the technical solution of this application and is not specifically limited here; the input and output interface 25 is used to obtain external input data or output data to the outside world. Its specific interface type can be selected according to specific application needs and is not specifically limited here.

[0079] In addition, the memory 22, as a carrier for resource storage, can be a read-only memory, random access memory, disk or CD, etc. The resources stored thereon can include an operating system 221, a computer program 222, etc., and the storage method can be temporary storage or permanent storage.

[0080] The operating system 221 is used to manage and control the hardware devices and computer program 222 on the electronic device 20, and can be Windows Server, Netware, Unix, Linux, etc. In addition to including a computer program that can be used to implement the full-integer neural network model inference method performed by the electronic device 20 disclosed in any of the aforementioned embodiments, the computer program 222 can further include a computer program that can be used to perform other specific tasks.

[0081] Furthermore, this application also discloses a computer-readable storage medium for storing a computer program; wherein, when executed by a processor, the computer program implements the aforementioned disclosed full-integer neural network model inference method. The specific steps of this method can be referred to the corresponding content disclosed in the aforementioned embodiments and will not be repeated here.

[0082] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from the other embodiments. Reference can be made to the descriptions of the identical or similar parts between the various embodiments. For the devices disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the descriptions are relatively simple, and the relevant parts can be referred to the descriptions of the methods.

[0083] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the above description has generally described the components and steps of each example according to their functions. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0084] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein may be implemented directly using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), internal memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0085] Finally, it should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the process, method, article, or device comprising the element.

[0086] The above is a detailed introduction to the technical solution provided by the present application. Specific examples are used herein to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea. At the same time, for those skilled in the art, according to the ideas of the present application, there may be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A full-integer neural network model inference method, characterized in that: Applicable to embedded devices that support integer calculations, including: quantizing the trained neural network model deployed in the embedded device using a preset method to obtain a weight coefficient of the neural network model; Based on the calculation method of each type of operator, the operators are divided into a first type of operator that performs quantization and inverse quantization calculations, a second type of operator that is calculated by a shift method, and a third type of operator that is calculated by a table lookup method; Perform corresponding calculations on the several types of operators to obtain intermediate data corresponding to the various types of operators; the data type of each intermediate data is target integer data; Performing computational reasoning of a full-integer neural network model based on the intermediate data and the weight coefficients of the neural network model; The corresponding calculations are performed on the several types of operators to obtain intermediate data corresponding to the various types of operators, including: When performing calculations on any first-type operator, if the current input is the output of another first-type operator, the corresponding convolution calculation or matrix multiplication calculation is directly performed by the any first-type operator to obtain the intermediate data corresponding to the any first-type operator; if the current input is the output of the second-type operator or the third-type operator, a quantization operation is performed on the any first-type operator, and then a corresponding convolution calculation or matrix multiplication calculation is performed to obtain the intermediate data corresponding to the any first-type operator; if the current output is the input of another first-type operator, an inverse quantization operation is performed on the any first-type operator and the data component in the preset intermediate representation constant is multiplied to obtain the intermediate data corresponding to the any first-type operator; When calculating the second type operator, performing corresponding calculations based on different shift operations to obtain the intermediate data of the second type operator; When calculating the third type operator, obtaining the intermediate data of the third type operator based on the hash table; The intermediate representation constant is a constant constructed based on the data component and the shift component, wherein the shift component represents the number of bits to be right-shifted in integer calculations, and the data component is a value obtained by using 2 as the base and the shift component as the exponent; The process of performing corresponding calculations on the several types of operators to obtain intermediate data corresponding to the various types of operators further includes: A pre-created data overflow detection module is used to detect whether overflow occurs when the intermediate data is currently calculated. If overflow occurs, the shift component is reduced.

2. The full-integer neural network model inference method according to claim 1, characterized in that: The method of quantizing the trained neural network model deployed in the embedded device by using a preset method includes: The trained neural network model deployed in the embedded device is quantized using the int data type.

3. A full-integer neural network model inference device, characterized in that: Applicable to embedded devices that support integer calculations, including: A weight coefficient acquisition module, configured to quantize the trained neural network model deployed in the embedded device using a preset method to obtain a weight coefficient of the neural network model; An operator acquisition module, configured to classify the operators into a first type of operator for performing quantization and inverse quantization calculations, a second type of operator for performing calculations by shifting, and a third type of operator for performing calculations by table lookup based on the calculation method of each type of operator; An intermediate data acquisition module is used to perform corresponding calculations on the several types of operators to obtain intermediate data corresponding to the various types of operators; the data type of each intermediate data is target integer data; A model reasoning module, configured to perform computational reasoning of a full-integer neural network model based on the intermediate data and the weight coefficients of the neural network model; The intermediate data acquisition module is used to, when performing calculations on any first-type operator, if the current input is the output of another first-type operator, then the corresponding convolution calculation or matrix multiplication calculation is directly performed by the any first-type operator to obtain the intermediate data corresponding to the any first-type operator; if the current input is the output of the second-type operator or the third-type operator, then the any first-type operator is quantized, and then the corresponding convolution calculation or matrix multiplication calculation is performed to obtain the intermediate data corresponding to the any first-type operator; if the current output is the input of another first-type operator, then the any first-type operator is inverse quantized. operation and multiplication by the data component in the preset intermediate representation constant to obtain the intermediate data corresponding to any one of the first type operators; when calculating the second type operator, performing corresponding calculations based on different shift operations to obtain the intermediate data of the second type operator; when calculating the third type operator, obtaining the intermediate data of the third type operator based on the hash table; the intermediate representation constant is a constant constructed based on the data component and the shift component, wherein the shift component represents the number of bits to be right-shifted in integer calculations, and the data component is a value obtained with 2 as the base and the shift component as the exponent; The intermediate data acquisition module is further configured to use a pre-created data overflow detection module to detect whether an overflow occurs when the intermediate data is currently being calculated, and if an overflow occurs, to reduce the shift component.

4. An electronic device, characterized in that: include: Memory, used to store computer programs; A processor, configured to execute the computer program to implement the full integer neural network model inference method as claimed in claim 1 or 2.

5. A computer-readable storage medium, characterized in that Used to store a computer program, which, when executed by a processor, implements the full integer neural network model inference method as described in claim 1 or 2.

Images