Quantization method, device and equipment of neural network model

Abstract

The application discloses a quantification method of a neural network model, comprising the following steps: obtaining a first operation branch and a second operation branch contained in a target operation module of a neural network model to be quantified; wherein the number of operation layers of the first operation branch is greater than the number of operation layers of the second operation branch; quantifying the first operation branch into a first precision; quantifying the second operation branch into a second precision; and the first precision is less than the second precision. The above method is used to solve the problems of low efficiency and insufficient optimization of the mixed strategy acquisition when the neural network model is quantified in the prior art.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a method and device for quantizing a neural network model. The present application also relates to an electronic device and a storage medium. Background Art

[0002] The demand for inference of neural network models on public clouds is increasing, and there is a need to optimize the performance of inference tasks. Currently, new GPUs (graphics processing units) on public clouds support INT8 (8-bit signed integer) operations, and the nominal computing power of INT8 is twice that of FP16 (half-precision floating-point numbers). To increase computing speed, it is often necessary to quantize trained FP16-precision neural network models to INT8-precision models. However, for some tasks with high precision requirements, the PTQ (Post-Training Quantization) technical solution is used. Directly quantizing the neural network model to INT8-precision models will result in unacceptable accuracy loss.

[0003] Under existing technology, for neural network models with customer acceleration requirements, a hybrid quantization solution is used to further improve the performance of neural network model inference while ensuring accuracy and maximizing the computing power of the target hardware. When quantizing a neural network model to INT8 precision, if the model accuracy does not meet the requirements, the sensitivity of individual computing layers to quantization is determined. Layers that are sensitive to quantization are retained at high precision, while layers that are insensitive to quantization are quantized to INT8.

[0004] The above solution has some drawbacks: First, for a large-scale model, measuring the sensitivity of each computation layer on a dataset takes a long time, and compiling the mixed-precision model takes a long time. Second, two computation layers that are highly sensitive to quantization do not necessarily need to be kept in high precision. In some cases, keeping only one of the computation layers in high precision can achieve the desired accuracy.

[0005] In summary, the existing technology has the problem of low efficiency and insufficient optimization in obtaining hybrid strategies when performing hybrid quantization on neural network models. Summary of the Invention

[0006] The present application provides a quantization method, device, and electronic device for a neural network model to solve the problems of low efficiency and insufficient optimization of hybrid strategy acquisition when performing hybrid quantization on a neural network model in the prior art.

[0007] This application provides a quantization method for a neural network model, including:

[0008] Obtaining a first operation branch and a second operation branch included in a target operation module of a neural network model to be quantized; wherein the number of operation layers of the first operation branch is greater than the number of operation layers of the second operation branch;

[0009] quantizing the first operation branch to a first precision;

[0010] The second operation branch is quantized to a second precision; the first precision is smaller than the second precision.

[0011] As an implementation manner, the first operation branch is a first operation branch including multi-layer operations; the second operation branch is a short-circuit branch.

[0012] As an implementation manner, the first precision is eight-bit integer precision; the second precision is half-precision floating point precision.

[0013] As an embodiment, the target operation module for obtaining the neural network model to be quantized includes a first operation branch and a second operation branch, including:

[0014] Determine the target operation module of the neural network model to be quantized;

[0015] According to the target operation module, a first operation branch and a second operation branch included in the target operation module of the neural network model to be quantized are obtained.

[0016] As an implementation manner, the target operation module for determining the neural network model to be quantized includes:

[0017] Traversing the neural network model to be quantized;

[0018] Determine whether the output of the current operation passes through two operation branches. If so, use the output of the current operation as the starting position of the target operation module;

[0019] Determine the merge position of the target operation module;

[0020] The operation between the starting position and the merging position of the target operation module is used as the target operation module.

[0021] As an implementation manner, determining the merge position of the target operation module includes:

[0022] Obtain the preset merge position identifier;

[0023] The merging position of the target operation module is determined according to the merging position identifier.

[0024] As an implementation method, it further includes:

[0025] The operation modules other than the target operation module in the neural network model are quantized to a second precision.

[0026] The present application also provides a quantization device for a neural network model, comprising:

[0027] A branch obtaining unit, configured to obtain a first operation branch and a second operation branch included in a target operation module of a neural network model to be quantized; wherein the number of operation layers of the first operation branch is greater than the number of operation layers of the second operation branch;

[0028] A first operation branch quantization unit, configured to quantize the first operation branch to a first precision;

[0029] The second operation branch quantization unit is used to quantize the second operation branch to a second precision; the first precision is smaller than the second precision.

[0030] As an implementation manner, the first operation branch is a first operation branch including multi-layer operations; the second operation branch is a short-circuit branch.

[0031] As an implementation manner, the first precision is eight-bit integer precision; the second precision is half-precision floating point precision.

[0032] As an implementation manner, the branch obtaining unit is specifically configured to:

[0033] Determine the target operation module of the neural network model to be quantized;

[0034] According to the target operation module, a first operation branch and a second operation branch included in the target operation module of the neural network model to be quantized are obtained.

[0035] As an implementation manner, the branch obtaining unit is specifically configured to:

[0036] Traversing the neural network model to be quantized;

[0037] Determine whether the output of the current operation passes through two operation branches. If so, use the output of the current operation as the starting position of the target operation module;

[0038] Determine the merging position of the target operation module; and use the operation between the starting position and the merging position of the target operation module as the target operation module.

[0039] As an implementation manner, the branch obtaining unit is specifically configured to:

[0040] Obtain the preset merge position identifier;

[0041] The merging position of the target operation module is determined according to the merging position identifier.

[0042] As an embodiment, the device further includes:

[0043] The remaining operation module quantization unit is used to quantize the operation modules other than the target operation module to the second precision.

[0044] The present application also provides an electronic device, comprising:

[0045] processor; and

[0046] The memory is used to store a program of a quantization method for a neural network model. After the device is powered on and the program of the quantization method for the neural network model is run by the processor, the following steps are performed:

[0047] Obtaining a first operation branch and a second operation branch included in a target operation module of a neural network model to be quantized; wherein the number of operation layers of the first operation branch is greater than the number of operation layers of the second operation branch;

[0048] quantizing the first operation branch to a first precision;

[0049] The second operation branch is quantized to a second precision; the first precision is smaller than the second precision.

[0050] As an implementation manner, the first operation branch is a first operation branch including multi-layer operations; the second operation branch is a short-circuit branch.

[0051] As an implementation manner, the first precision is eight-bit integer precision; the second precision is half-precision floating point precision.

[0052] As an embodiment, the target operation module for obtaining the neural network model to be quantized includes a first operation branch and a second operation branch, including:

[0053] Determine the target operation module of the neural network model to be quantized;

[0054] According to the target operation module, a first operation branch and a second operation branch included in the target operation module of the neural network model to be quantized are obtained.

[0055] As an implementation manner, the target operation module for determining the neural network model to be quantized includes:

[0056] Traversing the neural network model to be quantized;

[0057] Determine whether the output of the current operation passes through two operation branches. If so, use the output of the current operation as the starting position of the target operation module;

[0058] determining a merging position of the target operation module; and taking an operation operation between a starting position and the merging position of the target operation module as the target operation module.

[0059] As an implementation form, the determining the merging position of the target operation module comprises:

[0060] obtaining a preset merging position identifier;

[0061] determining the merging position of the target operation module according to the merging position identifier.

[0062] As an implementation form, the electronic device further performs the following steps:

[0063] quantizing operation modules other than the target operation module to a second precision.

[0064] The application further provides a storage medium storing a program of a data processing method, which is run by a processor to perform the following steps:

[0065] obtaining a first operation branch and a second operation branch contained in a target operation module of a neural network model to be quantized; wherein the number of operation layers of the first operation branch is greater than the number of operation layers of the second operation branch;

[0066] quantizing the first operation branch to a first precision;

[0067] quantizing the second operation branch to a second precision; and the first precision is less than the second precision.

[0068] The application provides a quantization method of a neural network model, comprising: obtaining a first operation branch and a second operation branch contained in a target operation module of a neural network model to be quantized; wherein the number of operation layers of the first operation branch is greater than the number of operation layers of the second operation branch; quantizing the first operation branch to a first precision; and quantizing the second operation branch to a second precision; and the first precision is less than the second precision. The quantization method of the neural network model provided by the application quantizes the first branch with a greater number of operation layers in the target operation module of the neural network model to be quantized to a low precision, and directly quantizes the second branch with a smaller number of operation layers to a high precision, without the need to measure the quantization sensitivity of each operation layer, so that the efficiency of obtaining a mixed strategy when performing mixed quantization on the neural network model is higher. In addition, when the first operation branch has an operation layer sensitive to quantization, the operation layer does not need to be quantized to a high precision, and only the operation layer in the second operation branch can be kept to a high precision, so that the purpose of ensuring the precision is achieved by keeping only one branch of the operation layer to a high precision. The application solves the problems of low efficiency and insufficient optimization of obtaining a mixed strategy when performing mixed quantization on the neural network model. BRIEF DESCRIPTION OF DRAWINGS

[0069] Figure 1A This is an application scenario diagram of a quantization method for a neural network model provided in this application.

[0070] Figure 1 This is a flowchart of a quantization method for a neural network model provided in the first embodiment of the present application.

[0071] Figure 2 This is a schematic diagram of a target operation module operation branch provided in the first embodiment of the present application.

[0072] Figure 3 It is a schematic diagram of a target operation module provided in the first embodiment of the present application.

[0073] Figure 4 This is a schematic diagram of a quantization device for a neural network model provided in the second embodiment of the present application. DETAILED DESCRIPTION

[0074] The following description sets forth numerous specific details to facilitate a thorough understanding of the present invention. However, the present invention can be implemented in many other ways than those described herein, and those skilled in the art can make similar generalizations without departing from the scope of the present invention. Therefore, the present invention is not limited to the specific implementations disclosed below.

[0075] In order to present the present application more clearly, the application scenario of the quantization method of the neural network model provided in the first embodiment of the present application is briefly introduced.

[0076] The quantization method of the neural network model provided in the first embodiment of the present application can be applied to scenarios where the client and the server interact, such as Figure 1A When a quantized neural network model needs to be obtained, the client usually first establishes a connection with the server. After the connection is established, the client sends the neural network model to be quantized to the server. After the server obtains the neural network model to be quantized, it first obtains the first operation branch and the second operation branch included in the target operation module of the neural network model to be quantized in the branch obtaining unit 101; wherein the number of operation layers of the first operation branch is greater than the number of operation layers of the second operation branch; then, in the first operation branch quantization unit 102, the first operation branch is quantized to the first precision; then, in the second operation branch quantization unit 103, the second operation branch is quantized to the first precision.

[0077] The first embodiment of the present application provides a quantization method for a neural network model, which is combined with Figure 1 Make an introduction.

[0078] like Figure 1As shown, in step S101, a first operation branch and a second operation branch included in a target operation module of a neural network model to be quantized are obtained; wherein the number of operation layers of the first operation branch is greater than the number of operation layers of the second operation branch.

[0079] The neural network model to be quantized refers to a neural network model that has been trained with training data. When the existing technology uses PTQ (Post-Training Quantization) to quantize the neural network model to be quantized, directly quantizing the neural network model to an INT8 precision model will result in unacceptable precision loss. This application adopts a hybrid quantization method that quantizes some operation layers to high precision and some operation layers to low precision to meet the precision requirements.

[0080] The target operation module may refer to a module in a neural network model that includes two operation branches. For example, Figure 2 The part from mul to contact at the top of the figure is a target operation module. Figure 2 It only captures a part of a neural network model. The entire neural network model contains multiple similar target operation modules.

[0081] The operation layer can refer to a single operation in a neural network model. For example, Figure 2 The conv (convolution operation), sigmoid, and mul (multiplication operation) in can all be called an operation layer.

[0082] The number of operation layers of the first operation branch may refer to the number of separate operations included in the first operation branch.

[0083] The number of operation layers of the second operation branch may refer to the number of individual operations included in the second operation branch. Figure 2 As shown, the number of operation layers of the second operation branch is 3.

[0084] like Figure 2 As shown, it is a part of the YoloV5 network, which can be regarded as an example of a target operation module. 2-1 is the first operation branch, which contains far more operation layers than the second operation branch 2-2.

[0085] The first operation branch may be a first operation branch including multi-layer operations; and the second operation branch may be a short-circuit branch.

[0086] like Figure 2 , 2-1 is the first operation branch including multiple layers of operations, and 2-2 is a short-circuit branch (shortcut).

[0087] The target operation module for obtaining the neural network model to be quantized includes a first operation branch and a second operation branch, including:

[0088] Determine the target operation module of the neural network model to be quantized;

[0089] According to the target operation module, a first operation branch and a second operation branch included in the target operation module of the neural network model to be quantized are obtained.

[0090] In specific implementation, the target operation module of the neural network model to be quantized can be determined first, and then the first operation branch and the second operation branch included in the target operation module of the neural network model to be quantized can be obtained according to the target operation module.

[0091] The target operation module for determining the neural network model to be quantized includes:

[0092] Traversing the neural network model to be quantized;

[0093] Determine whether the output of the current operation passes through two operation branches. If so, use the output of the current operation as the starting position of the target operation module;

[0094] Determine the merge position of the target operation module;

[0095] The operation between the starting position and the merging position of the target operation module is used as the target operation module.

[0096] For example, Figure 2 The output of mul passes through the two operation branches 2-1 and 2-1, and the output of mul is used as the starting position of the target operation module.

[0097] Determining the merge position of the target operation module includes:

[0098] Obtain the preset merge position identifier;

[0099] The merging position of the target operation module is determined according to the merging position identifier.

[0100] The preset merge position identifier can be contact, add, mul, etc. Figure 2 The contact in is a merged location identifier.

[0101] like Figure 1 As shown, in step S102, the first operation branch is quantized to a first precision.

[0102] like Figure 1 As shown, in step S103, the second operation branch is quantized to a second precision; the first precision is smaller than the second precision.

[0103] The first precision refers to the lower precision used when performing hybrid quantization of neural network models.

[0104] The second precision refers to the higher precision used when performing hybrid quantization of neural network models.

[0105] Hybrid quantization means that different segments of a neural network model run at different accuracies.

[0106] Specifically, the first precision may be eight-bit integer precision; the second precision may be half-precision floating point precision. Depending on the specific scenario, the first precision and the second precision may also be other precisions, as long as the first precision is less than the second precision.

[0107] The following combination Figure 3 The advantages of the first embodiment of the present application over the hybrid strategy of layer-by-layer sensitivity measurement provided in the prior art are introduced as follows:

[0108] 1. This application solution does not need to measure the sensitivity of each layer to quantization. By analyzing the network structure, a hybrid quantization strategy can be quickly given. Figure 3 In

[15] , by analyzing the network structure, we obtain the target operation module and directly set the 3-1 branch to the first precision and the 3-2 branch to the second precision.

[0109] 2. When two operation layers with high sensitivity to quantization can achieve guaranteed accuracy by keeping one of the operation layers as high precision, a better choice can be made than the algorithm mentioned in the layer-by-layer sensitivity measurement. As shown in 3 below, when tested separately, the modules in the 3-3 box and the modules in the 3-2 box are both sensitive to quantization. According to the hybrid algorithm of layer-by-layer sensitivity measurement, since the sensitivity of the 3-3 box and the 3-2 box both meet the FP16 precision, the operations of the 3-3 box and the 3-2 box are both retained in FP16 precision; the algorithm in this application only needs to keep the operations in the 3-2 box as FP16, and the operations in the 3-3 box as INT8 precision. The experimental results show that the two algorithms have basically the same impact on the final accuracy, and the algorithm in this application takes less time to infer the model after hybrid quantization.

[0110] This application determines the target operation module based on the structural characteristics of the neural network model, directly quantizes the first operation branch to the first precision, and quantizes the second operation branch to the second precision, without having to measure its sensitivity to quantization layer by layer, which can significantly improve the running speed of the model.

[0111] It should be noted that Figure 2 The screenshot in the middle only captures a part of the neural network model, not the entire network; a neural network model can contain multiple target operation modules (fork-merge structure), and can also contain modules other than the target operation module.

[0112] As an implementation manner, the first embodiment of the present application may further include:

[0113] The operation modules other than the target operation module in the neural network model are quantized to a second precision.

[0114] As an implementation manner, the first embodiment of the present application may further include:

[0115] Some operation modules other than the target operation module in the neural network model are quantized to the second precision, and another part of the operation modules are quantized to the first precision.

[0116] An operation module may refer to an operation layer.

[0117] Specifically, some operation modules other than the target operation module in the neural network model are quantized to the second precision, and another part of the operation modules are quantized to the first precision. Some operation modules with high sensitivity to quantization can be quantized to the second precision, and another part of the operation modules with low sensitivity to quantization can be quantized to the first precision.

[0118] The first embodiment of this application quantizes only the second operation branch of the target operation module, which contains fewer operations, to the second precision, and quantizes the first operation branch of the target operation module, which contains more operations, to the second precision, which can significantly improve the running speed. According to the experimental conclusions on the YoloV5l network, the hybrid quantization proposed in this solution can increase the running speed by 13.6% with less than 1% accuracy loss compared to quantization to FP16 precision.

[0119] The first embodiment of this application can be applied to scenarios such as image classification, object detection, environment segmentation, and face recognition based on convolutional neural network models. Taking object detection as an example, the technology of this solution can reduce the time required to process a single frame of image and improve throughput.

[0120] So far, the introduction to the first embodiment of the present application is completed. The quantization method of the neural network model provided by the present application quantizes the first branch with a large number of operation layers in the target operation module of the neural network model to be quantized as low precision, and directly quantizes the second branch with a small number of operation layers as high precision. There is no need to measure the quantization sensitivity of each operation layer. Therefore, the efficiency of obtaining the hybrid strategy when performing hybrid quantization on the neural network model is high. In addition, when there is an operation layer that is sensitive to quantization in the first operation branch, it is not necessary to quantize the operation layer therein as high precision. Only the operation layer in the second operation branch can be retained as high precision, thereby achieving the purpose of ensuring accuracy by only retaining the operation layer of one of the branches as high precision. The present application solves the problems of low efficiency and insufficient optimization in obtaining the hybrid strategy when performing hybrid quantization on the neural network model.

[0121] Corresponding to the quantization method of the neural network model provided in the first embodiment of the present application, the second embodiment of the present application provides a data processing device.

[0122] like Figure 4 As shown, the quantization device of the neural network model includes:

[0123] A branch obtaining unit 401 is configured to obtain a first operation branch and a second operation branch included in a target operation module of a neural network model to be quantized; wherein the number of operation layers of the first operation branch is greater than the number of operation layers of the second operation branch;

[0124] A first operation branch quantization unit 402, configured to quantize the first operation branch to a first precision;

[0125] The second operation branch quantization unit 403 is configured to quantize the second operation branch to a second precision; the first precision is smaller than the second precision.

[0126] As an implementation manner, the first operation branch is a first operation branch including multi-layer operations; the second operation branch is a short-circuit branch.

[0127] As an implementation manner, the first precision is eight-bit integer precision; the second precision is half-precision floating point precision.

[0128] As an implementation manner, the branch obtaining unit is specifically configured to:

[0129] Determine the target operation module of the neural network model to be quantized;

[0130] According to the target operation module, a first operation branch and a second operation branch included in the target operation module of the neural network model to be quantized are obtained.

[0131] As an implementation manner, the branch obtaining unit is specifically configured to:

[0132] Traversing the neural network model to be quantized;

[0133] Determine whether the output of the current operation passes through two operation branches. If so, use the output of the current operation as the starting position of the target operation module;

[0134] Determine the merging position of the target operation module; and use the operation between the starting position and the merging position of the target operation module as the target operation module.

[0135] As an implementation manner, the branch obtaining unit is specifically configured to:

[0136] Obtain the preset merge position identifier;

[0137] The merging position of the target operation module is determined according to the merging position identifier.

[0138] As an embodiment, the device further includes:

[0139] The remaining operation module quantization unit is used to quantize the operation modules other than the target operation module to the second precision.

[0140] The quantization device of the neural network model provided by the present application quantizes the first branch with a large number of operation layers in the target operation module of the neural network model to be quantized to low precision, and directly quantizes the second branch with a small number of operation layers to high precision. There is no need to measure the quantization sensitivity of each operation layer. Therefore, the efficiency of obtaining the hybrid strategy when hybrid quantization is performed on the neural network model is relatively high. In addition, when there is an operation layer in the first operation branch that is sensitive to quantization, it is not necessary to quantize the operation layer therein to high precision. Only the operation layer in the second operation branch can be retained as high precision, thereby achieving the purpose of ensuring accuracy by only retaining the operation layer of one of the branches as high precision. The present application solves the problems of low efficiency and insufficient optimization in obtaining the hybrid strategy when hybrid quantization is performed on the neural network model.

[0141] It should be noted that for the detailed description of the data processing device provided in the second embodiment of the present application, reference can be made to the relevant description of the first embodiment of the present application, and no further details will be given here.

[0142] Corresponding to the data processing method provided in the first embodiment of the present application, the third embodiment of the present application provides an electronic device.

[0143] The electronic device comprises:

[0144] processor; and

[0145] The memory is used to store a program of a quantization method for a neural network model. After the device is powered on and the program of the quantization method for the neural network model is run by the processor, the following steps are performed:

[0146] Obtaining a first operation branch and a second operation branch included in a target operation module of a neural network model to be quantized; wherein the number of operation layers of the first operation branch is greater than the number of operation layers of the second operation branch;

[0147] quantizing the first operation branch to a first precision;

[0148] The second operation branch is quantized to a second precision; the first precision is smaller than the second precision.

[0149] As an implementation manner, the first operation branch is a first operation branch including multi-layer operations; the second operation branch is a short-circuit branch.

[0150] As an implementation manner, the first precision is eight-bit integer precision; the second precision is half-precision floating point precision.

[0151] As an embodiment, the target operation module for obtaining the neural network model to be quantized includes a first operation branch and a second operation branch, including:

[0152] Determine the target operation module of the neural network model to be quantized;

[0153] According to the target operation module, a first operation branch and a second operation branch included in the target operation module of the neural network model to be quantized are obtained.

[0154] As an implementation manner, the target operation module for determining the neural network model to be quantized includes:

[0155] Traversing the neural network model to be quantized;

[0156] Determine whether the output of the current operation passes through two operation branches. If so, use the output of the current operation as the starting position of the target operation module;

[0157] Determine the merging position of the target operation module; and use the operation between the starting position and the merging position of the target operation module as the target operation module.

[0158] As an implementation manner, determining the merge position of the target operation module includes:

[0159] Obtain the preset merge position identifier;

[0160] The merging position of the target operation module is determined according to the merging position identifier.

[0161] As an embodiment, the electronic device further performs the following steps:

[0162] The operation modules other than the target operation module are quantized to a second precision.

[0163] The electronic device provided by the present application is powered on and runs the program of the quantization method of the neural network model stored in the memory through the processor. The first branch with a large number of operation layers in the target operation module of the neural network model to be quantized is quantized to low precision, and the second branch with a small number of operation layers is directly quantized to high precision. There is no need to measure the quantization sensitivity of each operation layer. Therefore, the efficiency of obtaining the hybrid strategy when hybrid quantization is performed on the neural network model is high. In addition, when there is an operation layer that is sensitive to quantization in the first operation branch, it is not necessary to quantize the operation layer therein to high precision. Only the operation layer in the second operation branch can be retained as high precision, thereby achieving the purpose of ensuring accuracy by only retaining the operation layer of one of the branches as high precision. The present application solves the problems of low efficiency and insufficient optimization in obtaining the hybrid strategy when hybrid quantization is performed on the neural network model.

[0164] It should be noted that for the detailed description of the electronic device provided in the third embodiment of the present application, reference can be made to the relevant description of the first embodiment of the present application, and no further details will be given here.

[0165] Corresponding to the data processing method provided in the first embodiment of the present application, the fourth embodiment of the present application provides a storage medium storing a program of the data processing method, which is executed by a processor to perform the following steps:

[0166] Obtaining a first operation branch and a second operation branch included in a target operation module of a neural network model to be quantized; wherein the number of operation layers of the first operation branch is greater than the number of operation layers of the second operation branch;

[0167] quantizing the first operation branch to a first precision;

[0168] The second operation branch is quantized to a second precision; the first precision is smaller than the second precision.

[0169] It should be noted that for the detailed description of the storage device provided in the fourth embodiment of the present application, reference can be made to the relevant description of the first embodiment of the present application, and no further details will be given here.

[0170] Although the present application is disclosed as above with the preferred embodiments, it is not intended to limit the present application. Any person skilled in the art may make possible changes and modifications without departing from the spirit and scope of the present application. Therefore, the scope of protection of the present application shall be based on the scope defined by the claims of the present application.

[0171] In a typical configuration, a computing device includes one or more processors (CPUs), memory-mapped input / output interfaces, network interfaces, and memory.

[0172] Memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. Memory is an example of a computer-readable medium.

[0173] Computer-readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. The information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include non-transitory media such as modulated data signals and carrier waves.

[0174] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present application may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

Claims

1. A quantization method for a neural network model, characterized in that: include: By dividing the network structure of the neural network model to be quantized, a first operation branch and a second operation branch are obtained, which are included in the target operation module of the neural network model to be quantized; wherein the input of the neural network model is image data, the output of the neural network model is a processing result of the image data, and the number of operation layers of the first operation branch is greater than the number of operation layers of the second operation branch; quantizing the first operation branch to a first precision; quantizing the second operation branch to a second precision; the first precision is smaller than the second precision; The step of obtaining the first operation branch and the second operation branch included in the target operation module of the neural network model to be quantized includes: Traversing the neural network model to be quantized; in response to the output of the current operation passing through two operation branches, determining the starting position and the merging position of the target operation module, and taking the operation operation between the starting position and the merging position of the target operation module as the target operation module of the neural network model to be quantized; obtaining the first operation branch and the second operation branch contained in the target operation module.

2. The method according to claim 1, characterized in that The first operation branch is a first operation branch including multi-layer operations; the second operation branch is a short-circuit branch.

3. The method according to claim 1, characterized in that The first precision is eight-bit integer precision; the second precision is half-precision floating-point precision.

4. The method according to claim 1, wherein Determining the starting position of the target operation module includes: In response to the output of the current operation passing through the two operation branches, the output of the current operation is used as the starting position of the target operation module.

5. The method according to claim 1, wherein Determining the merge position of the target operation module includes: Obtain the preset merge position identifier; The merging position of the target operation module is determined according to the merging position identifier.

6. The method according to claim 1, characterized in that Also includes: The operation modules other than the target operation module in the neural network model are quantized to a second precision.

7. A quantization device for a neural network model, characterized in that: include: a branch obtaining unit, configured to obtain a first operation branch and a second operation branch included in a target operation module of the neural network model to be quantized by dividing the network structure of the neural network model to be quantized; wherein the input of the neural network model is image data, the output of the neural network model is a processing result of the image data, and the number of operation layers of the first operation branch is greater than the number of operation layers of the second operation branch; A first operation branch quantization unit, configured to quantize the first operation branch to a first precision; A second operation branch quantization unit, configured to quantize the second operation branch to a second precision; the first precision is smaller than the second precision; The branch obtaining unit is also used to traverse the neural network model to be quantified; in response to the output of the current operation passing through two operation branches, determine the starting position and merging position of the target operation module, and use the operation operation between the starting position and the merging position of the target operation module as the target operation module of the neural network model to be quantified; obtain the first operation branch and the second operation branch contained in the target operation module.

8. An electronic device, characterized in that: include: processor; as well as The memory is used to store a program of a quantization method for a neural network model. After the device is powered on and the program of the quantization method for the neural network model is run by the processor, the following steps are performed: By dividing the network structure of the neural network model to be quantized, a first operation branch and a second operation branch are obtained, which are included in the target operation module of the neural network model to be quantized; wherein the input of the neural network model is image data, the output of the neural network model is a processing result of the image data, and the number of operation layers of the first operation branch is greater than the number of operation layers of the second operation branch; quantizing the first operation branch to a first precision; quantizing the second operation branch to a second precision; the first precision is smaller than the second precision; The step of obtaining the first operation branch and the second operation branch included in the target operation module of the neural network model to be quantized includes: Traversing the neural network model to be quantized; in response to the output of the current operation passing through two operation branches, determining the starting position and the merging position of the target operation module, and taking the operation operation between the starting position and the merging position of the target operation module as the target operation module of the neural network model to be quantized; obtaining the first operation branch and the second operation branch contained in the target operation module.

9. A storage medium, characterized in that: A program storing a data processing method is executed by a processor to perform the following steps: By dividing the network structure of the neural network model to be quantized, a first operation branch and a second operation branch are obtained, which are included in the target operation module of the neural network model to be quantized; wherein the input of the neural network model is image data, the output of the neural network model is a processing result of the image data, and the number of operation layers of the first operation branch is greater than the number of operation layers of the second operation branch; quantizing the first operation branch to a first precision; quantizing the second operation branch to a second precision; the first precision is smaller than the second precision; The step of obtaining the first operation branch and the second operation branch included in the target operation module of the neural network model to be quantized includes: Traversing the neural network model to be quantized; in response to the output of the current operation passing through two operation branches, determining the starting position and the merging position of the target operation module, and using the operation operation between the starting position and the merging position of the target operation module as the target operation module of the neural network model to be quantized; A first operation branch and a second operation branch included in the target operation module are obtained.

Images