Model quantization methods, apparatus, computer equipment and storage media

By quantizing and truncating the network layer weights and input feature values ​​of the image classification model, the problems of reduced quantization accuracy and increased computational overhead caused by long-tailed distribution are solved, and more efficient model operation is achieved.

CN119107479BActive Publication Date: 2026-11-10GUANGDONG OPPO MOBILE TELECOMMUNICATIONS CORP LTD
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202310671337.4
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2023-06-07
Publication Date
2026-11-10
Estimated Expiration
2043-06-07

AI Technical Summary

Technical Problem

During model quantization, since the weights and input feature values ​​of some network layers in the neural network model are distributed in a long tail, direct low-bit-width quantization will lead to a decrease in quantization accuracy, while increasing the operation time and computational overhead of the multiplier, thus reducing the running efficiency of the neural network model.

Method used

The weights and input feature values ​​are quantized based on the first quantization bit width to obtain the first quantized weights and feature values. Then, they are truncated with a length of the second quantization bit width to obtain the second quantized weights and feature values. Weighting is then performed to reduce the computational overhead and operation time of the multiplier.

Benefits of technology

While maintaining a large numerical representation range and high quantization accuracy, the computational overhead and operation time of the multiplier are effectively reduced, thereby improving the running efficiency of the image classification model.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119107479B_ABST
    Figure CN119107479B_ABST
Patent Text Reader

Abstract

This application provides a model quantization method, apparatus, computer device, and storage medium, belonging to the field of computer technology. The method includes: for any network layer of an image classification model, when the weights of the network layer and the input feature values ​​of the input image in the network layer exhibit a long-tailed distribution, quantizing the weights and input feature values ​​based on a first quantization bit width to obtain the first quantized weights and first quantized feature values ​​of the network layer; truncating the first quantized weights and first quantized feature values ​​respectively, when the first quantization bit width is not less than a second quantization bit width, to obtain second quantized weights and second quantized feature values ​​with a length of the second quantization bit width; and weighting the second quantized weights and second quantized feature values ​​to obtain the output feature values ​​of the network layer. This scheme can effectively reduce the computational overhead of the multiplier while maintaining a large numerical representation range and high quantization accuracy for the second quantized weights and second quantized feature values.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a model quantization method, apparatus, computer device, and storage medium. Background Technology

[0002] With the continuous development of artificial intelligence technology, neural network models are widely used in fields such as image denoising, image enhancement, image recognition, and segmentation. To improve the operating efficiency of neural network models, they are typically quantized before being deployed on the terminal. However, during model quantization, because the weights and input feature values ​​of some network layers exhibit a long-tailed distribution, directly quantizing these weights and feature values ​​increases the quantization interval, leading to a decrease in quantization accuracy. Therefore, how to reduce the impact of long-tailed distribution on model quantization is a technical problem that needs to be solved.

[0003] In related technologies, during the model quantization process, more quantization bit width is usually allocated to network layers with long-tail distribution (e.g., from 8 bits to 16 bits). By quantizing the weights and input feature values ​​of the network layers with high bit width, the quantization accuracy and numerical representation range of the weights and input feature values ​​of the network layers can be improved, thereby improving the quality of the image output by the neural network model.

[0004] However, after quantizing the weights and input feature values ​​of the network layer, a multiplier is still needed to process the quantized weights and input feature values ​​to obtain the output feature values ​​of the network layer. This approach, while increasing the quantization bit width, also increases the computation time and overhead of the multiplier, reducing the operating efficiency of the neural network model. Summary of the Invention

[0005] This application provides a model quantization method, apparatus, computer device, and storage medium, which effectively reduces the computational overhead and operation time of the multiplier while maintaining a large numerical representation range and high quantization accuracy of the second quantization weight and the second quantization feature value, thereby improving the operating efficiency of the image classification model. The technical solution is as follows:

[0006] According to one aspect of the embodiments of this application, a model quantization method is provided, the method comprising:

[0007] For any network layer of the image classification model, when the weights of the network layer and the input feature values ​​of the input image in the network layer have a long-tailed distribution, the weights and the input feature values ​​are quantized based on the first quantization bit width to obtain the first quantized weights of the network layer and the first quantized feature values ​​of the input image in the network layer. The image classification model is used to predict the category of the input image, and the input feature values ​​are used to represent the input feature map corresponding to the input image in the network layer.

[0008] When the first quantization bit width is not less than the second quantization bit width, the first quantization weight and the first quantization feature value are truncated respectively to obtain the second quantization weight and the second quantization feature value with a length of the second quantization bit width. The first bit data of the second quantization weight is the first binary number that is 1 from the high bit in the first quantization weight, and the first bit data of the second quantization feature value is the first binary number that is 1 from the high bit in the first quantization feature value.

[0009] The second quantization weight and the second quantization feature value are weighted to obtain the output feature value of the network layer. The output feature value is used to represent the output feature map of the input image in the network layer.

[0010] According to another aspect of the embodiments of this application, a model quantization apparatus is provided, the apparatus comprising:

[0011] The quantization module is used to quantize the weights and input feature values ​​of the input image in any network layer of the image classification model, based on a first quantization bit width, when the weights of the network layer and the input feature values ​​of the input image in the network layer have a long-tailed distribution, to obtain the first quantized weights of the network layer and the first quantized feature values ​​of the input image in the network layer. The image classification model is used to predict the category of the input image, and the input feature values ​​are used to represent the input feature map corresponding to the input image in the network layer.

[0012] The truncation module is used to truncate the first quantization weight and the first quantization feature value respectively when the first quantization bit width is not less than the second quantization bit width, to obtain a second quantization weight and a second quantization feature value with a length of the second quantization bit width. The first bit data of the second quantization weight is the first binary number with a 1 starting from the high bit in the first quantization weight, and the first bit data of the second quantization feature value is the first binary number with a 1 starting from the high bit in the first quantization feature value.

[0013] The weighting module is used to weight the second quantization weight and the second quantization feature value to obtain the output feature value of the network layer. The output feature value is used to represent the output feature map of the input image in the network layer.

[0014] In some embodiments, the first quantization bit width is p bits, the first quantization weight and the first quantization feature value are composed of p-bit binary numbers, the second quantization bit width is k bits, the second quantization weight and the second quantization feature value are composed of k-bit binary numbers, and p and k are both positive integers;

[0015] The truncation module includes:

[0016] The first determining unit is configured to determine the m-th binary number from the first quantization weight and the n-th binary number from the first quantization feature value, provided that the first quantization bit width is not less than the second quantization bit width. The m-th binary number is the first 1-bit binary number in the first quantization weight, and the n-th binary number is the first 1-bit binary number in the first quantization feature value, where m and n are both positive integers.

[0017] The first truncation unit is used to truncate a binary number of length k from the first quantization weight, starting from the m-th binary number, to obtain the second quantization weight;

[0018] The second truncation unit is used to extract a binary number of length k from the first quantization feature value, starting from the nth binary number, to obtain the second quantization feature value.

[0019] In some embodiments, the first truncation unit is configured to, when m is not less than k, determine the (m-k+1)th binary number from the first quantization weight based on the m-th binary number and the second quantization bit width; when the (m-k+1)th binary number is 0, set the (m-k+1)th binary number to 1; truncate the binary number from the m-th bit to the (m-k+1)-th bit from the first quantization weight to obtain the second quantization weight, wherein the (m-k+1)th binary number is the last bit of the second quantization weight.

[0020] In some embodiments, the first truncation unit is further configured to, when m is less than k, truncate m bits from the first quantization weight starting from the m-th binary number; fix a decimal point at the end of the m-th binary number; and add km zeros after the decimal point to obtain the second quantization weight.

[0021] In some embodiments, the second truncation unit is configured to, when n is not less than k, determine the (n-k+1)th binary bit from the first quantization feature value based on the nth binary bit and the second quantization bit width; when the (n-k+1)th binary bit is 0, set the (n-k+1)th binary bit to 1; and truncate the binary bits from the nth bit to the (n-k+1)th bit from the first quantization feature value to obtain the second quantization feature value, wherein the (n-k+1)th binary bit is the last bit of the second quantization feature value.

[0022] In some embodiments, the second truncation unit is further configured to, when n is less than k, truncate n bits from the first quantization feature value starting from the nth binary bit; fix a decimal point at the end of the n bits; and add kn zeros after the decimal point to obtain the second quantization feature value.

[0023] In some embodiments, the truncation module is further configured to: determine, when the first quantization bit width is not less than the second quantization bit width, a binary number starting from the high bit of the first quantization weight and the first quantization feature value; when the number of bits of the first 1 binary number of the first quantization weight is less than the second quantization bit width, truncate a binary number with a length equal to the second quantization bit width starting from the last bit of the first quantization weight to obtain the second quantization weight, wherein the last bit of the second quantization weight is the last bit of the first quantization weight; and when the number of bits of the first 1 binary number of the first quantization feature value is less than the second quantization bit width, truncate a binary number with a length equal to the second quantization bit width starting from the last bit of the first quantization feature value to obtain the second quantization feature value, wherein the last bit of the second quantization feature value is the last bit of the first quantization feature value.

[0024] In some embodiments, the weighting module includes:

[0025] A calculation unit is used to multiply the second quantization weight and the second quantization feature value to obtain the intermediate feature value of the network layer;

[0026] The second determining unit is used to determine a first truncation bit width based on the first quantization weight and the second quantization weight. The first truncation bit width is used to indicate the number of low-order binary numbers discarded in the first quantization weight.

[0027] The third determining unit is used to determine the second truncation bit width based on the first quantization feature value and the second quantization feature value. The second truncation bit width is used to indicate the number of low-order binary numbers discarded in the first quantization feature value.

[0028] A shift unit is used to shift the intermediate feature value based on the sum of the first truncation bit width and the second truncation bit width to obtain the output feature value.

[0029] According to another aspect of the embodiments of this application, a computer device is provided, the computer device including a processor and a memory; the memory stores at least one piece of program code, the at least one piece of program code being executed by the processor to implement the model quantization method as described above.

[0030] According to another aspect of the embodiments of this application, a chip is provided, the chip including programmable logic circuits and / or program instructions, which, when the chip is run on a terminal, are used to implement the model quantization method described above.

[0031] According to another aspect of the embodiments of this application, a computer-readable storage medium is provided, the storage medium storing at least one piece of program code, the at least one piece of program code being executed by a processor to implement the model quantization method as described above.

[0032] According to another aspect of the embodiments of this application, a computer program product is provided, which stores at least one piece of program code, the at least one piece of program code being executed by a processor to implement the model quantization method described above.

[0033] This application provides a model quantization method. When the weights and input feature values ​​of the network layer in an image classification model exhibit a long-tailed distribution, the method quantizes the weights and input feature values ​​based on a first quantization bit width. This yields first quantized weights and first quantized feature values ​​with a large numerical range and high quantization accuracy. Then, based on a second quantization bit width lower than the first quantization bit width, the first quantized weights and first quantized feature values ​​are truncated. This allows for the extraction of second quantized weights and second quantized feature values ​​with a lower quantization bit width. Therefore, while maintaining a large numerical range and high quantization accuracy for the second quantized weights and second quantized feature values, the method effectively reduces the computational overhead and operation time of the multiplier, thereby improving the operating efficiency of the image classification model. Attached Figure Description

[0034] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0035] Figure 1This is a schematic diagram of the implementation environment of a model quantization method provided in an embodiment of this application;

[0036] Figure 2 This is a flowchart of a model quantization method provided in an embodiment of this application;

[0037] Figure 3 This is a flowchart of another model quantization method provided in the embodiments of this application;

[0038] Figure 4 This is a schematic diagram of a long-tail distribution provided in an embodiment of this application;

[0039] Figure 5 This is a schematic diagram of a DRUM method provided in an embodiment of this application;

[0040] Figure 6 This is a schematic diagram of a first cutoff bit width provided in an embodiment of this application;

[0041] Figure 7 This is an overall framework diagram of a model quantization method provided in an embodiment of this application;

[0042] Figure 8 This is a schematic diagram of the structure of a model quantization device provided in an embodiment of this application;

[0043] Figure 9 This is a schematic diagram of another model quantization device provided in an embodiment of this application;

[0044] Figure 10 This is a structural block diagram of a computer device provided in an embodiment of this application. Detailed Implementation

[0045] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0046] In this article, "at least one" refers to one or more, and "multiple" refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. The character " / " generally indicates that the preceding and following related objects have an "or" relationship.

[0047] It should be noted that all information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data used for analysis, stored data, displayed data, etc.), and signals involved in this application are authorized by the user or fully authorized by all parties, and the collection, use, and processing of related data must comply with the relevant laws, regulations, and standards of the relevant countries and regions. For example, the image classification model and input images involved in this application were obtained with full authorization.

[0048] The model quantization method provided in this application can be executed by a computer device. In some embodiments, the computer device is a terminal or a server. The following describes the implementation environment of the model quantization method provided in this application, using a computer device as an example. Figure 1 This is a schematic diagram illustrating the implementation environment of a model quantization method provided in an embodiment of this application. See also... Figure 1 The implementation environment includes a terminal 101 and a server 102. The terminal 101 can be directly or indirectly connected to the server 102 via wired or wireless communication, which is not limited herein.

[0049] In some embodiments, terminal 101 can be various types of terminals such as mobile phones, desktop computers, laptops, tablets, smart voice interaction devices, and in-vehicle terminals. Terminal 101 has an application that supports image classification installed and running. Terminal 101 can quantize the image classification model through this application. Illustratively, a user can input the image to be processed into the image classification model through this application. During the image classification model's processing of the image, terminal 101 can quantize the weights and input feature values ​​of multiple network layers in the image classification model, thereby improving the operating efficiency of the image classification model.

[0050] In some embodiments, server 102 can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms. Server 102 can train an image classification model and send the trained image classification model to terminal 101, where terminal 101 quantizes the image classification model.

[0051] In some embodiments, server 102 undertakes the main computing work and terminal 101 undertakes the secondary computing work; or, server 102 undertakes the secondary computing work and terminal 101 undertakes the main computing work; or, server 102 and terminal 101 collaborate on computing using a distributed computing architecture.

[0052] It should be noted that the model quantization method provided in this application embodiment can not only quantize image classification models applied in the field of image classification, but also quantize image processing models applied in other image processing fields such as image denoising, image quality enhancement, image recognition, and image segmentation. The following uses an image classification model as an example to introduce the model quantization method provided in this application embodiment.

[0053] Figure 2 This is a flowchart of a model quantization method provided in an embodiment of this application. The method is executed by a terminal; see [link to relevant documentation]. Figure 2 The method includes:

[0054] 201. For any network layer of the image classification model, when the weights of the network layer and the input feature values ​​of the input image in the network layer have a long tail distribution, the terminal quantizes the weights and input feature values ​​based on the first quantization bit width to obtain the first quantized weights of the network layer and the first quantized feature values ​​of the input image in the network layer. The image classification model is used to predict the category of the input image, and the input feature values ​​are used to represent the input feature map corresponding to the input image in the network layer.

[0055] In this embodiment, the image classification model is a trained neural network model. The image classification model includes multiple network layers, each containing multiple nodes. The weights of the network layers reflect the connections between these nodes. During the processing of the input image by the image classification model, the first network layer extracts features from the input image, obtaining multiple feature values. These feature values ​​serve as input feature values ​​for the next network layer, representing the corresponding input feature map of the input image in that layer. Since the weights and input feature values ​​of the network layers are high-precision floating-point data, to improve the efficiency of the image classification model, the terminal can quantize the weights and input feature values, converting their data type from floating-point to fixed-point. During the quantization process, if the weights and input feature values ​​exhibit a long-tailed distribution, it indicates an unbalanced data distribution. Directly performing low-bit-width quantization would increase the quantization interval of the weights and input feature values, resulting in reduced quantization accuracy. Therefore, in this application, the terminal can perform high-bit-width quantization on the weights and input feature values ​​based on the first quantization bit width to obtain the first quantized weights and the first quantized feature values ​​of the network layer. The first quantized weights and the first quantized feature values ​​are fixed-point data with a length equal to the first quantization bit width, and this fixed-point data consists of multiple binary bits. Taking a first quantization bit width of 16 bits and weights and input feature values ​​of 32-bit floating-point numbers as an example, the terminal can quantize the weights and input feature values ​​based on the first quantization bit width, converting the weights and input feature values ​​from 32-bit floating-point numbers into 16-bit fixed-point numbers, thereby obtaining the first quantized weights and the second quantized feature values.

[0056] 202. When the first quantization bit width is not less than the second quantization bit width, the terminal truncates the first quantization weight and the first quantization feature value respectively to obtain the second quantization weight and the second quantization feature value with a length of the second quantization bit width. The first bit of the second quantization weight is the first binary number that is 1 from the high bit of the first quantization weight, and the first bit of the second quantization feature value is the first binary number that is 1 from the high bit of the first quantization feature value.

[0057] In this embodiment, the terminal converts the weights and input feature values ​​of the network layer from 32-bit floating-point numbers to 8-bit fixed-point numbers, thus setting the second quantization bit width to 8 bits. When the first quantization bit width is not less than the second quantization bit width, it indicates that the length of the first quantized weights and first quantized feature values ​​obtained based on the first quantization bit width is greater than the second quantization bit width. Therefore, to obtain weights and input feature values ​​with a length of the second quantization bit width, the terminal truncates the first quantization weights and first quantized feature values ​​to obtain second quantization weights and second quantization feature values ​​with a length of the second quantization bit width. Compared to the first quantization weights and first quantization feature values, the second quantization weights can represent the connection relationships of multiple nodes in the network layer with less data, and the second quantization feature values ​​can represent the input feature map corresponding to the input image in the network layer with less data. The first quantization weights and first quantization feature values ​​consist of multiple binary numbers arranged from left to right, with the leftmost binary number being the most significant bit. The first bit of the second quantization weight is the first 1 bit in the first quantization weight, and the first bit of the second quantization feature value is the first 1 bit in the first quantization feature value. By discarding the last bit of low-precision data in the first quantization weight and the first quantization feature value, the terminal can reduce the quantization bit width of the weight and input feature value while maintaining a large numerical representation range and high quantization accuracy.

[0058] 203. The terminal weights the second quantization weight and the second quantization feature value to obtain the output feature value of the network layer. The output feature value is used to represent the output feature map of the input image in the network layer.

[0059] In this embodiment, after extracting a second quantization weight and a second quantization feature value with a length equal to the second quantization bit width from the first quantization weight and the first quantization feature value, the terminal can process the second quantization feature value based on the second quantization weight to obtain the output feature value of the network layer. This output feature value represents the output feature map of the input image in the network layer. Compared to inputting the first quantization weight and the first quantization feature value into a multiplier to obtain the output feature value, the terminal uses a multiplier to process the shorter second quantization weight and the second quantization feature value to obtain the output feature value. This effectively reduces the computational overhead and operation time of the multiplier, accelerating the running efficiency of the image classification model.

[0060] This application provides a model quantization method. When the weights and input feature values ​​of the network layer in an image classification model exhibit a long-tailed distribution, the method quantizes the weights and input feature values ​​based on a first quantization bit width. This yields first quantized weights and first quantized feature values ​​with a large numerical range and high quantization accuracy. Then, based on a second quantization bit width lower than the first quantization bit width, the first quantized weights and first quantized feature values ​​are truncated. This allows for the extraction of second quantized weights and second quantized feature values ​​with a lower quantization bit width. Therefore, while maintaining a large numerical range and high quantization accuracy for the second quantized weights and second quantized feature values, the method effectively reduces the computational overhead and operation time of the multiplier, thereby improving the operating efficiency of the image classification model.

[0061] Figure 3 This is a flowchart of another model quantization method provided in an embodiment of this application. This method is executed by a terminal; see [link to relevant documentation]. Figure 3 The method includes:

[0062] 301. For any network layer of the image classification model, when the weights of the network layer and the input feature values ​​of the input image in the network layer have a long-tailed distribution, the terminal quantizes the weights and input feature values ​​based on the first quantization bit width to obtain the first quantized weights of the network layer and the first quantized feature values ​​of the input image in the network layer. The image classification model is used to predict the category of the input image, and the input feature values ​​are used to represent the input feature map corresponding to the input image in the network layer.

[0063] In this embodiment, the image classification model is a trained neural network model. The image classification model includes multiple network layers, each containing multiple nodes, and the weights of the network layers reflect the connections between the nodes. The image classification model is used to predict the category of the input image.

[0064] In the process of image classification models processing input images, the first network layer of the model extracts features from the input image, obtaining multiple feature values. These feature values ​​serve as input feature values ​​for the next network layer, representing the input feature map of the input image in that next layer. In other words, the input feature value of a network layer is the output feature value of the previous network layer, and the output feature value of a network layer is the input feature value of the next network layer. Since the weights and input feature values ​​of the network layers are high-precision floating-point data, to improve the efficiency of the image classification model, the terminal can quantize the weights and input feature values, converting their data type from floating-point to fixed-point. However, during the quantization process, if the weights and input feature values ​​exhibit a long-tailed distribution, it indicates an unbalanced data distribution. Directly performing low-bit-width quantization increases the quantization interval of the weights and input feature values, resulting in a decrease in quantization accuracy. Therefore, in this application, the terminal can first perform high-width quantization on the weights and input feature values ​​based on the first quantization bit width to obtain the first quantized weights and the first quantized feature values ​​of the network layer. The first quantized weights and the first quantized feature values ​​are fixed-point data with a length of the first quantization bit width, and this fixed-point data consists of multiple binary numbers. The first quantization bit width can be 16 bits, 12 bits, 10 bits, etc., and this application does not limit this.

[0065] For example, Figure 4 This is a schematic diagram of a long-tail distribution provided in an embodiment of this application. For example... Figure 4 As shown, the long-tail distribution is a data distribution with uneven distribution. A small subset of categories at the head of the curve have a larger weight, while the majority of categories at the tail of the curve have a smaller weight.

[0066] 302. When the first quantization bit width is not less than the second quantization bit width, the terminal determines the m-th binary number from the first quantization weight and the n-th binary number from the first quantization feature value. The m-th binary number is the first 1 from the high-order bit of the first quantization weight, and the n-th binary number is the first 1 from the high-order bit of the first quantization feature value. Both m and n are positive integers.

[0067] In this embodiment, when the first quantization bit width is not less than the second quantization bit width, it means that the length of the first quantization weight and the first quantization feature value obtained based on the first quantization bit width is greater than the second quantization bit width. During the truncation of the first quantization weight and the first quantization feature value, the terminal can determine the first 1 bit from the first quantization weight and use it as the first bit of the second quantization weight. Similarly, the terminal can determine the first 1 bit from the first quantization feature value and use it as the first bit of the second quantization feature value. Here, the first quantization bit width is p bits, and the first quantization weight and the first quantization feature value are composed of p bits arranged sequentially from left to right. The leftmost bit in the first quantization weight and the first quantization feature value is the most significant bit. The first 1 bit from the most significant bit in the first quantization weight is the m-th bit in the p-bit binary number, and the first 1 bit from the most significant bit in the first quantization feature value is the n-th bit in the p-bit binary number. m and n may be the same or different.

[0068] For example, if the first quantization weight is "0001011110010", starting from the left, we search for the first binary number that is 1 in the first quantization weight. The fourth binary number from the left is the first binary number that is 1 from the most significant bit in the first quantization weight, and in this case, m = 10. If the first quantization feature value is "0101011110010", starting from the left, we search for the first binary number that is 1 in the first quantization feature value. The second binary number from the left is the first binary number that is 1 from the most significant bit in the first quantization feature value, and in this case, n = 12.

[0069] 303. Starting from the m-th bit of the binary number, the terminal extracts a binary number of length k from the first quantization weight to obtain the second quantization weight. The first bit of the second quantization weight is the first 1 bit of the first quantization weight.

[0070] In this embodiment, the first bit of the second quantization weight is the first 1 bit of the first quantization weight, and this first 1 bit is the m-th bit of the first quantization weight. Therefore, the terminal can extract a k-bit binary number from the p-bit binary number of the first quantization weight, starting from the m-th bit, to obtain the second quantization weight with a length equal to the second quantization bit width. Compared to the first quantization weight, the second quantization weight can represent the connection relationship of multiple nodes in the network layer with less data. The second quantization bit width is k bits, and the second quantization weight consists of k bits. Since the terminal typically converts the weights of the network layer from 32-bit floating-point numbers to 8-bit fixed-point numbers, the second quantization bit width is usually set to 8 bits, i.e., k = 8.

[0071] In some embodiments, the terminal can extract a binary number of length k starting from the m-th bit of the first quantization weight to obtain the second quantization weight. Accordingly, when m is not less than k, the terminal determines the (m-k+1)-th binary number from the first quantization weight based on the m-th bit and the second quantization bit width; if the (m-k+1)-th bit is 0, it sets the (m-k+1)-th bit to 1; and extracts the binary number from the m-th bit to the (m-k+1)-th bit from the first quantization weight to obtain the second quantization weight, where the (m-k+1)-th bit is the last bit of the second quantization weight. Here, the condition that m is not less than k indicates that the number of bits of the first 1 bit in the first quantization weight is not less than the second quantization bit width. Therefore, the terminal can determine the (m-k+1)-th binary number from the first quantization weight based on the m-th bit and the second quantization bit width. Since the m-th and (m-k+1)-th binary bits are the first and last bits of the second quantization weight, respectively, the terminal can obtain a second quantization weight of length k by truncating bits from the m-th to (m-k+1)-th binary bits of the first quantization weight. Here, the first bit represents the most significant bit of the second quantization weight, and the last bit represents the least significant bit. When the (m-k+1)-th binary bit is 0, setting it to 1 allows for unbiased, approximately low-bit-width quantization while maintaining a relatively high numerical range. By discarding the low-precision data from the 1st to the mk-th bits of the first quantization weight, the quantization bit width of the weight can be reduced while ensuring a large numerical representation range and high quantization precision.

[0072] For example, Figure 5 This is a schematic diagram of a DRUM (Dynamic Range Unbiased Multiplier) method provided in an embodiment of this application. Figure 5 As shown, taking an unsigned fixed-point integer as the first quantization weight, which is "0000001011110010", and the second quantization bit width is 8 bits, the terminal first searches for the first binary number that is 1, starting from the most significant bit of the first quantization weight. This first binary number is the 10th bit of the first quantization weight. Starting from the 10th bit, the terminal determines the 3rd bit, from the most significant bit to the least significant bit. Since the 3rd bit is 0, it is set to 1. Then, the terminal extracts bits 10 to 3 from the first quantization weight, resulting in the 8-bit binary number "10111101". The terminal uses these 8 bits as the second quantization weight.

[0073] In some embodiments, when m is less than k, the terminal starts from the m-th binary digit and truncates m binary digits from the first quantization weight. Then, a decimal point is fixed at the end of the m binary digits, and k-m zeros are added after the decimal point to obtain the second quantization weight. When m is less than k, it indicates that the number of bits of the first binary digit that is 1 starting from the high bit in the first quantization weight is smaller than the second quantization bit width. To obtain the second quantization weight with a length of k, the terminal may first truncate the binary digits from the m-th bit to the 1st bit from the first quantization weight to obtain m binary digits. By fixing the decimal point at the end of the m binary digits and adding k-m zeros after the decimal point, the terminal can obtain the second quantization weight that meets the second quantization bit width without changing the first quantization weight.

[0074] For example, taking the first quantization weight as "0000000001110010" and k=8 as an example, the first binary digit that is 1 starting from the high bit in the first quantization weight is the 7th binary digit, that is, m=7<k. The terminal starts from the 7th binary digit of the first quantization weight, truncates 7 binary digits, and obtains "1110010". The terminal fixes the decimal point at the end of the 7 binary digits to obtain "1110010.". Then, one zero is added after the decimal point to obtain "1110010.0", that is, the second quantization weight is "1110010.0".

[0075] 304. The terminal starts from the n-th binary digit, truncates a binary number with a length of k from the first quantization feature value to obtain a second quantization feature value, where the first digit of the second quantization feature value is the first binary digit that is 1 starting from the high bit in the first quantization feature value.

[0076] In the embodiments of the present application, since the first digit of the second quantization feature value is the first binary digit that is 1 starting from the high bit in the first quantization feature value, and the first binary digit that is 1 starting from the high bit is the n-th binary digit of the first quantization feature value. Therefore, the terminal can start from the n-th binary digit and truncate a binary number with a length of k from the p binary digits in the first quantization feature value, so as to obtain the second quantization feature value with the length of the second quantization bit width. Compared with the first quantization feature value, the second quantization feature value can represent the input feature map corresponding to the network layer of the input image with a smaller data volume. The second quantization bit width is k bits, and the second quantization feature value is composed of k binary digits. Since the terminal usually converts the input feature value of the network layer from a 32-bit floating point number to an 8-bit fixed-point number, the second quantization bit width is usually set to 8 bits, that is, k=8.

[0077] In some embodiments, the terminal can extract a second quantization feature value by truncating a binary number of length k, starting from the nth binary bit of the first quantization feature value. Accordingly, when n is not less than k, the terminal determines the (n-k+1)th binary bit from the first quantization feature value based on the nth binary bit and the second quantization bit width; if the (n-k+1)th binary bit is 0, it sets the (n-k+1)th binary bit to 1; and extracts the binary number from the nth bit to the (n-k+1)th bit from the first quantization feature value to obtain the second quantization feature value, where the (n-k+1)th binary bit is the last bit of the second quantization feature value. Wherein, when n is not less than k, it indicates that the number of bits of the first 1 in the first quantization feature value is not less than the second quantization bit width. Therefore, the terminal can determine the (n-k+1)th binary bit from the first quantization feature value based on the nth binary bit and the second quantization bit width. Since the nth and (n-k+1)th binary bits are the first and last bits of the second quantization feature value, respectively, the terminal can obtain a second quantization feature value of length k by truncating bits n to (n-k+1)th bits from the first quantization feature value. Here, the first bit represents the most significant bit of the second quantization feature value, and the last bit represents the least significant bit. When the (n-k+1)th binary bit is 0, setting it to 1 allows for unbiased, approximately low-bit-width quantization while maintaining a relatively high numerical range. By discarding the low-precision data from the 1st to the nkth bits of the first quantization feature value, the quantization bit width of the feature value can be reduced while ensuring a large numerical representation range and high quantization precision.

[0078] In some embodiments, when n is less than k, the terminal starts from the nth binary bit and extracts n bits from the first quantization feature value; a decimal point is fixed at the end of the n bits; and kn zeros are padded after the decimal point to obtain the second quantization feature value. Here, n being less than k indicates that the number of bits in the first quantization feature value that is 1 from the most significant bit is less than the second quantization bit width. To obtain a second quantization feature value of length k, the terminal can first extract the binary bits from the nth to the 1st bit of the first quantization feature value to obtain an n-bit binary number. By fixing the decimal point at the end of the n-bit binary number and padded with kn zeros after the decimal point, the terminal can obtain a second quantization feature value that satisfies the second quantization bit width without changing the first quantization feature value.

[0079] In some embodiments, the terminal can further extract a binary number with a length equal to the second quantization bit width, starting from the last bit of the first quantization weight and the first quantization feature value, to obtain the second quantization weight and the second quantization feature value. Accordingly, when the first quantization bit width is not less than the second quantization bit width, the terminal determines the first binary number starting with a 1 from the first quantization weight and the first quantization feature value; when the number of bits of the first 1 in the first quantization weight is less than the second quantization bit width, the terminal extracts a binary number with a length equal to the second quantization bit width, starting from the last bit of the first quantization weight, to obtain the second quantization weight, with the last bit of the second quantization weight being the last bit of the first quantization weight; when the number of bits of the first 1 in the first quantization feature value is less than the second quantization bit width, the terminal extracts a binary number with a length equal to the second quantization bit width, starting from the last bit of the first quantization feature value, to obtain the second quantization feature value, with the last bit of the second quantization feature value being the last bit of the first quantization feature value. Wherein, "when the first quantization bit width is not less than the second quantization bit width" means that the length of the first quantization weight and the first quantization feature value obtained based on the first quantization bit width is greater than the second quantization bit width. During the truncation of the first quantization weight and the first quantization feature value, the terminal can determine the first 1 binary number starting from the most significant bit in both the first quantization weight and the first quantization feature value. If the number of bits in the first 1 binary number is less than the second quantization bit width, truncation starting from the first 1 binary number would not yield the second quantization weight and the second quantization feature value with a length equal to the second quantization bit width from the first quantization weight and the first quantization feature value. Therefore, the terminal can use the last bit of the first quantization weight and the first quantization feature value as the last bit of the second quantization weight and the second quantization feature value, respectively. Then, starting from the last bit, it truncates binary numbers with a length equal to the second quantization bit width from the first quantization weight and the first quantization feature value to obtain the second quantization weight and the second quantization feature value.

[0080] For example, taking a first quantization weight of "0000000001110010" and a second quantization bit width of 8 bits as an example, the first 1 bit in the first quantization weight is the 7th bit, meaning the number of bits in the first 1 bit is less than the second quantization bit width. Therefore, the terminal can extract a binary number with a length equal to the second quantization bit width, starting from the last bit of the first quantization weight, to obtain "01110010", which means the second quantization weight is "01110010".

[0081] 305. The terminal multiplies the second quantization weight and the second quantization feature value to obtain the intermediate feature value of the network layer.

[0082] In this embodiment, the terminal inputs the second quantization weight and the second quantization feature value, which are of a length of the second quantization bit width, into a multiplier. The multiplier multiplies the multiple binary numbers in the second quantization weight and the second quantization feature value to obtain the intermediate feature value of the network layer.

[0083] For example, if the multiplier is an 8-bit multiplier, the first quantization bit width is 8 bits, the second quantization weight is "10111101", and the second quantization feature value is "10010011", then the intermediate feature value obtained by multiplication is "110110010000111".

[0084] 306. The terminal determines the first truncation bit width based on the first quantization weight and the second quantization weight. The first truncation bit width is used to indicate the number of binary numbers whose low bits are discarded in the first quantization weight.

[0085] In this embodiment, the first truncation bit width is the length of the discarded low-order binary number in the first quantization weight. The terminal determines the position of the first 1 binary number from the first quantization weight, and then truncates the binary number of the second quantization bit width starting from the first 1 binary number to obtain the second quantization weight. Therefore, the untrunculated low-order binary number starting from the first 1 binary number in the first quantization weight is the discarded binary number, and the length of the discarded low-order binary number is the first truncation bit width.

[0086] For example, Figure 6 This is a schematic diagram of a first truncation bit width provided in an embodiment of this application. For example... Figure 6 As shown, the first 1 in the first quantization weight is the t-th bit, and the second quantization bit width is k bits. Therefore, the number of lower-order bits discarded in the first quantization weight is tk, which is also the first truncation bit width.

[0087] 307. The terminal determines the second truncation bit width based on the first quantization feature value and the second quantization feature value. The second truncation bit width is used to indicate the number of binary numbers whose low bits are discarded in the first quantization feature value.

[0088] In this embodiment, the second truncation bit width is the length of the discarded low-order binary number in the first quantization feature value. The terminal determines the position of the first 1 binary number from the first quantization feature value, and then truncates the binary number of the second quantization bit width starting from the first 1 binary number to obtain the second quantization feature value. Therefore, the untrunculated low-order binary number starting from the first 1 binary number in the first quantization feature value is the discarded binary number, and the length of the discarded low-order binary number is the second truncation bit width.

[0089] 308. The terminal shifts the intermediate feature value based on the sum of the first truncation bit width and the second truncation bit width to obtain the output feature value of the network layer. The output feature value is used to represent the output feature map of the input image in the network layer.

[0090] In this embodiment, the terminal sums the first truncated bit width and the second truncated bit width to obtain a shifted value. Based on this shifted value, the terminal can perform a base shift on the intermediate feature value obtained by multiplication to obtain the output feature value of the network layer.

[0091] For example, if the intermediate feature value is "110110010000111", the first truncation bit width is 2, and the second truncation bit width is 3, then the shift bit value is 5, and the output feature value obtained by shifting the intermediate feature value is "001000011100000".

[0092] To more clearly describe the model quantization method provided in the embodiments of this application, the following is combined with... Figure 7 The overall process of the model quantization method will be further described. For example... Figure 7 As shown, the terminal first quantizes the weights and input feature values ​​of the network layer based on a first quantization bit width, obtaining a first quantized weight and a first quantized feature value with a length of the first quantization bit width. Then, the terminal uses the DRUM method provided in this embodiment to truncate the first quantized weight and the second quantized feature value, obtaining a second quantized weight and a second quantized feature value with a length of the second quantization bit width. Furthermore, during the truncation process, the terminal can determine the number of low-order binary bits discarded from the first quantized weight and the number of low-order binary bits discarded from the first quantized feature value, i.e., the first truncation bit width and the second truncation bit width. Finally, the second quantized weight and the second quantized feature value are input into a multiplier to obtain an intermediate feature value. The terminal shifts the intermediate feature value based on the sum of the first truncation bit width and the second truncation bit width, i.e., the shift value, to obtain the output feature value of the network layer.

[0093] This application provides a model quantization method. When the weights and input feature values ​​of the network layer in an image classification model exhibit a long-tailed distribution, the weights and input feature values ​​are quantized based on a first quantization bit width. This yields first quantized weights and first quantized feature values ​​with a large numerical range and high quantization accuracy. Then, based on a second quantization bit width lower than the first quantization bit width, the first quantized weights and first quantized feature values ​​are truncated. This allows for the extraction of second quantized weights and second quantized feature values ​​with a lower quantization bit width. Therefore, while maintaining a large numerical range and high quantization accuracy for the second quantized weights and second quantized feature values, the computational overhead and operation time of the multiplier are effectively reduced, thus improving the quantization efficiency of the image classification model.

[0094] All of the above-mentioned optional technical solutions can be combined in any way to form the optional embodiments of this application, and will not be described in detail here.

[0095] The following are embodiments of the apparatus described in this application, which can be used to execute the embodiments of the method described in this application. For details not disclosed in the apparatus embodiments of this application, please refer to the embodiments of the method described in this application.

[0096] Figure 8 This is a schematic diagram of a model quantization device provided in an embodiment of this application. See also... Figure 8 The device includes:

[0097] The quantization module 801 is used to quantize the weights and input feature values ​​of the input image in any network layer of the image classification model, when the weights of the network layer and the input feature values ​​of the input image in the network layer have a long tail distribution, based on the first quantization bit width, to obtain the first quantized weights of the network layer and the first quantized feature values ​​of the input image in the network layer. The image classification model is used to predict the category of the input image, and the input feature values ​​are used to represent the input feature map corresponding to the input image in the network layer.

[0098] The truncation module 802 is used to truncate the first quantization weight and the first quantization feature value respectively when the first quantization bit width is not less than the second quantization bit width, to obtain the second quantization weight and the second quantization feature value with a length of the second quantization bit width. The first bit data of the second quantization weight is the first binary number with a 1 starting from the high bit in the first quantization weight, and the first bit data of the second quantization feature value is the first binary number with a 1 starting from the high bit in the first quantization feature value.

[0099] The weighting module 803 is used to weight the second quantization weight and the second quantization feature value to obtain the output feature value of the network layer. The output feature value is used to represent the output feature map of the input image in the network layer.

[0100] In some embodiments, the first quantization bit width is p bits, the first quantization weight and the first quantization feature value are composed of p-bit binary numbers, the second quantization bit width is k bits, the second quantization weight and the second quantization feature value are composed of k-bit binary numbers, and p and k are both positive integers;

[0101] Figure 9 This is a schematic diagram of another model quantization device provided in an embodiment of this application. See also... Figure 9 The cutoff module 802 includes:

[0102] The first determining unit 901 is used to determine the m-th binary number from the first quantization weight and the n-th binary number from the first quantization feature value when the first quantization bit width is not less than the second quantization bit width. The m-th binary number is the first 1 from the high-order bit of the first quantization weight and the n-th binary number is the first 1 from the high-order bit of the first quantization feature value. Both m and n are positive integers.

[0103] The first truncation unit 902 is used to truncate a binary number of length k from the first quantization weight, starting from the m-th bit, to obtain the second quantization weight;

[0104] The second truncation unit 903 is used to extract a binary number of length k from the first quantization feature value, starting from the nth binary number, to obtain the second quantization feature value.

[0105] In some embodiments, the first truncation unit 902 is configured to, when m is not less than k, determine the (m-k+1)th binary number from the first quantization weight based on the m-th binary number and the second quantization bit width; when the (m-k+1)th binary number is 0, set the (m-k+1)th binary number to 1; truncate the binary number from the m-th bit to the (m-k+1)-th bit from the first quantization weight to obtain the second quantization weight, wherein the (m-k+1)th binary number is the last bit of the second quantization weight.

[0106] In some embodiments, the first truncation unit 902 is further configured to, when m is less than k, truncate m bits from the first quantization weight starting from the m-th binary number; fix a decimal point at the end of the m-bit binary number; and add km zeros after the decimal point to obtain the second quantization weight.

[0107] In some embodiments, the second truncation unit 903 is configured to, when n is not less than k, determine the (n-k+1)th binary bit from the first quantization feature value based on the nth binary bit and the second quantization bit width; when the (n-k+1)th binary bit is 0, set the (n-k+1)th binary bit to 1; truncate the binary bits from the nth bit to the (n-k+1)th bit from the first quantization feature value to obtain the second quantization feature value, wherein the (n-k+1)th binary bit is the last bit of the second quantization feature value.

[0108] In some embodiments, the second truncation unit 903 is further configured to, when n is less than k, truncate n bits from the first quantization feature value starting from the nth bit of the binary number; fix the decimal point at the end of the n bits of the binary number; and add kn zeros after the decimal point to obtain the second quantization feature value.

[0109] In some embodiments, the truncation module 802 is further configured to: determine, when the first quantization bit width is not less than the second quantization bit width, a binary number starting from the high bit of the first quantization weight and the first quantization feature value respectively; when the number of bits of the first 1 binary number of the first quantization weight is less than the second quantization bit width, truncate a binary number with a length equal to the second quantization bit width starting from the last bit of the first quantization weight to obtain the second quantization weight, wherein the last bit of the second quantization weight is the last bit of the first quantization weight; and when the number of bits of the first 1 binary number of the first quantization feature value is less than the second quantization bit width, truncate a binary number with a length equal to the second quantization bit width starting from the last bit of the first quantization feature value to obtain the second quantization feature value, wherein the last bit of the second quantization feature value is the last bit of the first quantization feature value.

[0110] In some embodiments, see continue to see Figure 9 Weighted module 803 includes:

[0111] The computation unit 904 is used to multiply the second quantization weight and the second quantization feature value to obtain the intermediate feature value of the network layer.

[0112] The second determining unit 905 is used to determine the first truncation bit width based on the first quantization weight and the second quantization weight. The first truncation bit width is used to indicate the number of binary numbers whose low bits are discarded in the first quantization weight.

[0113] The third determining unit 906 is used to determine the second truncation bit width based on the first quantization feature value and the second quantization feature value. The second truncation bit width is used to indicate the number of binary numbers whose low bits are discarded in the first quantization feature value.

[0114] The shift unit 907 is used to shift the intermediate feature value based on the sum of the first truncation bit width and the second truncation bit width to obtain the output feature value.

[0115] This application provides a model quantization device. When the weights and input feature values ​​of the network layer in an image classification model exhibit a long-tailed distribution, the device quantizes the weights and input feature values ​​based on a first quantization bit width. This yields first quantized weights and first quantized feature values ​​with a large numerical range and high quantization accuracy. Then, based on a second quantization bit width lower than the first quantization bit width, the first quantized weights and first quantized feature values ​​are truncated. This allows for the extraction of second quantized weights and second quantized feature values ​​with a lower quantization bit width. Therefore, while maintaining a large numerical range and high quantization accuracy for the second quantized weights and second quantized feature values, the device effectively reduces the computational overhead and processing time of the multiplier, thereby improving the quantization efficiency of the image classification model.

[0116] It should be noted that the apparatus provided in the above embodiments is only illustrated by the division of the above functional modules when implementing its functions. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the server can be divided into different functional modules to complete all or part of the functions described above. In addition, the apparatus and method embodiments provided in the above embodiments belong to the same concept, and their specific implementation process can be found in the method embodiments, which will not be repeated here.

[0117] This application provides a computer device including a processor and a memory; the memory stores at least one piece of program code, which is executed by the processor to implement the model quantization method provided in the above-described method embodiments.

[0118] Figure 10 This is a structural block diagram of a computer device 1000 according to an embodiment of this application. In some embodiments, the computer device 1000 is a smartphone, tablet computer, computer with wireless communication capabilities, or wearable device, etc. The computer device 1000 in this application includes at least one or more of the following components: a processor 1010, a memory 1020, and at least two wireless links 1030.

[0119] In some embodiments, the processor 1010 includes one or more processing cores. The processor 1010 connects to various parts within the computer device 1000 using various interfaces and lines, and performs various functions and processes data of the computer device 1000 by running or executing program code stored in the memory 1020 and calling data stored in the memory 1020. In some embodiments, the processor 1010 is implemented using at least one hardware form of Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), or Programmable Logic Array (PLA). The processor 1010 can integrate one or more of the following: Central Processing Unit (CPU), Graphics Processing Unit (GPU), Neural-network Processing Unit (NPU), and modem. Specifically, the CPU primarily handles the operating system, user interface, and applications; the GPU is responsible for rendering and drawing the content to be displayed on the screen; the NPU is used to implement Artificial Intelligence (AI) functions; and the modem is used to handle wireless communication. It is understandable that the aforementioned modem could also be implemented as a separate chip without being integrated into the processor 1010.

[0120] In some embodiments, the processor 1010 is used to control the operating status of at least two wireless links 1030. Accordingly, the processor 1010 is a processor integrating a Wireless Fidelity (Wi-Fi) chip. This Wi-Fi chip is a chip with dual Wi-Fi processing capabilities. For example, the Wi-Fi chip is a dual-band dual-concurrent (DBDC) chip, or a dual-band simultaneous (DBS) chip, etc.

[0121] In some embodiments, the memory 1020 includes random access memory (RAM), and in some embodiments, the memory 1020 includes read-only memory (ROM). In some embodiments, the memory 1020 includes non-transitory computer-readable storage medium. The memory 1020 can be used to store program code. The memory 1020 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as touch function, sound playback function, image playback function, etc.), instructions for implementing the various method embodiments described below, etc.; the data storage area may store data created according to the use of the computer device 1000 (such as audio data, telephone book, etc.).

[0122] In some embodiments, the memory 1020 stores reception schemes for different wireless links 1030 to receive beacon frames, as well as identifiers of access nodes connected to different wireless links 1030, identifiers of wireless links 1030, etc.

[0123] The at least two wireless links 1030 are used to connect different access points (APs). They receive downlink data from the APs. These different access points can be access points within the same router or access points within different routers.

[0124] In some embodiments, the computer device 1000 further includes a display screen. The display screen is a display component used to display a user interface. In some embodiments, the display screen is a touch-enabled display screen, allowing users to perform touch operations on the display screen using fingers, styluses, or any suitable object. In some embodiments, the display screen is typically located on the front panel of the computer device 1000. In some embodiments, the display screen is designed as a full-screen, curved screen, irregularly shaped screen, dual-sided screen, or foldable screen. In some embodiments, the display screen is also designed as a combination of a full-screen and a curved screen, or a combination of an irregularly shaped screen and a curved screen, etc., which are not limited in this embodiment.

[0125] In addition, those skilled in the art will understand that the structure of the computer device 1000 shown in the above figures does not constitute a limitation on the computer device 1000. The computer device 1000 includes more or fewer components than shown, or combines certain components, or has different component arrangements. For example, the computer device 1000 also includes components such as microphones, speakers, input units, sensors, audio circuits, modules, power supplies, and Bluetooth modules, which will not be described in detail here.

[0126] This application also provides a computer-readable storage medium storing at least one piece of program code, which is loaded and executed by the processor to implement the model quantization method shown in the above embodiments.

[0127] This application also provides a chip including programmable logic circuits and / or program instructions, which, when the chip is run on a computer device, are used to implement the model quantization methods shown in the above embodiments.

[0128] This application also provides a computer program product that stores at least one piece of program code for execution by a processor to implement the model quantization method shown in the above embodiments.

[0129] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0130] Those skilled in the art will understand that all or part of the steps in the model quantization method of the above embodiments can be implemented by hardware, or by a program instructing related hardware. The program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk. The above descriptions are merely optional embodiments of this application and are not intended to limit the application. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A model quantization method, characterized in that, The method includes: For any network layer of the image classification model, when the weights of the network layer and the input feature values ​​of the input image in the network layer have a long-tailed distribution, the weights and the input feature values ​​are quantized based on the first quantization bit width to obtain the first quantized weights of the network layer and the first quantized feature values ​​of the input image in the network layer. The image classification model is used to predict the category of the input image, and the input feature values ​​are used to represent the input feature map corresponding to the input image in the network layer. When the first quantization bit width is not less than the second quantization bit width, the first quantization weight and the first quantization feature value are truncated respectively to obtain the second quantization weight and the second quantization feature value with a length of the second quantization bit width. The first bit data of the second quantization weight is the first binary number that is 1 from the high bit in the first quantization weight, and the first bit data of the second quantization feature value is the first binary number that is 1 from the high bit in the first quantization feature value. Multiply the second quantization weight and the second quantization feature value to obtain the intermediate feature value of the network layer; Based on the first quantization weight and the second quantization weight, a first truncation bit width is determined, which is used to indicate the number of low-order bits of the binary number discarded in the first quantization weight. Based on the first quantization feature value and the second quantization feature value, a second truncation bit width is determined, which is used to indicate the number of low-order bits of the binary number discarded in the first quantization feature value. Based on the sum of the first truncation bit width and the second truncation bit width, the intermediate feature value is shifted to obtain the output feature value of the network layer. The output feature value is used to represent the output feature map of the input image in the network layer.

2. The method according to claim 1, characterized in that, The first quantization bit width is p bits, the first quantization weight and the first quantization feature value are composed of p bits of binary numbers, the second quantization bit width is k bits, the second quantization weight and the second quantization feature value are composed of k bits of binary numbers, where p and k are both positive integers; The step of truncating the first quantization weight and the first quantization feature value to obtain a second quantization weight and a second quantization feature value with a length equal to the second quantization width, when the first quantization bit width is not less than the second quantization bit width, includes: When the first quantization bit width is not less than the second quantization bit width, the m-th binary number is determined from the first quantization weight, and the n-th binary number is determined from the first quantization feature value. The m-th binary number is the first 1 from the high-order bit of the first quantization weight, and the n-th binary number is the first 1 from the high-order bit of the first quantization feature value. Both m and n are positive integers. Starting from the m-th binary number, extract a binary number of length k from the first quantization weight to obtain the second quantization weight; Starting from the nth binary number, a binary number of length k is extracted from the first quantization feature value to obtain the second quantization feature value.

3. The method according to claim 2, characterized in that, The step of extracting a binary number of length k from the first quantization weight, starting from the m-th bit, to obtain the second quantization weight includes: When m is not less than k, the (m-k+1)th binary number is determined from the first quantization weight based on the mth binary number and the second quantization bit width. If the (m-k+1)th binary bit is 0, set the (m-k+1)th binary bit to 1; The second quantization weight is obtained by extracting the binary number from the m-th bit to the (m-k+1)-th bit from the first quantization weight, where the (m-k+1)-th bit is the last bit of the second quantization weight.

4. The method according to claim 3, characterized in that, The method further includes: When m is less than k, starting from the m-th binary number, m bits of binary number are extracted from the first quantization weight; A decimal point is fixed at the end of the m-bit binary number; Add km zeros after the decimal point to obtain the second quantization weight.

5. The method according to claim 2, characterized in that, The step of extracting a binary number of length k from the first quantization feature value, starting from the nth binary bit, to obtain the second quantization feature value includes: When n is not less than k, the (n-k+1)th binary bit is determined from the first quantization feature value based on the nth binary bit and the second quantization bit width; If the (n-k+1)th binary bit is 0, set the (n-k+1)th binary bit to 1; The second quantization feature value is obtained by extracting the binary numbers from the nth bit to the (n-k+1)th bit from the first quantization feature value, wherein the (n-k+1)th bit is the last bit of the second quantization feature value.

6. The method according to claim 5, characterized in that, The method further includes: When n is less than k, starting from the nth binary number, n binary numbers are extracted from the first quantization feature value; The last decimal point of the n-bit binary number is fixed; Add kn zeros after the decimal point to obtain the second quantization feature value.

7. The method according to claim 1, characterized in that, The method further includes: When the first quantization bit width is not less than the second quantization bit width, the binary number starting from the high bit and first 1 is determined from the first quantization weight and the first quantization feature value, respectively. If the number of bits of the first binary number that is 1 in the first quantization weight is less than the second quantization bit width, starting from the last bit of the first quantization weight, a binary number with a length equal to the second quantization bit width is extracted to obtain the second quantization weight, and the last bit of the second quantization weight is the last bit of the first quantization weight. If the number of bits of the first binary number that is 1 in the first quantization feature value is less than the second quantization bit width, starting from the last bit of the first quantization feature value, a binary number with a length equal to the second quantization bit width is extracted to obtain the second quantization feature value, and the last bit of the second quantization feature value is the last bit of the first quantization feature value.

8. A model quantization device, characterized in that, The device includes: The quantization module is used to quantize the weights and input feature values ​​of the input image in any network layer of the image classification model, based on a first quantization bit width, when the weights of the network layer and the input feature values ​​of the input image in the network layer have a long-tailed distribution, to obtain the first quantized weights of the network layer and the first quantized feature values ​​of the input image in the network layer. The image classification model is used to predict the category of the input image, and the input feature values ​​are used to represent the input feature map corresponding to the input image in the network layer. The truncation module is used to truncate the first quantization weight and the first quantization feature value respectively when the first quantization bit width is not less than the second quantization bit width, to obtain a second quantization weight and a second quantization feature value with a length of the second quantization bit width. The first bit data of the second quantization weight is the first binary number with a 1 starting from the high bit in the first quantization weight, and the first bit data of the second quantization feature value is the first binary number with a 1 starting from the high bit in the first quantization feature value. A weighting module is used to multiply the second quantization weight and the second quantization feature value to obtain the intermediate feature value of the network layer; based on the first quantization weight and the second quantization weight, a first truncation bit width is determined, which indicates the number of low-order binary numbers discarded in the first quantization weight; based on the first quantization feature value and the second quantization feature value, a second truncation bit width is determined, which indicates the number of low-order binary numbers discarded in the first quantization feature value; based on the sum of the first truncation bit width and the second truncation bit width, the intermediate feature value is shifted to obtain the output feature value of the network layer, which represents the output feature map corresponding to the input image in the network layer.

9. A computer device, characterized in that, The computer device includes a processor and a memory; the memory stores at least one piece of program code, which is executed by the processor to implement the model quantization method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The storage medium stores at least one piece of program code, which is executed by a processor to implement the model quantization method as described in any one of claims 1 to 7.

11. A chip, characterized in that, The chip includes programmable logic circuits and / or program instructions, which, when the chip is run on a computer device, are used to implement the model quantization method as described in any one of claims 1 to 7.

12. A computer program product, comprising a computer program, characterized in that, The computer program product stores at least one piece of program code, which is executed by a processor to implement the model quantization method as described in any one of claims 1 to 7.

Citation Information

Patent Citations

  • Image classification method, device and equipment and computer readable storage medium

    CN110782021A

  • Data processing method, electronic device, storage medium and program product

    CN115393633A