Model quantification method and device, computer equipment and storage medium

Abstract

The invention provides a model quantification method and device, computer equipment and a storage medium, and belongs to the technical field of computers. The method comprises the following steps: for any network layer of an image classification model, under the condition that the weight of the network layer and an input feature value of an input image in the network layer are distributed in a long tail manner, respectively quantizing the weight and the input feature value based on a first quantization bit width to obtain a first quantization weight and a first quantization feature value of the network layer; under the condition that the first quantization bit width is not smaller than the second quantization bit width, the first quantization weight and the first quantization feature value are cut off, and a second quantization weight and a second quantization feature value with the length being the second quantization bit width are obtained; and weighting the second quantization weight and the second quantization feature value to obtain an output feature value of the network layer. According to the scheme, the calculation overhead of the multiplier can be effectively reduced under the condition that the large numerical value representation range and the high quantization precision of the second quantization weight and the second quantization feature value are kept.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a model quantization method, device, computer equipment and storage medium. Background Art

[0002] With the continuous development of artificial intelligence technology, neural network models are widely used in the fields of image noise reduction, image quality enhancement, image recognition and segmentation. In order to improve the operating efficiency of the neural network model, the neural network model is usually quantized, and then the quantized neural network model is deployed on the terminal. However, in the process of model quantization, since the weights and input eigenvalues ​​of some network layers in the neural network model have a long-tail distribution, directly quantizing the weights and input eigenvalues ​​of the network layer will increase the quantization interval and cause a decrease in quantization accuracy. Therefore, how to reduce the impact of long-tail distribution on model quantization is a technical problem that needs to be solved.

[0003] In the related technology, in the process of model quantization, more quantization bit widths are usually allocated to network layers with long-tail distribution (such as increasing from 8 bits to 16 bits). By quantizing the weights and input eigenvalues ​​of the network layer with high bit width, the quantization accuracy and numerical representation range of the weights and input eigenvalues ​​of the network layer can be improved, thereby improving the quality of the image output by the neural network model.

[0004] However, after the weights and input eigenvalues ​​of the network layer are quantized, they still need to be processed by a multiplier to obtain the output eigenvalues ​​of the network layer. While increasing the quantization bit width, the above solution will increase the operation time and computational overhead of the multiplier, reducing the operating efficiency of the neural network model. Summary of the invention

[0005] The embodiments of the present application provide a model quantization method, apparatus, computer equipment and storage medium, which can effectively reduce the computational overhead and operation time of the multiplier while maintaining a large numerical representation range and high quantization accuracy of the second quantization weight and the second quantization feature value, thereby improving the operating efficiency of the image classification model. The technical solution is as follows:

[0006] According to one aspect of an embodiment of the present application, a model quantization method is provided, the method comprising:

[0007] For any network layer of the image classification model, when the weight of the network layer and the input feature value of the input image at the network layer present a long-tail distribution, based on the first quantization bit width, the weight and the input feature value are quantized respectively to obtain a first quantization weight of the network layer and a first quantization feature value of the input image at the network layer, the image classification model is used to predict the category of the input image, and the input feature value is used to represent the input feature map corresponding to the input image at the network layer;

[0008] In the case where the first quantization bit width is not less than the second quantization bit width, the first quantization weight and the first quantization feature value are truncated respectively to obtain a second quantization weight and a second quantization feature value with a length of the second quantization bit width, the first bit data of the second quantization weight is the first binary number that is 1 starting from the high bit in the first quantization weight, and the first bit data of the second quantization feature value is the first binary number that is 1 starting from the high bit in the first quantization feature value;

[0009] The second quantization weight and the second quantization feature value are weighted to obtain an output feature value of the network layer, where the output feature value is used to represent an output feature map corresponding to the input image at the network layer.

[0010] According to another aspect of an embodiment of the present application, a model quantization device is provided, the device comprising:

[0011] A quantization module, for any network layer of an image classification model, when the weight of the network layer and the input feature value of the input image at the network layer are in a long-tail distribution, quantize the weight and the input feature value based on a first quantization bit width, respectively, to obtain a first quantization weight of the network layer and a first quantization feature value of the input image at the network layer, the image classification model is used to predict the category of the input image, and the input feature value is used to represent an input feature map corresponding to the input image at the network layer;

[0012] A truncation module, configured to, when the first quantization bit width is not less than the second quantization bit width, respectively truncate the first quantization weight and the first quantization feature value to obtain a second quantization weight and a second quantization feature value having a length of the second quantization bit width, wherein the first bit data of the second quantization weight is the first binary number that is 1 starting from a high bit in the first quantization weight, and the first bit data of the second quantization feature value is the first binary number that is 1 starting from a high bit in the first quantization feature value;

[0013] A weighting module is used to weight the second quantization weight and the second quantization feature value to obtain an output feature value of the network layer, and the output feature value is used to represent the output feature map corresponding to the input image in the network layer.

[0014] In some embodiments, the first quantization bit width is p bits, the first quantization weight and the first quantization feature value are composed of p-bit binary numbers, the second quantization bit width is k bits, the second quantization weight and the second quantization feature value are composed of k-bit binary numbers, and p and k are both positive integers;

[0015] The truncation module comprises:

[0016] A first determining unit is configured to determine, when the first quantization bit width is not less than the second quantization bit width, an m-th binary number from the first quantization weight, and determine an n-th binary number from the first quantization feature value, wherein the m-th binary number is the first binary number that is 1 starting from a high bit in the first quantization weight, and the n-th binary number is the first binary number that is 1 starting from a high bit in the first quantization feature value, and both m and n are positive integers;

[0017] A first truncation unit, configured to truncate a binary number having a length of k from the first quantization weight, starting from the m-th binary number, to obtain the second quantization weight;

[0018] The second truncation unit is used to truncate a binary number with a length of k from the first quantitative feature value starting from the n-th binary number to obtain the second quantitative feature value.

[0019] In some embodiments, the first truncation unit is used to determine the m-k+1th binary number from the first quantization weight based on the m-th binary number and the second quantization bit width when m is not less than k; when the m-k+1th binary number is 0, set the m-k+1th binary number to 1; truncate the binary numbers from the m-th to the m-k+1th bits from the first quantization weight to obtain the second quantization weight, and the m-k+1th binary number is the last bit data of the second quantization weight.

[0020] In some embodiments, the first truncation unit is further used to, when m is less than k, truncate m-bit binary numbers from the first quantization weight, starting from the m-th binary number; fix a decimal point at the end of the m-bit binary number; and add km zeros after the decimal point to obtain the second quantization weight.

[0021] In some embodiments, the second truncation unit is used to determine the n-k+1th binary number from the first quantization feature value based on the n-k+1th binary number and the second quantization bit width when n is not less than k; when the n-k+1th binary number is 0, set the n-k+1th binary number to 1; and truncate the binary numbers from the n-k+1th to the n-k+1th from the first quantization feature value to obtain the second quantization feature value, the n-k+1th binary number being the last bit data of the second quantization feature value.

[0022] In some embodiments, the second truncation unit is further used to, when n is less than k, truncate n binary digits from the first quantitative feature value starting from the n-th binary digit; fix a decimal point at the end of the n-bit binary number; and add kn zeros after the decimal point to obtain the second quantitative feature value.

[0023] In some embodiments, the truncation module is also used to determine the first binary number that is 1 starting from the high bit from the first quantization weight and the first quantization characteristic value respectively when the first quantization bit width is not less than the second quantization bit width; when the number of bits of the first binary number that is 1 of the first quantization weight is less than the second quantization bit width, starting from the last bit of the first quantization weight, the binary number with a length of the second quantization bit width is truncated to obtain the second quantization weight, and the last bit data of the second quantization weight is the last bit data of the first quantization weight; when the number of bits of the first binary number that is 1 of the first quantization characteristic value is less than the second quantization bit width, starting from the last bit of the first quantization characteristic value, the binary number with a length of the second quantization bit width is truncated to obtain the second quantization characteristic value, and the last bit data of the second quantization characteristic value is the last bit data of the first quantization characteristic value.

[0024] In some embodiments, the weighting module comprises:

[0025] a calculation unit, configured to multiply the second quantization weight and the second quantization eigenvalue to obtain an intermediate eigenvalue of the network layer;

[0026] A second determining unit, configured to determine a first truncation bit width based on the first quantization weight and the second quantization weight, wherein the first truncation bit width is used to indicate the number of binary numbers whose low bits in the first quantization weight are discarded;

[0027] A third determining unit, configured to determine a second truncation bit width based on the first quantization feature value and the second quantization feature value, wherein the second truncation bit width is used to indicate the number of binary numbers whose low bits in the first quantization feature value are discarded;

[0028] A shifting unit is used to shift the intermediate eigenvalue based on the sum of the first truncation bit width and the second truncation bit width to obtain the output eigenvalue.

[0029] According to another aspect of an embodiment of the present application, a computer device is provided, comprising a processor and a memory; the memory stores at least one program code, and the at least one program code is used to be executed by the processor to implement the model quantization method as described in the above aspects.

[0030] According to another aspect of an embodiment of the present application, a chip is provided, which includes a programmable logic circuit and / or program instructions. When the chip runs on a terminal, it is used to implement the model quantization method described in the above aspects.

[0031] According to another aspect of an embodiment of the present application, a computer-readable storage medium is provided, wherein the storage medium stores at least one program code, and the at least one program code is used to be executed by a processor to implement the model quantization method as described in the above aspects.

[0032] According to another aspect of an embodiment of the present application, a computer program product is provided. The computer program product stores at least one program code, and the at least one program code is used to be executed by a processor to implement the model quantization method described in the above aspect.

[0033] The embodiment of the present application provides a model quantization method, in which, when the weights and input feature values ​​of the network layer of the image classification model are in a long-tail distribution, the weights and input feature values ​​of the network layer are quantized based on the first quantization bit width, so that the first quantization weight and the first quantization feature value with a large numerical range and high quantization accuracy can be obtained. Then, based on the second quantization bit width lower than the first quantization bit width, the first quantization weight and the first quantization feature value are truncated, and the second quantization weight and the second quantization feature value with a lower quantization bit width can be obtained from the first quantization weight and the first quantization feature value, so that the computational overhead and operation time of the multiplier can be effectively reduced while maintaining the larger numerical representation range and higher quantization accuracy of the second quantization weight and the second quantization feature value, thereby improving the operating efficiency of the image classification model. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0035] Figure 1It is a schematic diagram of an implementation environment of a model quantization method provided in an embodiment of the present application;

[0036] Figure 2 is a flow chart of a model quantization method provided in an embodiment of the present application;

[0037] Figure 3 is a flow chart of another model quantization method provided in an embodiment of the present application;

[0038] Figure 4 is a schematic diagram of a long-tail distribution provided in an embodiment of the present application;

[0039] Figure 5 is a schematic diagram of a DRUM method provided in an embodiment of the present application;

[0040] Figure 6 is a schematic diagram of a first truncated bit width provided in an embodiment of the present application;

[0041] Figure 7 It is an overall framework diagram of a model quantization method provided in an embodiment of the present application;

[0042] Figure 8 is a structural schematic diagram of a model quantization device provided in an embodiment of the present application;

[0043] Fig. 9 is a structural schematic diagram of another model quantization device provided in an embodiment of the present application;

[0044] Fig.10 It is a structural block diagram of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0045] In order to make the objectives, technical solutions and advantages of the present application clearer, the implementation methods of the present application will be further described in detail below with reference to the accompanying drawings.

[0046] In this article, "at least one" means one or more, and "plurality" means two or more. "And / or" describes the association relationship of the associated objects, indicating that three relationships can exist. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the associated objects are in an "or" relationship.

[0047] It should be noted that the information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data used for analysis, stored data, displayed data, etc.) and signals involved in this application are all authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions. For example, the image classification model and input images involved in this application are obtained with full authorization.

[0048] The model quantization method provided in the embodiment of the present application can be executed by a computer device. In some embodiments, the computer device is a terminal or a server. The following takes the computer device as an example to introduce the implementation environment of the model quantization method provided in the embodiment of the present application. Figure 1 Schematic diagram of an implementation environment of a model quantization method provided in an embodiment of the present application. Figure 1 The implementation environment includes a terminal 101 and a server 102. The terminal 101 can be directly or indirectly connected to the server 102 through wired or wireless communication, which is not limited in this application.

[0049] In some embodiments, the terminal 101 can be a mobile phone, a desktop computer, a laptop computer, a tablet computer, an intelligent voice interaction device, a vehicle-mounted terminal, and other types of terminals. An application supporting image classification is installed and run on the terminal 101. The terminal 101 can quantify the image classification model through the application. Schematically, the user can input the image to be processed into the image classification model through the application. In the process of the image classification model processing the image, the terminal 101 can quantify the weights and input feature values ​​of multiple network layers in the image classification model, thereby improving the operating efficiency of the image classification model.

[0050] In some embodiments, the server 102 is an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms. The server 102 can train the image classification model and send the trained image classification model to the terminal 101, which quantifies the image classification model.

[0051] In some embodiments, the server 102 undertakes the main computing work and the terminal 101 undertakes the secondary computing work; or, the server 102 undertakes the secondary computing work and the terminal 101 undertakes the main computing work; or, the server 102 and the terminal 101 adopt a distributed computing architecture to perform collaborative computing.

[0052] It should be noted that the model quantization method provided in the embodiment of the present application can not only quantify the image classification model used in the field of image classification, but also quantify the image processing model used in other image processing fields such as image denoising, image quality enhancement, image recognition, and image segmentation. The following takes the image classification model as an example to introduce the model quantization method provided in the embodiment of the present application.

[0053] Figure 2 is a flow chart of a model quantization method provided in an embodiment of the present application. The method is executed by a terminal, see Figure 2 , the method comprising:

[0054] 201. For any network layer of the image classification model, when the weight of the network layer and the input feature value of the input image in the network layer present a long-tail distribution, the terminal quantizes the weight and the input feature value based on a first quantization bit width, respectively, to obtain a first quantization weight of the network layer and a first quantization feature value of the input image in the network layer. The image classification model is used to predict the category of the input image, and the input feature value is used to represent an input feature map corresponding to the input image in the network layer.

[0055] In an embodiment of the present application, the image classification model is a trained neural network model. The image classification model includes multiple network layers, each network layer includes multiple nodes, and the weight of the network layer is used to reflect the connection relationship between the multiple nodes. In the process of the image classification model processing the input image, for the first network layer in the image classification model, the network layer can extract features of the input image to obtain multiple feature values ​​of the input image. The multiple feature values ​​can be used as input feature values ​​of the next network layer to represent the input feature map corresponding to the input image in the next network layer. Since the weights and input feature values ​​of the network layer are both high-precision floating-point data, in order to improve the operating efficiency of the image classification model, the terminal can quantize the weights and input feature values ​​of the network layer, and convert the data types of the weights and input feature values ​​from floating point numbers to fixed point numbers. In the process of quantizing the weights and input feature values, when the weights and input feature values ​​of the network layer are distributed in a long-tailed manner, the data distribution of the weights and input feature values ​​is uneven. If low-bit width quantization is performed directly, the quantization interval of the weights and input feature values ​​will be increased, resulting in a decrease in quantization accuracy. Therefore, in the present application, the terminal can perform high-bit-width quantization on the weights and input eigenvalues ​​based on the first quantization bit width, respectively, to obtain the first quantization weight and the first quantization eigenvalue of the network layer. Among them, the first quantization weight and the first quantization eigenvalue are fixed-point data with a length of the first quantization bit width, and the fixed-point data is composed of multiple binary numbers. Taking the first quantization bit width as 16 bits, and the weights and input eigenvalues ​​as 32-bit floating-point numbers as an example, the terminal quantizes the weights and input eigenvalues ​​based on the first quantization bit width, respectively, and can convert the weights and input eigenvalues ​​from 32-bit floating-point numbers to 16-bit fixed-point numbers, thereby obtaining the first quantization weight and the second quantization eigenvalue.

[0056] 202. When the first quantization bit width is not less than the second quantization bit width, the terminal truncates the first quantization weight and the first quantization characteristic value respectively to obtain a second quantization weight and a second quantization characteristic value having a length of the second quantization bit width, wherein the first datum of the second quantization weight is the first binary number that is 1 starting from the high bit in the first quantization weight, and the first datum of the second quantization characteristic value is the first binary number that is 1 starting from the high bit in the first quantization characteristic value.

[0057] In an embodiment of the present application, the terminal converts the weight and input feature value of the network layer from a 32-bit floating point number to an 8-bit fixed point number, so the second quantization bit width is set to 8 bits. When the first quantization bit width is not less than the second quantization bit width, it means that the length of the first quantization weight and the first quantization feature value obtained by quantization based on the first quantization bit width is greater than the second quantization bit width. Therefore, in order to obtain the weight and input feature value of the second quantization bit width, the terminal can obtain the second quantization weight and the second quantization feature value of the second quantization bit width by truncating the first quantization weight and the first quantization feature value. Compared with the first quantization weight and the first quantization feature value, the second quantization weight can represent the connection relationship of multiple nodes in the network layer with less data, and the second quantization feature value can represent the input feature map corresponding to the input image in the network layer with less data. Among them, the first quantization weight and the first quantization feature value are composed of multiple binary numbers arranged from left to right, and the leftmost binary number in the first quantization weight and the first quantization feature value is the highest binary number. The first bit data of the second quantization weight is the first binary number of 1 starting from the high bit in the first quantization weight, and the first bit data of the second quantization eigenvalue is the first binary number of 1 starting from the high bit in the first quantization eigenvalue. By discarding the low-precision data at the end of the first quantization weight and the first quantization eigenvalue, the terminal can reduce the quantization bit width of the weight and the input eigenvalue while ensuring a larger numerical representation range and higher quantization accuracy.

[0058] 203. The terminal weights the second quantization weight and the second quantization eigenvalue to obtain an output eigenvalue of the network layer, where the output eigenvalue is used to represent an output feature map corresponding to the input image at the network layer.

[0059] In an embodiment of the present application, after extracting the second quantization weight and the second quantization feature value having a length of the second quantization bit width from the first quantization weight and the first quantization feature value, the terminal can process the second quantization feature value based on the second quantization weight to obtain the output feature value of the network layer. The output feature value is used to represent the output feature map corresponding to the input image at the network layer. Compared to the process of inputting the first quantization weight and the first quantization feature value into the multiplier to obtain the output feature value, the terminal uses a multiplier to process the second quantization weight and the second quantization feature value having a shorter length to obtain the output feature value, which can effectively reduce the computational overhead and operation time of the multiplier and speed up the operation efficiency of the image classification model.

[0060] The embodiment of the present application provides a model quantization method, in which, when the weights and input feature values ​​of the network layer of the image classification model are in a long-tail distribution, the weights and input feature values ​​of the network layer are quantized based on the first quantization bit width, so that the first quantization weight and the first quantization feature value with a large numerical range and high quantization accuracy can be obtained. Then, based on the second quantization bit width lower than the first quantization bit width, the first quantization weight and the first quantization feature value are truncated, and the second quantization weight and the second quantization feature value with a lower quantization bit width can be obtained from the first quantization weight and the first quantization feature value, so that the computational overhead and operation time of the multiplier can be effectively reduced while maintaining the larger numerical representation range and higher quantization accuracy of the second quantization weight and the second quantization feature value, thereby improving the operating efficiency of the image classification model.

[0061] Figure 3 is a flowchart of another model quantization method provided in an embodiment of the present application. The method is executed by a terminal, see Figure 3 , the method comprising:

[0062] 301. For any network layer of the image classification model, when the weight of the network layer and the input feature value of the input image in the network layer present a long-tail distribution, the terminal quantizes the weight and the input feature value based on a first quantization bit width, respectively, to obtain a first quantization weight of the network layer and a first quantization feature value of the input image in the network layer. The image classification model is used to predict the category of the input image, and the input feature value is used to represent an input feature map corresponding to the input image in the network layer.

[0063] In the embodiment of the present application, the image classification model is a trained neural network model. The image classification model includes multiple network layers, each network layer includes multiple nodes, and the weight of the network layer is used to reflect the connection relationship between the multiple nodes. Among them, the image classification model is used to predict the category of the input image.

[0064] In the process of the image classification model processing the input image, for the first network layer in the image classification model, the network layer can extract features of the input image and obtain multiple feature values ​​of the input image. The multiple feature values ​​can be used as the input feature values ​​of the next network layer to represent the input feature map corresponding to the input image in the next network layer, that is, the input feature value of a network layer is the output feature value of the previous network layer of the network layer, and the output feature value of a network layer is the input feature value of the next network layer of the network layer. Since the weights and input feature values ​​of the network layer are both high-precision floating-point data, in order to improve the operating efficiency of the image classification model, the terminal can quantize the weights and input feature values ​​of the network layer to convert the data types of the weights and input feature values ​​from floating-point numbers to fixed-point numbers. In the process of quantizing the weights and input feature values, when the weights and input feature values ​​of the network layer are distributed in a long-tailed manner, the data distribution of the weights and input feature values ​​is uneven. If low-bit-width quantization is performed directly, the quantization interval of the weights and input feature values ​​is increased, resulting in a decrease in quantization accuracy. Therefore, in the present application, the terminal can first quantize the weight and the input feature value with a high bit width based on the first quantization bit width, respectively, to obtain the first quantization weight and the first quantization feature value of the network layer. Among them, the first quantization weight and the first quantization feature value are fixed-point data with a length of the first quantization bit width, and the fixed-point data is composed of a multi-bit binary number. The first quantization bit width can be 16 bits, 12 bits, 10 bits, etc., and the embodiments of the present application are not limited to this.

[0065] For example, Figure 4 Schematic diagram of a long-tail distribution provided in the embodiment of the present application. Figure 4 As shown in Figure 1, long-tail distribution is a data distribution with uneven data distribution, in which a small number of categories at the head of the curve contain a larger number of weights, while most of the categories at the tail of the curve contain a smaller number of weights.

[0066] 302. When the first quantization bit width is not less than the second quantization bit width, the terminal determines the m-th binary number from the first quantization weight and determines the n-th binary number from the first quantization characteristic value, the m-th binary number is the first binary number that is 1 starting from the high bit in the first quantization weight, the n-th binary number is the first binary number that is 1 starting from the high bit in the first quantization characteristic value, and both m and n are positive integers.

[0067] In the embodiment of the present application, when the first quantization bit width is not less than the second quantization bit width, it means that the length of the first quantization weight and the first quantization feature value obtained by quantization based on the first quantization bit width is greater than the second quantization bit width. In the process of truncating the first quantization weight and the first quantization feature value, the terminal can determine the first binary number that is 1 from the high position from the first quantization weight, and use it as the first data of the second quantization weight. Similarly, the terminal can determine the first binary number that is 1 from the high position from the first quantization feature value, and use it as the first data of the second quantization feature value. Among them, the first quantization bit width is p bits, the first quantization weight and the first quantization feature value are composed of p-bit binary numbers arranged from left to right, and the leftmost binary number in the first quantization weight and the first quantization feature value is the highest binary number. The first binary number that is 1 from the high position in the first quantization weight is the m-th binary number in the p-bit binary number, and the first binary number that is 1 from the high position in the first quantization feature value is the n-th binary number in the p-bit binary number, and m and n may be the same or different.

[0068] For example, the first quantization weight is "0001011110010". Starting from the left, the first binary number that is 1 is searched in the first quantization weight. The fourth binary number from left to right is the first binary number that is 1 in the first quantization weight starting from the high bit. In this case, m = 10. The first quantization feature value is "0101011110010". Starting from the left, the first binary number that is 1 is searched in the first quantization feature value. The second binary number from left to right is the first binary number that is 1 in the first quantization feature value starting from the high bit. In this case, n = 12.

[0069] 303. The terminal starts from the mth binary number and extracts a binary number of length k from the first quantization weight to obtain a second quantization weight, wherein the first bit data of the second quantization weight is the first binary number of 1 starting from the high bit in the first quantization weight.

[0070] In an embodiment of the present application, since the first bit data of the second quantization weight is the first binary number that is 1 from the high bit in the first quantization weight, and the first binary number that is 1 from the high bit is the mth binary number of the first quantization weight. Therefore, the terminal can start from the mth binary number, intercept a binary number of length k from the p-bit binary number in the first quantization weight, and can obtain a second quantization weight with a length of the second quantization bit width. Compared with the first quantization weight, the second quantization weight can represent the connection relationship of multiple nodes in the network layer with a smaller amount of data. Among them, the second quantization bit width is k bits, and the second quantization weight consists of k-bit binary numbers. Since the terminal usually converts the weight of the network layer from a 32-bit floating point number to an 8-bit fixed point number, the second quantization bit width is usually set to 8 bits, that is, k=8.

[0071] In some embodiments, the terminal can start from the mth binary number of the first quantization weight and intercept the binary number of length k to obtain the second quantization weight. Accordingly, when m is not less than k, the terminal determines the m-k+1th binary number from the first quantization weight based on the mth binary number and the second quantization bit width; when the m-k+1th binary number is 0, the m-k+1th binary number is set to 1; intercept the binary numbers from the mth to the m-k+1th from the first quantization weight to obtain the second quantization weight, and the m-k+1th binary number is the last bit of the second quantization weight. Among them, when m is not less than k, it indicates that the number of bits of the first binary number that is 1 from the high bit in the first quantization weight is not less than the second quantization bit width. Therefore, the terminal can determine the m-k+1th binary number from the first quantization weight based on the mth binary number and the second quantization bit width. Since the m-th binary number and the m-k+1-th binary number are the first data and the last data of the second quantization weight respectively, the terminal intercepts the binary numbers from the m-th to the m-k+1-th from the first quantization weight to obtain the second quantization weight of length k. Among them, the first data represents the highest bit data in the second quantization weight, and the last data represents the lowest bit data of the second quantization weight. When the m-k+1-th binary number is 0, the terminal can achieve unbiased approximate low-bit width quantization while maintaining a higher numerical range by setting the m-k+1-th binary number to 1. By discarding the low-precision data from the 1st to the mk-th in the first quantization weight, the quantization bit width of the weight can be reduced while ensuring a larger numerical representation range and higher quantization accuracy.

[0072] For example, Figure 5 DRUM (Dynamic Range Unbiased Multiplier) method provided in the embodiment of the present application is a schematic diagram. Figure 5 As shown, taking the first quantization weight as an unsigned fixed-point integer as an example, the first quantization weight is "0000001011110010", and the second quantization bit width is 8 bits. First, start from the high bit of the first quantization weight to find the first binary number that is 1, and the first binary number that is 1 is the 10th binary number in the first quantization weight. The terminal takes the 10th binary number as the starting point, and determines the 3rd binary number from high to low. Since the 3rd binary number is 0, the 3rd binary number is set to 1. Then the terminal intercepts the binary numbers from the 10th to the 3rd bit from the first quantization weight to obtain the 8-bit binary number "10111101". The terminal determines this 8-bit binary number as the second quantization weight.

[0073] In some embodiments, when m is less than k, the terminal starts from the m-th binary digit and intercepts m binary digits from the first quantization weight. Then, a decimal point is fixed at the end of the m binary digits, and k - m zeros are appended after the decimal point to obtain the second quantization weight. When m is less than k, it indicates that the number of bits of the first binary digit that is 1 starting from the high bit in the first quantization weight is less than the second quantization bit width. To obtain the second quantization weight with a length of k, the terminal can first intercept the binary digits from the m-th bit to the 1st bit from the first quantization weight to obtain m binary digits. By fixing a decimal point at the end of the m binary digits and appending k - m zeros after the decimal point, the terminal can obtain the second quantization weight that meets the second quantization bit width without changing the first quantization weight.

[0074] For example, taking the first quantization weight as "0000000001110010" and k = 8 as an example, the first binary digit that is 1 starting from the high bit in the first quantization weight is the 7th binary digit, that is, m = 7 < k. The terminal starts from the 7th binary digit of the first quantization weight and intercepts 7 binary digits to obtain "1110010". The terminal fixes a decimal point at the end of the 7 binary digits to obtain "1110010.". Then, a zero is appended after the decimal point to obtain "1110010.0", that is, the second quantization weight is "1110010.0".

[0075] 304. The terminal starts from the n-th binary digit and intercepts a binary number with a length of k from the first quantization eigenvalue to obtain a second quantization eigenvalue, and the first data bit of the second quantization eigenvalue is the first binary digit that is 1 starting from the high bit in the first quantization eigenvalue.

[0076] In the embodiments of the present application, since the first data bit of the second quantization eigenvalue is the first binary digit that is 1 starting from the high bit in the first quantization eigenvalue, and the first binary digit that is 1 starting from the high bit is the n-th binary digit of the first quantization eigenvalue. Therefore, the terminal can start from the n-th binary digit and intercept a binary number with a length of k from the p binary digits in the first quantization eigenvalue, and can obtain the second quantization eigenvalue with a length of the second quantization bit width. Compared with the first quantization eigenvalue, the second quantization eigenvalue can represent the input feature map corresponding to the input image in the network layer with less data volume. Among them, the second quantization bit width is k bits, and the second quantization eigenvalue is composed of k binary digits. Since the terminal usually converts the input eigenvalue of the network layer from 32-bit floating-point numbers to 8-bit fixed-point numbers, the second quantization bit width is usually set to 8 bits, that is, k = 8.

[0077] In some embodiments, the terminal can start from the nth binary number of the first quantization feature value and intercept the binary number of length k to obtain the second quantization feature value. Accordingly, when n is not less than k, the terminal determines the n-k+1th binary number from the first quantization feature value based on the nth binary number and the second quantization bit width; when the n-k+1th binary number is 0, the n-k+1th binary number is set to 1; intercept the binary numbers from the nth to the n-k+1th from the first quantization feature value to obtain the second quantization feature value, and the n-k+1th binary number is the last bit data of the second quantization feature value. Among them, when n is not less than k, it indicates that the number of bits of the first binary number that is 1 from the high bit in the first quantization feature value is not less than the second quantization bit width. Therefore, the terminal can determine the n-k+1th binary number from the first quantization feature value based on the nth binary number and the second quantization bit width. Since the n-th binary number and the n-k+1-th binary number are the first data and the last data of the second quantized eigenvalue, respectively, the terminal intercepts the binary numbers from the n-th to the n-k+1-th from the first quantized eigenvalue to obtain the second quantized eigenvalue of length k. Among them, the first data represents the highest bit data in the second quantized eigenvalue, and the last data represents the lowest bit data in the second quantized eigenvalue. When the n-k+1-th binary number is 0, the terminal can achieve unbiased approximate low-bit width quantization while maintaining a higher numerical range by setting the n-k+1-th binary number to 1. By discarding the low-precision data from the 1st to the n-k-th in the first quantized eigenvalue, the quantization bit width of the eigenvalue can be reduced while ensuring a larger numerical representation range and higher quantization accuracy.

[0078] In some embodiments, when n is less than k, the terminal starts from the nth binary number and intercepts n-bit binary numbers from the first quantization feature value; fixes the decimal point at the end of the n-bit binary number; and fills kn zeros after the decimal point to obtain a second quantization feature value. Wherein, when n is less than k, it indicates that the number of bits of the first binary number that is 1 starting from the high bit in the first quantization feature value is less than the second quantization bit width. In order to obtain a second quantization feature value of length k, the terminal can first intercept the binary numbers from the nth bit to the 1st bit from the first quantization feature value to obtain an n-bit binary number. By fixing the decimal point at the end of the n-bit binary number and filling kn zeros after the decimal point, the terminal can obtain a second quantization feature value that satisfies the second quantization bit width without changing the first quantization feature value.

[0079] In some embodiments, the terminal can also start from the last bit of the first quantization weight and the first quantization feature value, intercept the binary number with a length of the second quantization bit width, and obtain the second quantization weight and the second quantization feature value. Correspondingly, when the first quantization bit width is not less than the second quantization bit width, the terminal determines the first binary number that is 1 from the high bit from the first quantization weight and the first quantization feature value respectively; when the number of bits of the first binary number that is 1 in the first quantization weight is less than the second quantization bit width, starting from the last bit of the first quantization weight, intercept the binary number with a length of the second quantization bit width, and obtain the second quantization weight, and the last bit data of the second quantization weight is the last bit data of the first quantization weight; when the number of bits of the first binary number that is 1 in the first quantization feature value is less than the second quantization bit width, starting from the last bit of the first quantization feature value, intercept the binary number with a length of the second quantization bit width, and obtain the second quantization feature value, and the last bit data of the second quantization feature value is the last bit data of the first quantization feature value. Wherein, when the first quantization bit width is not less than the second quantization bit width, it means that the length of the first quantization weight and the first quantization feature value obtained based on the first quantization bit width is greater than the second quantization bit width. In the process of truncating the first quantization weight and the first quantization characteristic value, the terminal can determine the first binary number that is 1 starting from the high bit from the first quantization weight and the first quantization characteristic value, respectively. In the case where the number of bits of the first binary number that is 1 is less than the second quantization bit width, if truncation starts from the first binary number that is 1, the second quantization weight and the second quantization characteristic value with a length of the second quantization bit width cannot be obtained from the first quantization weight and the first quantization characteristic value. Therefore, the terminal can use the last bit data in the first quantization weight and the first quantization characteristic value as the last bit data of the second quantization weight and the second quantization characteristic value, respectively. Then, starting from the last bit, binary numbers with a length of the second quantization bit width are truncated from the first quantization weight and the first quantization characteristic value, respectively, to obtain the second quantization weight and the second quantization characteristic value.

[0080] For example, taking the first quantization weight as "0000000001110010" and the second quantization bit width as 8 bits, the first binary number that is 1 from the high bit in the first quantization weight is the 7th binary number, that is, the number of bits of the first binary number that is 1 is less than the second quantization bit width. Then the terminal can start from the last bit of the first quantization weight, intercept the binary number with a length of the second quantization bit width, and obtain "01110010", that is, the second quantization weight is "01110010".

[0081] 305. The terminal multiplies the second quantization weight and the second quantization eigenvalue to obtain an intermediate eigenvalue of the network layer.

[0082] In an embodiment of the present application, the terminal inputs a second quantization weight and a second quantization eigenvalue having a length of the second quantization bit width into a multiplier, and the multiplier multiplies the multi-bit binary numbers in the second quantization weight and the second quantization eigenvalue to obtain an intermediate eigenvalue of the network layer.

[0083] For example, the multiplier is an 8-bit multiplier, the first quantization bit width is 8 bits, the second quantization weight is "10111101", and the second quantization eigenvalue is "10010011", then the intermediate eigenvalue obtained by multiplication is "110110010000111".

[0084] 306. The terminal determines a first truncation bit width based on the first quantization weight and the second quantization weight, where the first truncation bit width is used to indicate the number of binary numbers whose low bits in the first quantization weight are discarded.

[0085] In the embodiment of the present application, the first truncated bit width is the length of the low-order binary number discarded in the first quantization weight. The terminal determines the position of the first binary number of 1 from the first quantization weight, and then intercepts the binary number of the second quantization bit width starting from the first binary number of 1 to obtain the second quantization weight. Then, starting from the first binary number of 1 in the first quantization weight, the low-order binary number that is not intercepted is the discarded binary number, wherein the length of the low-order discarded binary number is the first truncated bit width.

[0086] For example, Figure 6 is a schematic diagram of a first truncated bit width provided in an embodiment of the present application. Figure 6 As shown, the first binary number that is 1 in the first quantization weight is the t-th binary number, and the second quantization bit width is k bits. Then the number of low-order binary numbers discarded in the first quantization weight is tk, that is, the first truncation bit width is tk.

[0087] 307. The terminal determines a second truncation bit width based on the first quantization feature value and the second quantization feature value, where the second truncation bit width is used to indicate the number of binary numbers whose low bits in the first quantization feature value are discarded.

[0088] In the embodiment of the present application, the second truncated bit width is the length of the low-order discarded binary number in the first quantized feature value. The terminal determines the position of the first binary number that is 1 from the first quantized feature value, and then intercepts the binary number of the second quantized bit width starting from the first binary number that is 1 to obtain the second quantized feature value. Then, starting from the first binary number that is 1 in the first quantized feature value, the low-order binary number that is not intercepted is the discarded binary number, wherein the length of the low-order discarded binary number is the second truncated bit width.

[0089] 308. The terminal shifts the intermediate eigenvalue based on the sum of the first truncated bit width and the second truncated bit width to obtain an output eigenvalue of the network layer, where the output eigenvalue is used to represent an output feature map corresponding to the input image at the network layer.

[0090] In the embodiment of the present application, the terminal sums the first truncated bit width and the second truncated bit width to obtain a shift value. Based on the shift value, the terminal can perform a base shift on the intermediate eigenvalue obtained by multiplication to obtain an output eigenvalue of the network layer.

[0091] For example, the intermediate eigenvalue is "110110010000111", the first truncation bit width is 2, and the second truncation bit width is 3. Then the shift bit value is 5, and the output eigenvalue obtained by the terminal after shifting the intermediate eigenvalue is "001000011100000".

[0092] In order to more clearly describe the model quantization method provided in the embodiment of the present application, Figure 7 , the overall process of the model quantization method is further described. Figure 7 As shown, the terminal first quantizes the weight and input eigenvalue of the network layer based on the first quantization bit width, respectively, to obtain the first quantization weight and the first quantization eigenvalue of the length of the first quantization bit width. Then the terminal adopts the DRUM method provided in the embodiment of the present application to truncate the first quantization weight and the second quantization eigenvalue, respectively, to obtain the second quantization weight and the second quantization eigenvalue of the length of the second quantization bit width. Moreover, during the truncation process, the terminal can determine the number of low-order binary numbers discarded from the first quantization weight and the number of low-order binary numbers discarded from the first quantization eigenvalue, that is, the first truncation bit width and the second truncation bit width. Finally, the second quantization weight and the second quantization eigenvalue are input into the multiplier to obtain the intermediate eigenvalue, and the terminal shifts the intermediate eigenvalue based on the sum of the first truncation bit width and the second truncation bit width, that is, the shift value, to obtain the output eigenvalue of the network layer.

[0093] The embodiment of the present application provides a model quantization method, in which, when the weights and input feature values ​​of the network layer of the image classification model are in a long-tail distribution, the weights and input feature values ​​of the network layer are quantized based on the first quantization bit width, so that the first quantization weight and the first quantization feature value with a large numerical range and high quantization accuracy can be obtained. Then, based on the second quantization bit width lower than the first quantization bit width, the first quantization weight and the first quantization feature value are truncated, and the second quantization weight and the second quantization feature value with a lower quantization bit width can be obtained from the first quantization weight and the first quantization feature value, so that the computational overhead and operation time of the multiplier can be effectively reduced while maintaining the larger numerical representation range and higher quantization accuracy of the second quantization weight and the second quantization feature value, thereby improving the quantization efficiency of the image classification model.

[0094] All the above optional technical solutions can be arbitrarily combined to form optional embodiments of the present application, which will not be described one by one here.

[0095] The following is an embodiment of the device of the present application, which can be used to execute the embodiment of the method of the present application. For details not disclosed in the embodiment of the device of the present application, please refer to the embodiment of the method of the present application.

[0096] Figure 8 is a structural diagram of a model quantization device provided in an embodiment of the present application. Figure 8 , the device comprises:

[0097] A quantization module 801 is used for quantizing the weights and input feature values ​​of any network layer of the image classification model, based on a first quantization bit width, respectively, when the weights of the network layer and the input feature values ​​of the input image at the network layer are in a long-tail distribution, to obtain a first quantization weight of the network layer and a first quantization feature value of the input image at the network layer, the image classification model is used to predict the category of the input image, and the input feature value is used to represent an input feature map corresponding to the input image at the network layer;

[0098] A truncation module 802 is used to truncate the first quantization weight and the first quantization feature value respectively when the first quantization bit width is not less than the second quantization bit width, so as to obtain a second quantization weight and a second quantization feature value with a length of the second quantization bit width, wherein the first bit data of the second quantization weight is the first binary number that is 1 starting from the high bit in the first quantization weight, and the first bit data of the second quantization feature value is the first binary number that is 1 starting from the high bit in the first quantization feature value;

[0099] The weighting module 803 is used to weight the second quantization weight and the second quantization eigenvalue to obtain the output eigenvalue of the network layer, and the output eigenvalue is used to represent the output feature map corresponding to the input image in the network layer.

[0100] In some embodiments, the first quantization bit width is p bits, the first quantization weight and the first quantization feature value are composed of p-bit binary numbers, the second quantization bit width is k bits, the second quantization weight and the second quantization feature value are composed of k-bit binary numbers, and p and k are both positive integers;

[0101] Fig. 9 is a schematic diagram of the structure of another model quantization device provided in an embodiment of the present application. Fig. 9 , the truncation module 802 includes:

[0102] A first determining unit 901 is configured to determine, when the first quantization bit width is not less than the second quantization bit width, an m-th binary number from the first quantization weight, and determine an n-th binary number from the first quantization feature value, wherein the m-th binary number is the first binary number that is 1 starting from the high bit in the first quantization weight, and the n-th binary number is the first binary number that is 1 starting from the high bit in the first quantization feature value, and both m and n are positive integers;

[0103] A first truncation unit 902 is used to truncate a binary number with a length of k from the first quantization weight starting from the m-th binary number to obtain a second quantization weight;

[0104] The second truncation unit 903 is used to truncate the binary number with a length of k from the first quantized feature value starting from the nth binary number to obtain the second quantized feature value.

[0105] In some embodiments, the first truncation unit 902 is used to determine the m-k+1th binary number from the first quantization weight based on the m-th binary number and the second quantization bit width when m is not less than k; when the m-k+1th binary number is 0, set the m-k+1th binary number to 1; truncate the binary numbers from the m-th to the m-k+1th bits from the first quantization weight to obtain the second quantization weight, and the m-k+1th binary number is the last bit data of the second quantization weight.

[0106] In some embodiments, the first truncation unit 902 is also used to, when m is less than k, truncate m-bit binary numbers from the first quantization weight, starting from the m-th binary number; fix the decimal point at the end of the m-bit binary number; and add km zeros after the decimal point to obtain the second quantization weight.

[0107] In some embodiments, the second truncation unit 903 is used to determine the n-k+1th binary number from the first quantization feature value based on the n-th binary number and the second quantization bit width when n is not less than k; when the n-k+1th binary number is 0, set the n-k+1th binary number to 1; truncate the binary numbers from the n-k+1th to the n-k+1th bits from the first quantization feature value to obtain the second quantization feature value, and the n-k+1th binary number is the last bit data of the second quantization feature value.

[0108] In some embodiments, the second truncation unit 903 is also used to, when n is less than k, truncate n-bit binary numbers from the first quantitative characteristic value starting from the n-th binary number; fix the decimal point at the end of the n-bit binary number; and add kn zeros after the decimal point to obtain the second quantitative characteristic value.

[0109] In some embodiments, the truncation module 802 is further used to determine the first binary number that is 1 starting from the high bit from the first quantization weight and the first quantization characteristic value respectively when the first quantization bit width is not less than the second quantization bit width; when the number of bits of the first binary number that is 1 in the first quantization weight is less than the second quantization bit width, starting from the last bit of the first quantization weight, a binary number with a length of the second quantization bit width is truncated to obtain the second quantization weight, and the last bit data of the second quantization weight is the last bit data of the first quantization weight; when the number of bits of the first binary number that is 1 in the first quantization characteristic value is less than the second quantization bit width, starting from the last bit of the first quantization characteristic value, a binary number with a length of the second quantization bit width is truncated to obtain the second quantization characteristic value, and the last bit data of the second quantization characteristic value is the last bit data of the first quantization characteristic value.

[0110] In some embodiments, see Fig. 9 , the weighting module 803 includes:

[0111] A calculation unit 904 is used to multiply the second quantization weight and the second quantization eigenvalue to obtain an intermediate eigenvalue of the network layer;

[0112] A second determining unit 905 is used to determine a first truncation bit width based on the first quantization weight and the second quantization weight, where the first truncation bit width is used to indicate the number of binary numbers whose low bits are discarded in the first quantization weight;

[0113] A third determining unit 906 is used to determine a second truncation bit width based on the first quantization feature value and the second quantization feature value, where the second truncation bit width is used to indicate the number of binary numbers whose low bits in the first quantization feature value are discarded;

[0114] The shift unit 907 is used to shift the intermediate eigenvalue based on the sum of the first truncation bit width and the second truncation bit width to obtain the output eigenvalue.

[0115] The embodiment of the present application provides a model quantization device, in the case where the weights and input feature values ​​of the network layer of the image classification model are in a long-tail distribution, the weights and input feature values ​​of the network layer are quantized based on the first quantization bit width, and the first quantization weight and the first quantization feature value with a large numerical range and high quantization accuracy can be obtained. Then, based on the second quantization bit width lower than the first quantization bit width, the first quantization weight and the first quantization feature value are truncated, and the second quantization weight and the second quantization feature value with a lower quantization bit width can be obtained from the first quantization weight and the first quantization feature value, so that the calculation overhead and operation time of the multiplier can be effectively reduced while maintaining the second quantization weight and the second quantization feature value with a large numerical representation range and high quantization accuracy, thereby improving the quantization efficiency of the image classification model.

[0116] It should be noted that the device provided in the above embodiment, when implementing its functions, only uses the division of the above functional modules as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the server is divided into different functional modules to complete all or part of the functions described above. In addition, the device and method embodiments provided in the above embodiment belong to the same concept, and their specific implementation process is detailed in the method embodiment, which will not be repeated here.

[0117] An embodiment of the present application provides a computer device, which includes a processor and a memory; the memory stores at least one program code, and the at least one program code is used to be executed by the processor to implement the model quantization method provided in the above-mentioned various method embodiments.

[0118] Fig.10 1 is a block diagram of a computer device 1000 provided according to an embodiment of the present application. In some embodiments, the computer device 1000 is a smart phone, a tablet computer, a computer with wireless communication function, or a wearable device. The computer device 1000 in the present application includes at least one or more of the following components: a processor 1010, a memory 1020, and at least two wireless links 1030.

[0119] In some embodiments, the processor 1010 includes one or more processing cores. The processor 1010 uses various interfaces and lines to connect various parts of the entire computer device 1000, and executes various functions and processes data of the computer device 1000 by running or executing program codes stored in the memory 1020, and calling data stored in the memory 1020. In some embodiments, the processor 1010 is implemented in at least one hardware form of digital signal processing (DSP), field-programmable gate array (FPGA), and programmable logic array (PLA). The processor 1010 can integrate one or a combination of a central processing unit (CPU), a graphics processing unit (GPU), a neural network processor (NPU), and a modem. Among them, the CPU mainly processes the operating system, user interface, and application programs; the GPU is responsible for rendering and drawing the content that needs to be displayed on the display screen; the NPU is used to implement artificial intelligence (AI) functions; and the modem is used to process wireless communications. It is understandable that the above-mentioned modem may not be integrated into the processor 1010, but may be implemented by a separate chip.

[0120] In some embodiments, the processor 1010 is used to control the working conditions of at least two wireless links 1030. Accordingly, the processor 1010 is a processor integrated with a wireless fidelity (Wireless Fidelity, Wi-Fi) chip. The Wi-Fi chip is a chip with dual Wi-Fi processing capabilities. For example, the Wi-Fi chip is a dual-band dual concurrent (Dual Band Dual Concurrent, DBDC) chip, or a dual-band synchronous (Dual Band Simultaneous, DBS) chip, etc.

[0121] In some embodiments, the memory 1020 includes a random access memory (RAM), and in some embodiments, the memory 1020 includes a read-only memory (ROM). In some embodiments, the memory 1020 includes a non-transitory computer-readable storage medium. The memory 1020 can be used to store program codes. The memory 1020 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as a touch function, a sound playback function, an image playback function, etc.), instructions for implementing the following various method embodiments, etc.; the data storage area may store data (such as audio data, a phone book) created according to the use of the computer device 1000, etc.

[0122] In some embodiments, the memory 1020 stores reception schemes for receiving beacon frames of different wireless links 1030, as well as identifiers of access nodes connected to different wireless links 1030, identifiers of wireless links 1030, and the like.

[0123] The at least two wireless links 1030 are used to connect different access points (APs) to receive downlink data sent by the APs, wherein the different access points are access points in the same router or access points in different routers.

[0124] In some embodiments, the computer device 1000 also includes a display screen. The display screen is a display component for displaying a user interface. In some embodiments, the display screen is a display screen with a touch function, through which the user can use any suitable object such as a finger or a touch pen to perform touch operations on the display screen. In some embodiments, the display screen is usually arranged on the front panel of the computer device 1000. In some embodiments, the display screen is designed to be a full screen, a curved screen, a special-shaped screen, a double-sided screen or a folding screen. In some embodiments, the display screen is also designed to be a combination of a full screen and a curved screen, a combination of a special-shaped screen and a curved screen, etc., which is not limited in this embodiment.

[0125] In addition, those skilled in the art will appreciate that the structure of the computer device 1000 shown in the above figures does not limit the computer device 1000, and the computer device 1000 includes more or fewer components than shown, or combines certain components, or arranges components differently. For example, the computer device 1000 also includes components such as a microphone, a speaker, an input unit, a sensor, an audio circuit, a module, a power supply, and a Bluetooth module, which will not be described in detail here.

[0126] The present application also provides a computer-readable storage medium, which stores at least one program code, and the at least one program code is loaded and executed by the processor to implement the model quantization method shown in the above embodiments.

[0127] The present application also provides a chip, which includes a programmable logic circuit and / or program instructions. When the chip runs on a computer device, it is used to implement the model quantization method shown in the above embodiments.

[0128] The present application also provides a computer program product, which stores at least one program code, and the at least one program code is used to be executed by a processor to implement the model quantization method shown in the above embodiments.

[0129] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0130] Those skilled in the art can understand that all or part of the steps in the model quantization method of the above embodiment can be completed by hardware, or by a program to instruct the relevant hardware to complete, and the program can be stored in a computer-readable storage medium, and the above-mentioned storage medium can be a read-only memory, a disk or an optical disk, etc. The above is only an optional embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. A model quantization method, characterized in that: The method comprises: For any network layer of the image classification model, when the weight of the network layer and the input feature value of the input image at the network layer present a long-tail distribution, based on the first quantization bit width, the weight and the input feature value are quantized respectively to obtain a first quantization weight of the network layer and a first quantization feature value of the input image at the network layer, the image classification model is used to predict the category of the input image, and the input feature value is used to represent the input feature map corresponding to the input image at the network layer; In the case where the first quantization bit width is not less than the second quantization bit width, the first quantization weight and the first quantization feature value are truncated respectively to obtain a second quantization weight and a second quantization feature value with a length of the second quantization bit width, the first bit data of the second quantization weight is the first binary number that is 1 starting from the high bit in the first quantization weight, and the first bit data of the second quantization feature value is the first binary number that is 1 starting from the high bit in the first quantization feature value; The second quantization weight and the second quantization feature value are weighted to obtain an output feature value of the network layer, where the output feature value is used to represent an output feature map corresponding to the input image at the network layer.

2. The method according to claim 1, characterized in that The first quantization bit width is p bits, the first quantization weight and the first quantization feature value are composed of p-bit binary numbers, the second quantization bit width is k bits, the second quantization weight and the second quantization feature value are composed of k-bit binary numbers, and p and k are both positive integers; The step of truncating the first quantization weight and the first quantization characteristic value respectively to obtain a second quantization weight and a second quantization characteristic value having a length of the second quantization bit width when the first quantization bit width is not less than the second quantization bit width comprises: In the case where the first quantization bit width is not less than the second quantization bit width, determining an m-th binary number from the first quantization weight, and determining an n-th binary number from the first quantization feature value, the m-th binary number being the first binary number that is 1 starting from a high bit in the first quantization weight, and the n-th binary number being the first binary number that is 1 starting from a high bit in the first quantization feature value, and m and n are both positive integers; Starting from the m-th binary number, truncating a binary number of length k from the first quantization weight to obtain the second quantization weight; Starting from the nth binary number, a binary number with a length of k is truncated from the first quantitative characteristic value to obtain the second quantitative characteristic value.

3. The method according to claim 2, characterized in that The step of starting from the m-th binary number and extracting a binary number having a length of k from the first quantization weight to obtain the second quantization weight comprises: When m is not less than k, determining the m-k+1th binary number from the first quantization weight based on the mth binary number and the second quantization bit width; When the m-k+1th binary number is 0, the m-k+1th binary number is set to 1; The binary numbers from the mth bit to the m-k+1th bit are intercepted from the first quantization weight to obtain the second quantization weight, and the m-k+1th bit binary number is the last bit data of the second quantization weight.

4. The method according to claim 3, characterized in that The method further comprises: When m is less than k, starting from the m-th binary number, an m-bit binary number is intercepted from the first quantization weight; Fixing a decimal point at the end of the m-bit binary number; km zeros are added after the decimal point to obtain the second quantization weight.

5. The method according to claim 2, characterized in that: The step of extracting a binary number having a length of k from the first quantized feature value starting from the n-th binary number to obtain the second quantized feature value comprises: When n is not less than k, based on the n-th binary number and the second quantization bit width, determining the n-k+1th binary number from the first quantization feature value; When the n-k+1th binary number is 0, the n-k+1th binary number is set to 1; The binary numbers from the nth bit to the n-k+1th bit are intercepted from the first quantitative feature value to obtain the second quantitative feature value, and the n-k+1th bit binary number is the last bit data of the second quantitative feature value.

6. The method according to claim 5, characterized in that The method further comprises: When n is less than k, starting from the nth binary number, an n-bit binary number is intercepted from the first quantitative feature value; A decimal point is fixed at the end of the n-digit binary number; Add kn zeros after the decimal point to obtain the second quantitative characteristic value.

7. The method according to claim 1, characterized in that The method further comprises: When the first quantization bit width is not less than the second quantization bit width, determining the first binary number that is 1 starting from a high bit from the first quantization weight and the first quantization feature value respectively; When the number of bits of the first binary number of the first quantization weight being 1 is less than the second quantization bit width, starting from the last bit of the first quantization weight, a binary number having a length of the second quantization bit width is intercepted to obtain the second quantization weight, and the last bit data of the second quantization weight is the last bit data of the first quantization weight; When the number of bits of the first binary number that is 1 of the first quantization feature value is smaller than the second quantization bit width, starting from the last bit of the first quantization feature value, a binary number with a length of the second quantization bit width is truncated to obtain the second quantization feature value, and the last bit data of the second quantization feature value is the last bit data of the first quantization feature value.

8. The method according to claim 1, characterized in that The step of weighting the second quantization weight and the second quantization eigenvalue to obtain the output eigenvalue of the network layer includes: multiplying the second quantization weight and the second quantization eigenvalue to obtain an intermediate eigenvalue of the network layer; Determine a first truncation bit width based on the first quantization weight and the second quantization weight, where the first truncation bit width is used to indicate the number of binary numbers whose low bits in the first quantization weight are discarded; Determine a second truncation bit width based on the first quantization feature value and the second quantization feature value, where the second truncation bit width is used to indicate the number of binary numbers whose low bits in the first quantization feature value are discarded; The intermediate eigenvalue is shifted based on the sum of the first truncation bit width and the second truncation bit width to obtain the output eigenvalue.

9. A model quantization device, characterized in that: The device comprises: A quantization module, for any network layer of an image classification model, when the weight of the network layer and the input feature value of the input image at the network layer are in a long-tail distribution, quantize the weight and the input feature value based on a first quantization bit width, respectively, to obtain a first quantization weight of the network layer and a first quantization feature value of the input image at the network layer, the image classification model is used to predict the category of the input image, and the input feature value is used to represent an input feature map corresponding to the input image at the network layer; A truncation module, configured to, when the first quantization bit width is not less than the second quantization bit width, respectively truncate the first quantization weight and the first quantization feature value to obtain a second quantization weight and a second quantization feature value having a length of the second quantization bit width, wherein the first bit data of the second quantization weight is the first binary number that is 1 starting from a high bit in the first quantization weight, and the first bit data of the second quantization feature value is the first binary number that is 1 starting from a high bit in the first quantization feature value; A weighting module is used to weight the second quantization weight and the second quantization feature value to obtain an output feature value of the network layer, and the output feature value is used to represent the output feature map corresponding to the input image in the network layer.

10. A computer device, characterized in that: The computer device includes a processor and a memory; the memory stores at least one program code, and the at least one program code is used to be executed by the processor to implement the model quantization method as described in any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that: The storage medium stores at least one program code, and the at least one program code is used to be executed by a processor to implement the model quantization method as described in any one of claims 1 to 8.

12. A chip, characterized in that: The chip includes a programmable logic circuit and / or program instructions, and when the chip is run on a computer device, it is used to implement the model quantization method as described in any one of claims 1 to 8.

13. A computer program product, comprising a computer program, characterized in that The computer program product stores at least one program code, and the at least one program code is used to be executed by a processor to implement the model quantization method according to any one of claims 1 to 8.

Images