Quantization method and device of neural network, computer device and storage medium

Abstract

The present disclosure provides a neural network quantization method and device, computer equipment and storage medium, wherein the method comprises: in each iteration period of a plurality of iteration periods, performing: determining to quantize a target network layer in a first neural network, a plurality of quantization objects corresponding to quantization coefficients; based on the quantization coefficients, performing a quantization operation on the plurality of quantization objects respectively to obtain quantization results of the plurality of quantization objects respectively corresponding to the current iteration period; based on the quantization results of the plurality of quantization objects respectively corresponding to the current iteration period, determining quantization losses of the plurality of quantization objects respectively corresponding to the current iteration period, and based on the quantization losses, determining loss weights corresponding to the quantization losses of the plurality of quantization objects respectively; and determining the quantization coefficients determined in the last iteration period as target quantization coefficients, and performing quantization processing on the first neural network using the target quantization coefficients to generate a second neural network.

Description

Technical Field

[0001] This disclosure relates to the field of neural network technology, and more specifically, to a method, apparatus, computer device, and storage medium for quantizing neural networks. Background Technology

[0002] As neural networks have evolved, their scale and complexity have increased, leading to greater computational and storage resources required for processing tasks. However, the computational and storage resources of terminal devices deploying neural networks are limited. Therefore, neural networks need to be compressed to be deployed on these devices. Quantization, as a compression method, converts the network layer parameters and data transmitted within the network from floating-point to fixed-point numbers. Fixed-point arithmetic requires fewer computational and storage resources compared to floating-point arithmetic. However, current quantization methods often suffer from a significant decrease in accuracy. Summary of the Invention

[0003] This disclosure provides at least one method, apparatus, computer device, and storage medium for quantizing neural networks.

[0004] In a first aspect, embodiments of this disclosure provide a method for quantizing a neural network, comprising: performing the following in each of a plurality of iteration cycles: determining quantization coefficients for quantizing a target network layer in a first neural network and quantization objects corresponding to a plurality of quantization objects; wherein the quantization coefficients are determined based on original quantization coefficients, or based on quantization losses and loss weights determined in historical iteration cycles; performing quantization operations on the plurality of quantization objects based on the quantization coefficients to obtain quantization results for the plurality of quantization objects in the current iteration cycle; determining quantization losses for the plurality of quantization objects in the current iteration cycle based on the quantization results for the plurality of quantization objects in the current iteration cycle, and determining loss weights corresponding to the quantization losses of the plurality of quantization objects based on the quantization losses; and determining the quantization coefficients determined in the last iteration cycle as target quantization coefficients, and using the target quantization coefficients to quantize the first neural network to generate a second neural network.

[0005] In one optional implementation, the quantization object includes at least one of the following: the input data, output data, and intrinsic parameters of the target network layer.

[0006] In one optional implementation, the quantization object includes the intrinsic parameters of the target network layer; the step of determining the current iteration cycle to quantize the target network layer in the first neural network, and the quantization coefficients corresponding to multiple quantization objects, includes: in response to the current iteration cycle being the first iteration cycle, determining the original quantization coefficients of the intrinsic parameters of the network layer based on a preset quantization coefficient space; and determining the quantization coefficients of the current iteration cycle based on the original quantization coefficients of the intrinsic parameters of the network layer.

[0007] In one optional implementation, the quantization object includes: input data of the target network layer and / or output data of the target network layer; determining the quantization coefficients corresponding to the multiple quantization objects for the target network layer in the first neural network in the current iteration cycle includes: in response to the current iteration cycle being the first iteration cycle, determining first distribution feature information of the input data based on the input data; determining the original quantization coefficients of the input data based on the first distribution feature information; and / or, in response to the current iteration cycle being the first iteration cycle, determining second distribution feature information of the output data based on the output data; determining the original quantization coefficients of the output data based on the second distribution feature information; and determining the quantization coefficients of the current iteration cycle based on the original quantization coefficients of the input data and / or the original quantization coefficients of the output data.

[0008] In one optional implementation, the quantization object includes input data, network layer intrinsic parameters, and output data of the target network layer; the quantization coefficient includes a first quantization coefficient of the input data and a second quantization coefficient of the network layer intrinsic parameters; the step of performing quantization operations on the plurality of quantization objects based on the quantization coefficients to obtain quantization results corresponding to the plurality of quantization objects in the current iteration cycle includes: for a first quantization object of type input data, performing a quantization operation on the first quantization object based on the first quantization coefficient to obtain a first quantization result of the first quantization object in the current iteration cycle; for a second quantization object of type network layer intrinsic parameters, performing a quantization operation on the second quantization object based on the second quantization coefficient to obtain a second quantization result of the second quantization object in the current iteration cycle; for a third quantization object of type output data, performing a quantization operation on the third quantization object based on the third quantization coefficient to obtain a third quantization result of the third quantization object in the current iteration cycle; or, determining the third quantization result corresponding to the third quantization object in the current iteration cycle based on the first quantization result and the second quantization result.

[0009] In one optional implementation, determining the quantization loss of the plurality of quantization objects in the current iteration period based on their respective quantization results in the current iteration period includes: for each of the plurality of quantization objects, performing an inverse quantization operation on the quantization result of each quantization object in the current iteration period based on the quantization coefficient to obtain the inverse quantization result corresponding to each quantization object; and obtaining the quantization loss of each quantization object in the current iteration period based on each quantization object and its corresponding inverse quantization result.

[0010] In one optional implementation, determining the loss weights corresponding to the quantization losses of the plurality of quantization objects based on the quantization loss includes: normalizing the quantization losses corresponding to quantization objects of the same type to obtain the loss weights corresponding to the quantization losses of the quantization objects of the same type; or, processing the quantization objects of the same type based on a preset monotonic function to obtain the loss weights corresponding to the quantization losses of the quantization objects of the same type; or, using a pre-trained weight prediction neural network to perform weight prediction processing on the quantization objects of the same type to obtain the loss weights corresponding to the quantization losses of the quantization objects of the same type; or, for each quantization object, determining the target value interval to which the quantization loss of each quantization object belongs among multiple value intervals; and determining the loss weight corresponding to the quantization loss of each quantization object based on the weights corresponding to the target value interval.

[0011] In one optional implementation, after determining the quantization loss corresponding to each of the plurality of quantization objects in the current iteration cycle based on the quantization results corresponding to the plurality of quantization objects in the current iteration cycle, and determining the loss weight corresponding to the quantization loss of each of the plurality of quantization objects based on the quantization loss, the method further includes: determining whether the current iteration cycle meets the iteration stopping condition based on the loss weight corresponding to each of the plurality of quantization objects; and proceeding to the next iteration cycle in response to the current iteration cycle not meeting the iteration stopping condition; the step of determining the quantization coefficient determined in the last iteration cycle as the target quantization coefficient includes: in response to the current iteration cycle meeting the iteration stopping condition, determining the current iteration cycle as the last iteration cycle, and determining the quantization coefficient of the last iteration cycle as the target quantization coefficient.

[0012] In one optional implementation, the iteration stopping condition includes at least one of the following: the total quantization loss is less than a preset loss threshold; the total quantization loss is determined based on the loss weights corresponding to the plurality of quantization objects respectively; the difference between the total quantization losses corresponding to the plurality of target iteration cycles is less than a preset difference threshold; wherein, the target iteration cycle includes: the current iteration cycle and at least one historical iteration cycle that is continuous with the current iteration cycle.

[0013] In one optional implementation, the step of quantizing the first neural network using the target quantization coefficient to generate the second neural network includes: quantizing the quantization object in the target network layer using the target quantization coefficient to obtain the target quantization object; and generating the second neural network based on the target quantization object.

[0014] Secondly, embodiments of this disclosure also provide a quantization apparatus for a neural network, comprising: a processing module, configured to perform the following in each of a plurality of iteration cycles: determining quantization coefficients for a target network layer in a first neural network in the current iteration cycle, and quantization coefficients corresponding to a plurality of quantization objects; wherein the quantization coefficients are determined based on original quantization coefficients, or based on quantization losses and loss weights determined in historical iteration cycles; performing quantization operations on the plurality of quantization objects based on the quantization coefficients to obtain quantization results for the plurality of quantization objects in the current iteration cycle; determining quantization losses for the plurality of quantization objects in the current iteration cycle based on the quantization results for the plurality of quantization objects in the current iteration cycle, and determining loss weights corresponding to the quantization losses of the plurality of quantization objects based on the quantization losses; and a quantization module, configured to determine the quantization coefficients determined in the last iteration cycle as target quantization coefficients, and using the target quantization coefficients to quantize the first neural network to generate a second neural network.

[0015] In one optional implementation, the quantization object includes at least one of the following: input data, output data, and intrinsic parameters of the target network layer.

[0016] In one optional implementation, the quantization object includes the intrinsic parameters of the target network layer; when the processing module determines that the target network layer in the first neural network is quantized in the current iteration cycle, and the quantization coefficients corresponding to multiple quantization objects are determined, the processing module is configured to: in response to the current iteration cycle being the first iteration cycle, determine the original quantization coefficients of the intrinsic parameters of the network layer based on a preset quantization coefficient space; and determine the quantization coefficients of the current iteration cycle based on the original quantization coefficients of the intrinsic parameters of the network layer.

[0017] In one optional implementation, the quantization object includes: input data of the target network layer and / or output data of the target network layer; when the processing module determines that the target network layer in the first neural network is quantized in the current iteration cycle, and the quantization coefficients corresponding to multiple quantization objects are determined, the processing module is configured to: in response to the current iteration cycle being the first iteration cycle, determine first distribution feature information of the input data based on the input data; determine the original quantization coefficients of the input data based on the first distribution feature information; and / or, in response to the current iteration cycle being the first iteration cycle, determine second distribution feature information of the output data based on the output data; determine the original quantization coefficients of the output data based on the second distribution feature information; and determine the quantization coefficients of the current iteration cycle based on the original quantization coefficients of the input data and / or the original quantization coefficients of the output data.

[0018] In one optional implementation, the quantization object includes input data, network layer intrinsic parameters, and output data of the target network layer; the quantization coefficient includes a first quantization coefficient of the input data and a second quantization coefficient of the network layer intrinsic parameters; when the processing module performs quantization operations on the plurality of quantization objects based on the quantization coefficients to obtain quantization results of the plurality of quantization objects corresponding to the current iteration cycle, it is configured to: for the first quantization object of type input data, perform quantization operations on the first quantization object based on the first quantization coefficient to obtain a first quantization result of the first quantization object in the current iteration cycle; for the second quantization object of type network layer intrinsic parameters, perform quantization operations on the second quantization object based on the second quantization coefficient to obtain a second quantization result of the second quantization object in the current iteration cycle; for the third quantization object of type output data, perform quantization operations on the third quantization object based on the third quantization coefficient to obtain a third quantization result of the third quantization object in the current iteration cycle; or, determine the third quantization result of the third quantization object corresponding to the current iteration cycle based on the first quantization result and the second quantization result.

[0019] In one optional implementation, when the processing module determines the quantization loss of the plurality of quantization objects in the current iteration period based on the quantization results of the plurality of quantization objects in the current iteration period, it is configured to: for each of the plurality of quantization objects, perform an inverse quantization operation on the quantization result corresponding to each quantization object based on the quantization coefficient to obtain the inverse quantization result of each quantization object in the current iteration period; and obtain the quantization loss of each quantization object in the current iteration period based on each quantization object and the corresponding inverse quantization result.

[0020] In one optional implementation, when the processing module determines the loss weights corresponding to the quantization losses of the plurality of quantization objects based on the quantization loss, it is configured to: normalize the quantization losses corresponding to the quantization losses of the quantization objects belonging to the same type to obtain the loss weights corresponding to the quantization losses of the quantization objects belonging to the same type; or, process the quantization objects belonging to the same type based on a preset monotonic function to obtain the loss weights corresponding to the quantization losses of the quantization objects belonging to the same type; or, use a pre-trained weight prediction neural network to perform weight prediction processing on the quantization objects belonging to the same type to obtain the loss weights corresponding to the quantization losses of the quantization objects belonging to the same type; or, for each quantization object, determine the target value interval to which the quantization loss of each quantization object belongs among multiple value intervals; and determine the loss weight corresponding to the quantization loss of each quantization object based on the weights corresponding to the target value interval.

[0021] In one optional implementation, after determining the quantization loss corresponding to each of the plurality of quantized objects in the current iteration cycle based on the quantization results of the plurality of quantized objects in the current iteration cycle, and determining the loss weight corresponding to the quantization loss of each of the plurality of quantized objects based on the quantization loss, the processing module is further configured to: determine whether the current iteration cycle meets the iteration stop condition based on the loss weight corresponding to each of the plurality of quantized objects; and proceed to the next iteration cycle in response to the current iteration cycle not meeting the iteration stop condition; when determining the quantization coefficient determined in the last iteration cycle as the target quantization coefficient, the processing module is configured to: determine the current iteration cycle as the last iteration cycle in response to the current iteration cycle meeting the iteration stop condition, and determine the quantization coefficient of the last iteration cycle as the target quantization coefficient.

[0022] In one optional implementation, the iteration stopping condition includes at least one of the following: the total quantization loss is less than a preset loss threshold; the total quantization loss is determined based on the loss weights corresponding to the plurality of quantization objects respectively; the difference between the total quantization losses corresponding to the plurality of target iteration cycles is less than a preset difference threshold; wherein, the target iteration cycle includes: the current iteration cycle and at least one historical iteration cycle that is continuous with the current iteration cycle.

[0023] In one optional implementation, when the quantization module quantizes the first neural network using the target quantization coefficient to generate the second neural network, it is configured to: quantize the quantization object in the target network layer using the target quantization coefficient to obtain the target quantization object; and generate the second neural network based on the target quantization object.

[0024] Thirdly, an optional implementation of this disclosure also provides a computer device, a processor, and a memory, wherein the memory stores machine-readable instructions executable by the processor, and the processor is configured to execute the machine-readable instructions stored in the memory. When the machine-readable instructions are executed by the processor, the steps of the first aspect above, or any possible implementation of the first aspect, are performed.

[0025] Fourthly, an optional implementation of this disclosure also provides a computer-readable storage medium storing a computer program that, when run, performs the steps of the first aspect or any possible implementation of the first aspect.

[0026] For a description of the effects of the quantization device, computer equipment, and computer-readable storage medium of the aforementioned neural network, please refer to the description of the quantization method of the aforementioned neural network, which will not be repeated here.

[0027] To make the above-mentioned objects, features and advantages of this disclosure more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0028] To more clearly illustrate the technical solutions of the embodiments of this disclosure, the accompanying drawings used in the embodiments will be briefly described below. These drawings are incorporated in and constitute a part of this specification. They illustrate embodiments conforming to this disclosure and, together with the specification, serve to explain the technical solutions of this disclosure. It should be understood that the following drawings only show some embodiments of this disclosure and should not be considered as limiting the scope. Those skilled in the art can obtain other related drawings based on these drawings without creative effort.

[0029] Figure 1 A flowchart of a neural network quantization method provided in an embodiment of this disclosure is shown;

[0030] Figure 2 A schematic diagram of a quantization device for a neural network provided in an embodiment of the present disclosure is shown;

[0031] Figure 3 A schematic diagram of a computer device provided in an embodiment of the present disclosure is shown. Detailed Implementation

[0032] To make the objectives, technical solutions, and advantages of the embodiments of this disclosure clearer, the technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this disclosure, and not all of them. The components of the embodiments of this disclosure described and shown herein can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this disclosure is not intended to limit the scope of the claimed disclosure, but merely represents selected embodiments of this disclosure. All other embodiments obtained by those skilled in the art based on the embodiments of this disclosure without inventive effort are within the scope of protection of this disclosure.

[0033] Research has revealed that neural networks contain numerous intrinsic parameters and processing data (such as input and output data for each layer), all of which are presented in floating-point number format. Directly processing tasks using floating-point numbers results in significant computational and storage resource consumption due to the large number of bits required. To reduce computational and storage resource consumption while maintaining neural network performance, quantization can be performed on the intrinsic parameters and processing data. This converts floating-point data to fixed-point data with a fixed number of bits, reducing the number of bits required for computation and storage. However, since quantization inevitably reduces data precision, affecting neural network accuracy, the quantization parameters are typically determined by measuring the distribution difference between the quantized and original neural networks. Specifically, one approach is to directly determine the quantization difference of each layer between the quantized neural network and the initial neural network, and then calculate the average of the quantization differences for each layer to determine the quantization loss during quantization. The quantization loss is then used to determine the quantization coefficients. However, this method tends to overlook the impact of parameters with large quantization differences on the quantization loss, while amplifying the impact of parameters with small quantization differences. This imbalance leads to a significant decrease in the accuracy of the quantized neural network obtained after determining the quantization coefficients using the quantization loss and then quantizing the neural network using those coefficients.

[0034] Based on the above research, this disclosure provides a quantization method for neural networks. The corresponding quantization weight is determined by the magnitude of the quantization loss corresponding to each quantization object, and the quantization loss of the network layer is determined by the quantization weight. This method can balance the influence of quantization objects with different quantization differences on the quantization loss. Therefore, after obtaining the quantization coefficient using the quantization loss, the quantized neural network obtained based on the quantization coefficient has higher accuracy.

[0035] The shortcomings of the above solutions are the result of the inventor's practical experience and careful research. Therefore, the discovery process of the above problems and the solutions proposed in this disclosure below should be considered as the inventor's contribution to this disclosure.

[0036] It should be noted that similar labels and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.

[0037] To facilitate understanding of this embodiment, a detailed description of the neural network quantization method disclosed in this disclosure is provided first. The execution entity of the neural network quantization method provided in this disclosure is generally a computer device with certain computing capabilities. This computer device may include, for example, a terminal device, a server, or other processing devices. The terminal device may be a user equipment (UE), mobile device, user terminal, terminal, cellular phone, cordless phone, personal digital assistant (PDA), handheld device, computing device, in-vehicle device, wearable device, etc. In some possible implementations, the neural network quantization method can be implemented by a processor calling computer-readable instructions stored in memory.

[0038] The quantization method for neural networks provided in the embodiments of this disclosure will be described below.

[0039] See Figure 1 The diagram shows a flowchart of a quantization method for a neural network provided in an embodiment of this disclosure, wherein the quantization method for the neural network includes the following steps S101 to S102:

[0040] S101: Perform the following steps S1011 to S1013 in each of the multiple iteration cycles, wherein:

[0041] S1011: Determine the quantization of the target network layer in the first neural network during the current iteration cycle, and the quantization coefficients corresponding to multiple quantization objects; wherein, the quantization coefficients are determined based on the original quantization coefficients, or based on the quantization loss and loss weights determined during the historical iteration cycles;

[0042] S1012: Perform quantization operations on the plurality of quantization objects based on the quantization coefficients to obtain the quantization results of the plurality of quantization objects in the current iteration cycle;

[0043] S1013: Based on the quantization results of the plurality of quantization objects in the current iteration cycle, determine the quantization loss of the plurality of quantization objects in the current iteration cycle, and based on the quantization loss, determine the loss weight of the plurality of quantization objects.

[0044] S102: After completing at least one iteration cycle, the quantization coefficient determined in the last iteration cycle is determined as the target quantization coefficient, and the first neural network is quantized using the target quantization coefficient to generate the second neural network.

[0045] This embodiment of the disclosure determines the quantization coefficients corresponding to multiple quantization objects when quantizing the target network layer in the first neural network. The quantization coefficients are then used to perform quantization operations on each of the multiple quantization objects to obtain quantization results for each object. Based on these results, a quantization loss and its corresponding weight are determined. Thus, for each quantization object's quantization coefficient, the quantization weight determined for that object and the obtained quantization loss can be used to calculate the quantization coefficient for the next iteration cycle. By determining the corresponding quantization weight based on the magnitude of the quantization loss for each object, and using this weight to determine the quantization loss of the network layer, the influence of quantization objects with different quantization differences on the quantization loss can be balanced. Therefore, after obtaining the quantization coefficients using the quantization loss, the quantized neural network obtained based on these coefficients has higher accuracy.

[0046] The above S101 to S102 are explained in detail below.

[0047] First, taking the first iteration cycle as an example, S101 of this disclosure embodiment will be described in detail.

[0048] Regarding S1011 above, the first neural network described therein may be different types in different application scenarios. For example, in image processing applications, the corresponding first neural network may include convolutional neural networks (CNNs); in natural language processing applications, the corresponding first neural network may include recurrent neural networks (RNNs). For example, the quantization method of the neural network provided in this disclosure embodiment will be described below using a convolutional neural network as the first neural network.

[0049] In one possible scenario, any neural network to be deployed can be processed using the quantization method provided in the embodiments of this disclosure, so that the model parameters and processing data in the neural network to be deployed are quantized into data occupying fewer bits, thereby meeting the deployment requirements of hardware devices, such as deployment in small devices such as embedded devices.

[0050] In practical implementation, when quantizing the first neural network, specific target network layers can be quantized. The specific target network layer selected can be determined according to the actual situation. For example, if only the convolutional layers and fully connected layers with a large number of model parameters in the first neural network are quantized, then some or all of the convolutional layers and fully connected layers in the first neural network can be used as the target network layer. Alternatively, if the device to be deployed only supports neural networks quantized to a certain number of bits, then all network layers in the first neural network can be used as the target network layer for quantization.

[0051] Given a target network layer in the first neural network, multiple quantization objects can be determined accordingly based on the target network layer. Specifically, the quantization objects include at least one of the following: input data, output data, and intrinsic parameters of the target network layer.

[0052] For cases where the quantization object includes the intrinsic parameters of the target network layer, for example, if the target network layer includes a convolutional layer, the corresponding quantization object may include each convolution factor in the convolution kernel; for example, if the convolution kernel is a 3*3*5 convolution kernel, then each element therein becomes a convolution factor.

[0053] In specific implementation, when the quantization object includes the intrinsic parameters of the target network layer, the following method can be used to determine the quantization coefficients corresponding to multiple quantization objects in the current iteration cycle: in response to the current iteration cycle being the first iteration cycle, the original quantization coefficients of the intrinsic parameters of the network layer are determined based on a preset quantization coefficient space; the quantization coefficients of the current iteration cycle are determined based on the original quantization coefficients of the intrinsic parameters of the network layer.

[0054] Specifically, since the intrinsic parameters of each network layer in the first neural network are determined before quantization, if the number of bits to be quantized is determined, then the selectable quantization coefficients for each network layer intrinsic parameter can also be determined within a certain range. For example, the range of quantization coefficients can be determined empirically. For instance, if the first convolution factor b in the target network layer of the first neural network has a value of 0.1, then when quantized to 3 bits, the possible quantization coefficients are in the range (0, 0.1). Here, the determined range of possible values ​​for the quantization coefficients can be used as a preset quantization coefficient space.

[0055] For example, when determining the preset quantization coefficient space, uniform quantization or non-uniform quantization can be used. For any type of quantization object, such as the first convolution factor in a convolution kernel, taking uniform quantization as an example, multiple intervals can be determined based on the size range of the quantization result after quantization. For example, if the size range of the convolution factor is 0 to 1, it can be quantized into 5 intervals using an average division method: [0, 0.2], (0.2, 0.4], (0.4, 0.6], (0.6, 0.8], and (0.8, 1). When quantizing different convolution factors into 5 different intervals, a corresponding quantization coefficient of 0.2 can be determined for each of the 5 different intervals. Specifically, when determining the quantization coefficient corresponding to the quantization interval, different values ​​can be selected to determine whether the selected value can quantize the object within a certain value range into the divided intervals using a calculation method. If the selected quantization coefficient cannot quantize the object into the divided intervals, a new quantization coefficient is determined until a quantization coefficient that meets the quantization requirements is determined.

[0056] Specifically, when determining the quantization coefficients under uniform quantization, the distribution of the original data corresponding to the quantization object can be obtained and mapped to the target quantization interval to determine the quantization coefficients. For example, the maximum and minimum values ​​in the original data can be obtained and mapped to an 8-bit quantization interval. The quotient obtained by taking the difference between the maximum and minimum values ​​and 2 to the power of 8 can be used as the quantization coefficient.

[0057] Here, when uniform quantization is used to determine the quantization coefficients, the quantization coefficient space can be determined relatively easily because the division of the interval is relatively simple during uniform quantization.

[0058] If a non-uniform quantization method is used, multiple intervals can be determined based on the numerical distribution of the quantized object. Specifically, regions with a higher numerical distribution can be divided into more intervals, and regions with a lower numerical distribution can be divided into fewer intervals. Taking the convolution factor as an example, its numerical distribution can be determined based on the convolution factors of multiple convolution operations. For instance, a large number of values ​​might be distributed in the interval [0, 0.2], while the rest might be distributed in the intervals (0.2, 0.5] and (0.5, 1]. Since there are more values ​​in the interval [0, 0.2], this interval can be further refined, for example, into [0, 0.025], (0.025, 0.1], and (0.1, 0.2). When determining the corresponding quantization coefficient for each interval, for example, the interval [0, 0.025] can be set based on the interval and the value range of the quantized object. The corresponding quantization coefficient is 0.005 for the interval (0.025, 0.1], 0.015 for the interval (0.1, 0.2], 0.02 for the interval (0.2, 0.5], and 0.1 for the interval (0.5, 1]. The method for determining the quantization coefficient in non-uniform quantization can be the same as for uniform quantization, or the original data of the quantization object can be first subjected to a nonlinear transformation (e.g., logarithmic transformation), then the distribution of the original data after the nonlinear transformation is determined, and then the quantization coefficient is obtained similarly using the uniform quantization method.

[0059] Here, when using non-uniform quantization to determine the quantization coefficients, the distribution of the specific values ​​of the quantization coefficients is taken into account. Therefore, when dividing the intervals and determining the quantization coefficients for each interval, it can better meet the actual quantization requirements, and the resulting preset quantization coefficient space is more reasonable.

[0060] In practical implementation, for the aforementioned uniform and non-uniform quantization methods, for the quantized object, the quantization coefficients determined within the possible numerical ranges of the quantized object can constitute the quantization coefficient space for that quantized object, i.e., the preset quantization coefficient space described above. Taking the quantized object b as an example, in the quantization coefficient space, based on its range, the corresponding quantization coefficients for that range can be determined, and these determined quantization coefficients are used as the original quantization coefficients for quantized object b. For example, if the value of quantized object b is 0.06, then in the uniform quantization example, the corresponding quantization coefficient can be determined to be 0.2, while in the non-uniform quantization example, the corresponding quantization coefficient can be determined to be 0.015.

[0061] Similarly, the original quantization coefficients of other network layer intrinsic parameters in the target network layer can be determined in a similar manner. In one possible case, for the first iteration cycle, the original quantization coefficients determined from the quantization coefficient space can be directly used as the quantization coefficients corresponding to that first iteration cycle.

[0062] In another embodiment of this disclosure, the quantization object may further include the input data and / or output data of the target network layer.

[0063] In specific implementations, when the quantization objects include input data and / or output data of the target network layer, when determining the quantization coefficients corresponding to multiple quantization objects for quantizing the target network layer in the first neural network, the following methods can be used, for example: In response to the current iteration period being the first iteration period, based on the input data, determine the first distribution feature information of the input data; based on the first distribution feature information, determine the original quantization coefficients of the input data; and / or, in response to the current iteration period being the first iteration period, based on the output data, determine the second distribution feature information of the output data; based on the second distribution feature information, determine the original quantization coefficients of the output data; based on the original quantization coefficients of the input data and / or the original quantization coefficients of the output data, determine the quantization coefficients of the current iteration period.

[0064] For any layer in the target network, the corresponding input data may include, for example, the original image or a feature map output from the previous network layer; the corresponding output data may be, for example, another feature map obtained by processing the input data. Since, for the first neural network, taking image processing as an example, multiple different images are input into the first neural network, the output data obtained by the same network layer of the first neural network for each image will also be different, and consequently, the input data for the same network layer will also be different. Because the input data will differ due to the different original images or the feature maps output by the previous network layer, when determining the quantization coefficients corresponding to the input data, the first distribution feature information of the input data can be determined first. This first distribution feature information may include, for example, the value range and value distribution of the input data. Using this first distribution feature information, a method similar to that used to determine the quantization coefficient space of the network layer intrinsic parameters described above can be adopted to determine multiple intervals corresponding to the value range of the input data, and the quantization coefficients corresponding to each interval, thus determining the corresponding original quantization coefficients for the input data with specific values.

[0065] Furthermore, if the first layer of the first neural network is taken as the target layer, for the output data, for example, the input data of the target layer (i.e., the input data for the entire first neural network) can be processed once using the first neural network to obtain the output data of the target layer. For each layer of the first neural network, when determining the first distribution feature information corresponding to the input data and the second distribution feature information corresponding to the output data, the network layer is used as a benchmark to determine the input data and output data of that layer, and further determine the corresponding distribution feature information. Similar to the input data mentioned above, when determining the corresponding quantization coefficients for the output data of the target layer, the second distribution feature information of the output data can also be determined first, and the corresponding original quantization coefficients for the output data can be determined using the second distribution feature information.

[0066] In another embodiment of this disclosure, since directly using quantization coefficients to quantize an object will change the value of quantized objects with a value of 0, even though the value of a quantized object with a value of 0 should still be 0 after quantization, a zero-point value can be determined simultaneously with the quantization coefficient for the quantized object, and this zero-point value can be used as a part of the quantization coefficient. The zero-point value can be used to process quantized objects with a value of 0, ensuring that their value remains unchanged after quantization. That is, for any quantized object, a corresponding quantization coefficient and a zero-point value can be determined.

[0067] For the multiple quantization objects described above, for example, a first quantization coefficient s1 and a zero-point value z1 can be determined for the input data; a second quantization coefficient s2 and a zero-point value z2 can be determined for the network layer intrinsic parameters; and a third quantization coefficient s3 and a zero-point value z3 can be determined for the output data.

[0068] Regarding S1012 above, if a corresponding quantization coefficient is determined for each quantization object, the quantization coefficient can be used to perform quantization operations on multiple quantization objects separately to obtain the quantization results corresponding to each of the multiple quantization objects.

[0069] In specific implementation, when the quantization object includes input data, the corresponding quantization coefficient s1 and zero-point value z1, i.e., the first quantization coefficient, can be determined according to the above S101. When quantization processing is performed using the first quantization object (i.e., the quantization object whose type is input data) corresponding to the first quantization coefficient, the first quantization result of the first quantization object can be obtained accordingly. For example, the obtained first quantization result satisfies the following formula (1):

[0070]

[0071] Where, xl This represents the input data of the l-th layer, also known as the first quantized object; This indicates the first quantized object x l The quantization result, after quantization, is labeled with the subscript "int" to easily distinguish the data before and after quantization, indicating that the quantized object was quantized to a smaller number of bits. 'b' represents the number of bits quantized, which can be set according to the actual situation. 'round(·)' represents the rounding operation. Using the rounding operation and the zero-point value z1, the first quantized object can be processed, and 'clamp(·)' can be used to map it to [0, 2]. b Within the interval [-1], quantization is completed, and the first quantization result is obtained.

[0072] Similarly, when the quantization object includes network layer intrinsic parameters, its corresponding quantization coefficient s2 and zero-point value z2, i.e., the second quantization coefficient, can be determined according to S101 above. When performing quantization processing using the second quantization object (i.e., the quantization object of type network layer intrinsic parameters) corresponding to the second quantization coefficient, the second quantization result of the second quantization object can be obtained accordingly. For example, the obtained second quantization result satisfies the following formula (2):

[0073]

[0074] Among them, w l This represents the intrinsic parameters of the network layer, also known as the second quantization object; This indicates the second quantized object w. l The result obtained after quantification. The remaining terms have similar meanings to those expressed in (1) above, and will not be repeated here.

[0075] When the quantized object includes output data, since the output data of the previous network layer is the input data of the next network layer, the output data can be represented, for example, as x. l+1 That is, the third quantification object.

[0076] In determining the third quantization object x l+1 For the corresponding third quantization result, for example, but not limited to, the following methods (A1) or the following methods (A2) can be used:

[0077] (A1): Based on the third quantization coefficient, the third quantization object is quantized to obtain the third quantization result of the third quantization object.

[0078] Similar to the first quantization result for determining the first quantization object and the second quantization result for determining the second quantization object, when the quantization object includes output data, the corresponding quantization coefficient s3 and zero-point value z3, i.e., the third quantization coefficient, can be determined according to S101 above. When performing quantization processing using the third quantization object (i.e., the quantization object whose type is output data) corresponding to the third quantization coefficient, the third quantization result of the third quantization object can be obtained accordingly. For example, the obtained third quantization result satisfies the following formula (3):

[0079]

[0080] Among them, w l+1 This represents the output data, also known as the third quantization object; This indicates the third quantized object w. l+1 The result obtained after quantification. The remaining terms have similar meanings to those expressed in formulas (1) and (2) above, and will not be repeated here.

[0081] (A2): Based on the first quantization result and the second quantization result, determine the third quantization result corresponding to the third quantization object.

[0082] Due to the first quantized object x l The second quantization object w l And the third quantized object x l+1 Between them, the following formula (4) is satisfied:

[0083] x l+1 =w l x l (4)

[0084] Therefore, the third quantization result corresponding to the third quantization object The first quantization result of the first quantization object and the second quantization result corresponding to the second quantization object Between them, the following formula (5) can be satisfied:

[0085]

[0086] In other words, by using the first quantization result of the first quantization object and the second quantization result of the second quantization object, the third quantization result of the third quantization object can be directly determined.

[0087] Based on the method for determining the quantization results described above, the quantization results corresponding to the multiple quantization objects of the target network layer can be determined accordingly.

[0088] Regarding S1013 above, in the case where the quantization results corresponding to multiple quantization objects are determined respectively, a corresponding quantization loss can also be determined for each of the multiple quantization objects. Specifically, for example, for each of the multiple quantization objects, based on the quantization coefficient, an inverse quantization operation can be performed on the quantization result corresponding to each quantization object to obtain the inverse quantization result corresponding to each quantization object; based on each quantization object and the corresponding inverse quantization result, the quantization loss of each quantization object can be obtained.

[0089] The process of determining the quantization loss for quantization objects of the types of input data, network layer intrinsic parameters, and output data will be explained below.

[0090] For a quantized object whose type is input data, the first quantization coefficient corresponding to the input data (quantization coefficient s1) and the zero-point value z1 can be used to quantize the first quantization result. Perform a dequantization operation to obtain the first dequantization result. The first dequantization result, for example, satisfies the following formula (6):

[0091]

[0092] in, This represents the first inverse quantization result.

[0093] When the first quantization coefficient is chosen accurately, the first inverse quantization result obtained after inverse quantization of the first quantization result should have a small difference from the first quantized object, thus ensuring that the accuracy loss of the first quantized object is small. Therefore, the following formula (7) can be used to measure the quantization loss of the quantized object, thereby reflecting whether the selection of the first quantization coefficient is relatively accurate:

[0094]

[0095] In reality, the target network layer has multiple input data points, for example, N. For the i-th input data point among the N input data points, the quantization loss can be determined using, for example, the following formula (8):

[0096]

[0097] Where res(·) represents the interpolation function. The resulting... This is the first quantization loss corresponding to the i-th first quantization object.

[0098] For quantization objects classified as network layer intrinsic parameters, the second quantization coefficients corresponding to the network layer intrinsic parameters—quantization coefficient s2 and zero-point value z2—can be used to quantize the second quantization result. Perform a dequantization operation to obtain a second dequantization result. This second dequantization result may satisfy, for example, the following formula (9):

[0099]

[0100] in, This indicates the result of the second inverse quantization.

[0101] When the second quantization coefficient is chosen accurately, the second inverse quantization result obtained after performing an inverse quantization operation on the second quantization result should have a small difference from the second quantized object, thus ensuring that the second quantized object suffers minimal loss in accuracy. Therefore, the following formula (10) can be used to measure the quantization loss of the quantized object, thereby reflecting whether the choice of the second quantization coefficient is relatively accurate:

[0102]

[0103] In reality, for the target network layer, there are multiple intrinsic parameters corresponding to the target network layer, for example, J. Specifically, for the j-th intrinsic parameter among the J intrinsic parameters, the corresponding quantization loss can be determined using, for example, the following formula (11):

[0104]

[0105] Among them, the obtained This is the second quantization loss corresponding to the j-th second quantization object.

[0106] For the third quantization object, which is output data, the method for determining the third inverse quantization result of the third quantization object is first explained. Specifically, when determining the third inverse quantization result, for example, but not limited to, the following methods (B1) or (B2) can be used:

[0107] (B1): Using the third quantization coefficient corresponding to the output data: quantization coefficient s3 and zero-point value z3, the third quantization result can be... Perform a dequantization operation to obtain a third dequantization result. This third dequantization result may satisfy, for example, the following formula (12):

[0108]

[0109] in, This indicates the third inverse quantization result.

[0110] (B2): Using the first inverse quantization result (i.e., the dequantized result of the input data), and the second dequantized result. (That is, the inverse quantization result of the network layer intrinsic parameters), determine the third inverse quantization result.

[0111] Specifically, the first inverse quantization result corresponding to the first quantization object can be obtained according to the above formula (6). Furthermore, according to the above formula (9), the second inverse quantization result corresponding to the second quantization object can be obtained. Based on the above formulas (4) and (5), for example, the first inverse quantization result can be determined accordingly. Second inverse quantization result And the third inverse quantization result The following formula (13) applies to the terms between them:

[0112]

[0113] Similarly, in determining the third quantization result Then, the quantization loss corresponding to the third quantization result can be determined. For the third quantization object, if the third quantization object includes H, for the h-th output data among the H output data, the corresponding quantization loss can be determined, for example, by using the following (14):

[0114]

[0115] in, This represents the quantization loss corresponding to h output data points.

[0116] Given that the quantization loss for each of the multiple quantization objects is determined, the loss weight for each of the multiple quantization objects can also be determined based on the quantization loss.

[0117] Specifically, since the quantization loss varies for each quantization object, or there may be cases where the quantization loss for one or more quantization objects is relatively large, loss weights can be assigned to different quantization losses to reflect the differences in quantization losses. Specifically, the following methods can be used, but are not limited to, when determining loss weights:

[0118] A. Normalize the quantization losses corresponding to quantization objects of the same type to obtain the loss weights corresponding to the quantization objects of the same type.

[0119] Specifically, for the same type of quantization object, since the numerical values ​​of the quantization objects are not the same, the corresponding quantization coefficients are also different, and therefore the loss weights corresponding to each quantization object are also different. When determining the loss weights for different quantization losses, a normalization method can be used to determine the distribution of the weight loss in the [0,1] space. Based on the distribution, smaller loss weights are assigned to smaller quantization losses, and larger weights are assigned to larger quantization losses. In this way, using larger loss weights can highlight quantization objects with larger quantization losses, and in the next iteration cycle, the quantization coefficients corresponding to those quantization objects can be adjusted more precisely, thus making the quantization more accurate.

[0120] B. Based on a preset monotonic function, quantization objects of the same type are processed separately to obtain the loss weights corresponding to the quantization objects of the same type.

[0121] Similar to the method in A above, a monotonic function can also be used to determine the loss weights according to different quantization losses. This is because a monotonic function has, for example, e... x If the function is monotonically increasing, then the corresponding loss weights can be determined for quantization losses of different magnitudes.

[0122] For example, taking the first quantized object as an example, when determining the corresponding loss weight based on the first quantized loss corresponding to the first quantized object, the following formula (15) can be used:

[0123]

[0124] in, This represents the loss weight corresponding to the first quantization loss of the i-th first quantization object out of N first quantization objects. Since e x Since it is a monotonically increasing function, if the first quantization loss of the i-th first quantization object is large, the corresponding calculated loss weight will be... The value is also relatively large; if the first quantization loss of the i-th first quantization object is relatively small, then the corresponding calculated loss weight... The value is also relatively large.

[0125] C. Using a pre-trained weight prediction neural network, weight prediction processing is performed on quantized objects belonging to the same type to obtain the loss weights corresponding to the quantized objects belonging to the same type.

[0126] In the pre-training of the weight prediction neural network, taking the quantized objects belonging to the input data as an example, sample data pairs can be determined using the quantization coefficients corresponding to the input data in the already quantized neural network, as well as the loss weights corresponding to each quantization coefficient. These sample data pairs are then used to train the weight prediction neural network, resulting in a trained weight prediction neural network. The weight prediction neural network is then used to process the first quantization loss corresponding to the input data in the first neural network to determine the loss weight corresponding to each first quantization loss.

[0127] D. For each quantized object, determine the target value interval to which each quantized object belongs among multiple value intervals; based on the weights corresponding to the target value intervals, determine the loss weights corresponding to each quantized object.

[0128] Specifically, since the purpose of setting loss weights is to focus on adjusting the parts with larger quantization losses when adjusting quantization coefficients, weights can also be set by setting multiple value ranges. Specifically, multiple value ranges can be determined based on the possible value range of the quantization loss of the quantization object. In response to the quantization loss falling into any value range, that value range can be used as the target value range, and the weight corresponding to the target value range can be used as the loss weight corresponding to the quantization loss of the quantization object.

[0129] For example, for a quantized object belonging to input data, we can obtain the first quantization loss corresponding to each of the N input data. Based on the distribution of the first quantization loss, we can determine the distributional differences among the obtained quantization losses for the first quantized object. For example, for N input data, the corresponding quantization losses are concentrated in p intervals a1, a2, ..., a p In (where p is a positive integer), a corresponding quantization loss can be determined for each interval, for example, for the q-th interval a. q Determine the corresponding quantization loss r q For the i-th first quantization loss When determining the corresponding quantization loss, the quantization loss can be determined. The interval to be filled, for example, the qth interval a q and the interval a q The corresponding quantization loss r q As the i-th first quantization loss The corresponding quantization loss.

[0130] For example, the loss weight of the quantization loss corresponding to the input data can be expressed as follows: The loss weights of the quantization loss corresponding to the intrinsic parameters of the network layer can be expressed as follows: The loss weights of the quantization loss corresponding to the output data can be expressed as follows:

[0131] After determining the quantization loss and the corresponding loss weight for each of the multiple quantization objects in the first iteration cycle, for any quantization object, the quantization loss is processed using the loss weight corresponding to the quantization loss. This allows us to obtain the quantization loss and corresponding loss weight determined using that quantization object, and thus determine the total quantization loss.

[0132] For example, the total quantitative loss corresponding to the input data can be expressed by the following formula (16):

[0133]

[0134] in, This represents the total quantization loss corresponding to the input data. When calculating the total quantization loss corresponding to the input data, the quantization loss corresponding to the i-th input data is multiplied by the corresponding loss weight.

[0135] Similarly, for the intrinsic parameters of the network layers, the corresponding totalized loss can be expressed, for example, by the following formula (17):

[0136]

[0137] Furthermore, for the output data, the corresponding totalization loss can be expressed, for example, by the following formula (18):

[0138]

[0139] In determining the total quantization loss, one possible approach is to assign a separate total quantization loss to each quantized object in the target network layer. This is problematic because, with a large number of quantized objects, the resulting total quantization loss is also large, consuming more computational power and reducing efficiency. Furthermore, for each quantized object, its corresponding quantization loss can determine its corresponding quantization weight, allowing evaluation of whether the quantization result of that object is superior among similar quantized objects—specifically, whether the assigned loss weight is smaller. Therefore, to determine the total quantization loss, a different approach is to use a single network layer in the target network as the smallest unit, determining the total quantization loss for multiple quantized objects within that layer. Specifically, the total quantization loss is determined based on the loss weights corresponding to each quantized object, for example, by weighting the corresponding quantization losses using these weights. This results in a smaller number of total quantization losses, making judgment easier, and the obtained total quantization loss can better reflect the quantization results of the quantized objects in that network layer.

[0140] For example, for a single network layer in the target network layer, we can obtain a first total loss corresponding to the first quantized object (input data), a second total loss corresponding to the second quantized object (network layer intrinsic parameters), and a third total loss corresponding to the third quantized object (output data). Since the quantization coefficients of quantized objects of the same type are quite similar for different quantized objects, we can determine the quantization coefficients for different types of quantized objects one by one by independently determining the total loss corresponding to each type of quantized object.

[0141] In one possible scenario, if the quantization loss corresponding to the quantized object is small, the corresponding quantization weight is also small, resulting in a correspondingly small total quantization loss. In this case, the quantization coefficient set for the quantized object can be considered reasonable and can be used. In another possible scenario, if the quantization loss corresponding to the quantized object is large, the corresponding quantization weight is also large, resulting in a correspondingly large total quantization loss. In this case, the quantization coefficient set for the quantized object can be determined to be unreasonable, potentially leading to lower accuracy in the resulting second neural network. Therefore, the determined quantization coefficient cannot be used and further adjustments to the set quantization coefficient are necessary.

[0142] In practical implementation, for example, the reasonableness of the quantization coefficient set in the current iteration cycle can be determined by setting an iteration stopping condition, and this quantization coefficient can be used as the target quantization coefficient. Specifically, when determining the target quantization coefficient using the iteration stopping condition, the following method can be adopted: based on the loss weights corresponding to multiple quantization objects, determine whether the current iteration cycle meets the iteration stopping condition; in response to the current iteration cycle not meeting the iteration stopping condition, proceed to the next iteration cycle; the step of determining the quantization coefficient determined in the last iteration cycle as the target quantization coefficient includes: in response to the current iteration cycle meeting the iteration stopping condition, determining the current iteration cycle as the last iteration cycle, and determining the quantization coefficient of the last iteration cycle as the target quantization coefficient.

[0143] The iteration stopping condition may include, for example, total quantitative loss being less than a preset loss threshold.

[0144] Specifically, the loss threshold can be set to, for example, Ls. Since the input data, network layer parameters, and output data have different requirements for quantization accuracy and their numerical distributions also differ, different loss thresholds can be determined for different types of quantization objects. For example, determining the loss threshold for the input data... Determine the loss threshold for network layer intrinsic parameters And determine the loss threshold for the output data. For any type of quantization object, if the determined total quantization loss is greater than or equal to the corresponding loss threshold, the quantization coefficient determined in the current iteration cycle is considered inaccurate, and the quantization coefficient used in the next iteration cycle can be determined based on the difference between the quantization loss and the corresponding loss threshold. If the determined total quantization loss is less than the loss threshold, the quantization coefficient set in the current iteration cycle is considered to meet the actual quantization requirements, and the quantization coefficient set in the current iteration cycle is taken as the target quantization coefficient.

[0145] For example, in the first iteration cycle, if there is no case where the total quantization loss of any type of quantized object is less than the corresponding loss threshold for the input data, network layer intrinsic parameters, and output data, then the process is determined to proceed to the second iteration cycle.

[0146] When processing the second iteration cycle, the current iteration cycle is changed to the second iteration cycle. For example, based on the processing result of the previous iteration cycle corresponding to the current iteration cycle—that is, the total quantization loss corresponding to different types of quantized objects—the quantization coefficients for each type of quantized object in the current iteration cycle can be determined. For instance, for input data, if the total quantization loss corresponding to the input data exceeds the corresponding loss threshold, and the excess is significant, then when determining the quantization coefficients for the current iteration cycle using the quantization coefficients from the first iteration cycle, the quantization coefficients from the first iteration cycle can be adjusted considerably, and the adjusted quantization coefficients are used as the quantization coefficients for the current iteration cycle. If the excess is small, then when adjusting the quantization coefficients from the first iteration cycle, a smaller adjustment can be made, and the adjusted quantization coefficients are used as the quantization coefficients for the second iteration cycle.

[0147] In the second iteration cycle, having determined the quantization coefficients for multiple quantization objects, the quantization results for each object can be obtained using a method similar to that described in the first iteration cycle. This allows for the further determination of quantization loss and quantization weights. Consequently, the decision to proceed to the next iteration cycle can be made by determining the total quantization loss. If no further iteration cycle is initiated, the current iteration cycle is designated as the last iteration cycle, and the quantization coefficients corresponding to the current iteration cycle are taken as the target quantization coefficients.

[0148] In another embodiment of this disclosure, since the loss threshold may be set too low when initially set manually, even if the total quantitative loss is reduced to a low value through multiple iterations, the iteration stopping condition of the total quantitative loss being less than the preset loss threshold may still not be met; or, the loss threshold may be set too high, resulting in a lower progress of the second neural network after the determined target quantization coefficient is quantized on the first neural network. Therefore, the iteration stopping condition can also be set as follows: the difference between the total quantitative losses corresponding to multiple target iteration cycles is less than a preset difference threshold; wherein, the target iteration cycle includes: the current iteration cycle and at least one historical iteration cycle continuous with the current iteration cycle.

[0149] In this scenario, if after multiple iterations the total quantization loss for the current iteration cycle and at least one consecutive historical iteration cycle fluctuates around a certain value, then continuing to adjust the quantization coefficient in the next iteration cycle would likely have a small adjustment range and would not significantly impact the quantization loss of the quantized object. Therefore, the iteration process can be stopped, and the quantization coefficient corresponding to the last iteration cycle can be used as the target quantization coefficient.

[0150] In another embodiment of this disclosure, when using a computer device to determine the target quantization coefficient, the above iterative process can be completed, for example, using the formula (19) below:

[0151]

[0152] in, This represents the joint search space for the quantization coefficients s1 and z1 included in the first quantization coefficient, within which the total quantization loss is determined. The quantization coefficient and zero-point value at the smallest value. Similarly, This represents the joint search space for the quantization coefficients s2 and z2 included in the second quantization coefficient, within which the total quantization loss is determined. The minimum quantization coefficient and zero-point value. Based on the interconnectivity of the network, the output of the current network layer becomes the input of the next network layer. Therefore, the third quantization coefficient corresponding to the output data is related to the first and second quantization coefficients. Thus, when searching for the third quantization coefficient, the corresponding joint search space and... as well as Related, expressed as

[0153] Alternatively, when using computer equipment to determine the target quantization coefficients, the above iterative process can be completed using, for example, the formula (20) below:

[0154]

[0155] in, This represents the joint search space for the quantization coefficients s3 and z3 included in the third quantization coefficient, within which the total quantization loss is determined. The minimum quantization coefficient and zero-point value. In one possible case, if any of the operations such as quantization or dequantization of the third quantization object are performed using the third quantization coefficient in the above S101 process, a search space can also be determined for the third quantization coefficient to determine the specific value of the third quantization coefficient corresponding to the minimum total loss of the third quantization object.

[0156] Regarding S102 above: After determining the quantization coefficient corresponding to the last iteration cycle and using it as the target quantization coefficient, the first neural network can be quantized using the target quantization coefficient to generate the second neural network.

[0157] In specific implementation, for example, the following method can be adopted: using the target quantization coefficient, the quantization object in the target network layer is quantized to obtain the target quantization object; based on the target quantization object, the second neural network is generated.

[0158] Specifically, by determining the target quantization coefficients corresponding to the quantized objects in the target network layer during the last iteration, it is possible to use these coefficients to minimize the quantization loss of the quantized objects in the target network layer. Therefore, although the precision of the quantized objects is reduced compared to the unquantized objects, the numerical difference is minimal. Thus, using the target quantization coefficients, the quantized objects in the first neural network can be processed to obtain the target quantized objects. Here, the precision of the obtained target quantized objects is reduced, but the numerical difference from the unquantized objects is not significant. Furthermore, they occupy fewer bits, thus requiring less computational power and storage space. Replacing the quantized objects at corresponding positions in the first neural network with the target quantized objects yields the second neural network.

[0159] In another embodiment of this disclosure, the obtained second neural network can be used in a specific application scenario based on the different types of the corresponding first neural network in the corresponding application scenario. For example, when it is determined that the first neural network includes a convolutional neural network and is applied to an image processing application scenario, the second neural network obtained by processing the first neural network using the embodiments of this disclosure can be used for image classification, object recognition, visual positioning, etc. in image processing.

[0160] In practice, the image data to be processed can be input into a neural network generated by quantization using the neural network quantization method disclosed in this application to process the image data and obtain the processing result.

[0161] For example, when the second neural network is applied to object recognition, the second neural network consumes less computing power and storage space when processing multiple images because the quantized objects in the second neural network require less computational power and storage space during operation. Therefore, the efficiency in processing multiple images is reduced compared to the first neural network before quantizing the objects. Furthermore, although the precision of the quantized objects is reduced in the second neural network, the object recognition results obtained after processing the image data using the second neural network can still meet the accuracy requirements for recognition, and thus it is still applicable to different desired application areas.

[0162] Furthermore, since the second neural network requires less computing power and storage space in practical applications, it can be deployed on small processing devices, such as vehicles and mobile devices. This overcomes the limitations and requirements of the first neural network in terms of computing power and storage space, making it suitable for a wider range of applications.

[0163] Those skilled in the art will understand that, in the above-described method of the specific implementation, the order in which each step is written does not imply a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of each step should be determined by its function and possible internal logic.

[0164] Based on the same inventive concept, this disclosure also provides a quantization device for a neural network corresponding to the quantization method of the neural network. Since the principle of the device in this disclosure for solving the problem is similar to the quantization method of the neural network described above in this disclosure, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be described again.

[0165] Reference Figure 2 The diagram shown is a schematic of a quantization device for a neural network according to an embodiment of this disclosure. The device includes a processing module 21 and a quantization module 22; wherein,

[0166] Processing module 21 is configured to perform the following in each of multiple iteration cycles: determining quantization coefficients corresponding to multiple quantization objects when quantizing the target network layer in the first neural network; wherein the quantization coefficients are determined based on the original quantization coefficients, or based on the quantization loss and loss weights determined in the historical iteration cycles; performing quantization operations on the multiple quantization objects based on the quantization coefficients to obtain the quantization results of the multiple quantization objects in the current iteration cycle; determining the quantization loss of the multiple quantization objects in the current iteration cycle based on the quantization results of the multiple quantization objects in the current iteration cycle, and determining the loss weights corresponding to the quantization losses of the multiple quantization objects based on the quantization losses;

[0167] The quantization module 22 is used to determine the quantization coefficient determined in the last iteration cycle as the target quantization coefficient, and use the target quantization coefficient to quantize the first neural network to generate the second neural network.

[0168] In one optional implementation, the quantization object includes at least one of the following: the input data, output data, and intrinsic parameters of the target network layer.

[0169] In one optional implementation, the quantization object includes the intrinsic parameters of the target network layer; when the processing module 21 determines that the target network layer in the first neural network is quantized in the current iteration cycle, and the quantization coefficients corresponding to multiple quantization objects are determined, the module 21 is configured to: in response to the current iteration cycle being the first iteration cycle, determine the original quantization coefficients of the intrinsic parameters of the network layer based on a preset quantization coefficient space; and determine the quantization coefficients of the current iteration cycle based on the original quantization coefficients of the intrinsic parameters of the network layer.

[0170] In one optional implementation, the quantization object includes: input data of the target network layer and / or output data of the target network layer; when the processing module 21 determines that the target network layer in the first neural network is quantized in the current iteration cycle, and the quantization coefficients corresponding to the multiple quantization objects are determined, the processing module 21 is configured to: in response to the current iteration cycle being the first iteration cycle, determine first distribution feature information of the input data based on the input data; determine the original quantization coefficients of the input data based on the first distribution feature information; and / or, in response to the current iteration cycle being the first iteration cycle, determine second distribution feature information of the output data based on the output data; determine the original quantization coefficients of the output data based on the second distribution feature information; and determine the quantization coefficients of the current iteration cycle based on the original quantization coefficients of the input data and / or the original quantization coefficients of the output data.

[0171] In one optional implementation, the quantization object includes the input data, intrinsic parameters, and output data of the target network layer; the quantization coefficient includes a first quantization coefficient of the input data and a second quantization coefficient of the intrinsic parameters; when the processing module 21 performs quantization operations on the plurality of quantization objects based on the quantization coefficients to obtain quantization results of the plurality of quantization objects corresponding to the current iteration cycle, it is configured to: for the first quantization object of type input data, perform quantization operations on the first quantization object based on the first quantization coefficient to obtain a first quantization result of the first quantization object in the current iteration cycle; for the second quantization object of type network layer intrinsic parameters, perform quantization operations on the second quantization object based on the second quantization coefficient to obtain a second quantization result of the second quantization object in the current iteration cycle; for the third quantization object of type output data, perform quantization operations on the third quantization object based on the third quantization coefficient to obtain a third quantization result of the third quantization object in the current iteration cycle; or, determine the third quantization result of the third quantization object corresponding to the current iteration cycle based on the first quantization result and the second quantization result.

[0172] In one optional implementation, when the processing module 21 determines the quantization loss of the plurality of quantization objects in the current iteration period based on the quantization results of the plurality of quantization objects in the current iteration period, it is configured to: for each of the plurality of quantization objects, perform an inverse quantization operation on the quantization result of each quantization object in the current iteration period based on the quantization coefficient, to obtain the inverse quantization result corresponding to each quantization object; and obtain the quantization loss of each quantization object in the current iteration period based on each quantization object and the corresponding inverse quantization result.

[0173] In one optional implementation, when determining the loss weights corresponding to the quantization losses of the plurality of quantization objects based on the quantization loss, the processing module 21 is configured to: normalize the quantization losses corresponding to the quantization losses of the quantization objects belonging to the same type to obtain the loss weights corresponding to the quantization losses of the quantization objects belonging to the same type; or, process the quantization objects belonging to the same type based on a preset monotonic function to obtain the loss weights corresponding to the quantization losses of the quantization objects belonging to the same type; or, use a pre-trained weight prediction neural network to perform weight prediction processing on the quantization objects belonging to the same type to obtain the loss weights corresponding to the quantization losses of the quantization objects belonging to the same type; or, for each quantization object, determine the target value interval to which the quantization loss of each quantization object belongs among multiple value intervals; and determine the loss weight corresponding to the quantization loss of each quantization object based on the weights corresponding to the target value intervals.

[0174] In one optional implementation, after determining the quantization loss corresponding to each of the plurality of quantization objects in the current iteration cycle based on the quantization results corresponding to each of the plurality of quantization objects in the current iteration cycle, and determining the loss weight corresponding to the quantization loss of each of the plurality of quantization objects based on the quantization loss, the processing module 21 is further configured to: determine whether the current iteration cycle meets the iteration stop condition based on the loss weight corresponding to each of the plurality of quantization objects; and enter the next iteration cycle in response to the current iteration cycle not meeting the iteration stop condition; when the processing module 21 determines the quantization coefficient determined in the last iteration cycle as the target quantization coefficient, it is configured to: determine the current iteration cycle as the last iteration cycle in response to the current iteration cycle meeting the iteration stop condition, and determine the quantization coefficient of the last iteration cycle as the target quantization coefficient.

[0175] In one optional implementation, the iteration stopping condition includes at least one of the following: the total quantization loss is less than a preset loss threshold; the total quantization loss is determined based on the loss weights corresponding to the plurality of quantization objects respectively; the difference between the total quantization losses corresponding to the plurality of target iteration cycles is less than a preset difference threshold; wherein, the target iteration cycle includes: the current iteration cycle and at least one historical iteration cycle that is continuous with the current iteration cycle.

[0176] In one optional implementation, when the quantization module 22 quantizes the first neural network using the target quantization coefficient to generate the second neural network, it is configured to: quantize the quantization object in the target network layer using the target quantization coefficient to obtain the target quantization object; and generate the second neural network based on the target quantization object.

[0177] The processing flow of each module in the device and the interaction flow between each module can be referred to the relevant descriptions in the above method embodiments, and will not be detailed here.

[0178] This disclosure also provides a computer device, such as... Figure 3 The diagram shown is a schematic representation of a computer device structure provided in an embodiment of this disclosure, including:

[0179] Processor 10 and memory 20; the memory 20 stores machine-readable instructions executable by processor 10, and processor 10 executes the machine-readable instructions stored in memory 20. When the machine-readable instructions are executed by processor 10, processor 10 performs the following steps:

[0180] In each of the multiple iteration cycles, the following steps are performed: determining the quantization coefficients for a target network layer in the first neural network, and the quantization coefficients for multiple quantization objects; wherein the quantization coefficients are determined based on the original quantization coefficients, or based on the quantization loss and loss weights determined in the previous iteration cycles; performing quantization operations on the multiple quantization objects based on the quantization coefficients to obtain the quantization results for the multiple quantization objects in the current iteration cycle; determining the quantization loss for the multiple quantization objects in the current iteration cycle based on the quantization results for the multiple quantization objects, and determining the loss weights corresponding to the quantization losses for the multiple quantization objects based on the quantization losses; and determining the quantization coefficients determined in the last iteration cycle as the target quantization coefficients, and using the target quantization coefficients to quantize the first neural network to generate a second neural network.

[0181] The aforementioned memory 20 includes a main memory 210 and an external memory 220; the main memory 210, also known as internal memory, is used to temporarily store the computational data in the processor 10, as well as the data exchanged with external memory 220 such as a hard disk. The processor 10 exchanges data with the external memory 220 through the main memory 210.

[0182] The specific execution process of the above instructions can be referred to the steps of the quantization method of the neural network described in the embodiments of this disclosure, and will not be repeated here.

[0183] This disclosure also provides a computer-readable storage medium storing a computer program that, when executed by a processor, performs the steps of the neural network quantization method described in the above method embodiments. The storage medium can be a volatile or non-volatile computer-readable storage medium.

[0184] This disclosure also provides a computer program product carrying program code. The program code includes instructions that can be used to execute the steps of the neural network quantization method described in the above method embodiments. For details, please refer to the above method embodiments, which will not be repeated here.

[0185] The aforementioned computer program product can be implemented through hardware, software, or a combination thereof. In one optional embodiment, the computer program product is specifically embodied in a computer storage medium; in another optional embodiment, the computer program product is specifically embodied in a software product, such as a software development kit (SDK), etc.

[0186] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems and devices described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here. In the several embodiments provided in this disclosure, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division; in actual implementation, there may be other division methods. Furthermore, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Another point is that the displayed or discussed mutual coupling or direct coupling or communication connection may be through some communication interfaces; the indirect coupling or communication connection of devices or units may be electrical, mechanical, or other forms.

[0187] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0188] In addition, the functional units in the various embodiments of this disclosure can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0189] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a processor-executable, non-volatile, computer-readable storage medium. Based on this understanding, the technical solution of this disclosure, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this disclosure. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0190] Finally, it should be noted that the above-described embodiments are merely specific implementations of this disclosure, used to illustrate the technical solutions of this disclosure, and not to limit it. The protection scope of this disclosure is not limited thereto. Although this disclosure has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can still modify or easily conceive of changes to the technical solutions described in the foregoing embodiments, or make equivalent substitutions for some of the technical features, within the scope of the technology disclosed in this disclosure. Such modifications, changes, or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this disclosure, and should all be covered within the protection scope of this disclosure. Therefore, the protection scope of this disclosure should be determined by the protection scope of the claims.

Claims

1. A method for quantizing neural networks, characterized in that, include: Execute in each of the multiple iteration cycles: The current iteration cycle is used to quantize the target network layer in the first neural network, and the quantization coefficients are corresponding to multiple quantization objects; wherein, the quantization coefficients are determined based on the original quantization coefficients, or based on the quantization loss and loss weights determined in the historical iteration cycle; Based on the quantization coefficient, quantization operations are performed on the plurality of quantization objects respectively to obtain the quantization results of the plurality of quantization objects in the current iteration cycle; Based on the quantization results of the multiple quantization objects in the current iteration cycle, the quantization loss of the multiple quantization objects in the current iteration cycle is determined, and based on the quantization loss, the loss weight corresponding to the quantization loss of the multiple quantization objects is determined. The step of determining the quantization loss of each of the multiple quantization objects in the current iteration cycle based on their respective quantization results in the current iteration cycle, and determining the loss weight corresponding to the quantization loss of each of the multiple quantization objects based on the quantization loss, includes: For each of the multiple quantization objects, based on the quantization coefficient, an inverse quantization operation is performed on the quantization result corresponding to each quantization object in the current iteration cycle to obtain the inverse quantization result corresponding to each quantization object; Based on each quantization object and its corresponding inverse quantization result, the quantization loss of each quantization object in the current iteration cycle is obtained; Based on the quantization loss of each quantization object in the current iteration cycle, a corresponding loss weight is determined for each quantization object; The process of determining the quantization loss of each of the multiple quantization objects in the current iteration cycle based on their respective quantization results in the current iteration cycle, and determining the loss weights corresponding to the quantization losses of each of the multiple quantization objects based on their respective quantization losses, further includes: Based on the loss weights corresponding to multiple quantized objects, determine whether the current iteration cycle meets the iteration stop condition; If the iteration stopping condition is not met in the current iteration cycle, proceed to the next iteration cycle; The quantization coefficients determined in the last iteration are used as the target quantization coefficients, including: In response to the current iteration cycle satisfying the iteration stop condition, the current iteration cycle is determined as the last iteration cycle, and the quantization coefficient of the last iteration cycle is determined as the target quantization coefficient; The quantization object includes the input data and output data of the target network layer. The input data includes the original image or a feature map output by the preceding network layer of the target network layer. The output data is another feature map obtained by processing the input data. Furthermore, the first neural network is quantized using the target quantization coefficient to generate a second neural network, and the second neural network is stored. The second neural network is suitable for deployment on mobile devices.

2. The quantization method according to claim 1, characterized in that, The quantization object also includes the network layer intrinsic parameters of the target network layer; The determination of the current iteration period for quantizing the target network layer in the first neural network, and the quantization coefficients corresponding to multiple quantization objects, include: In response to the current iteration cycle being the first iteration cycle, the original quantization coefficients of the network layer intrinsic parameters are determined based on a preset quantization coefficient space. The quantization coefficients for the current iteration cycle are determined based on the original quantization coefficients of the network layer intrinsic parameters.

3. The quantization method according to claim 2, characterized in that, The quantization object includes: the input data of the target network layer and / or the output data of the target network layer; The determination of the current iteration period for quantizing the target network layer in the first neural network, and the quantization coefficients corresponding to multiple quantization objects, include: In response to the current iteration period being the first iteration period, based on the input data, a first distribution feature information of the input data is determined; based on the first distribution feature information, the original quantization coefficients of the input data are determined; and / or, in response to the current iteration period being the first iteration period, based on the output data, a second distribution feature information of the output data is determined; based on the second distribution feature information, the original quantization coefficients of the output data are determined. The quantization coefficients for the current iteration period are determined based on the original quantization coefficients of the input data and / or the original quantization coefficients of the output data.

4. The quantization method according to claim 3, characterized in that, The quantization object includes the input data, intrinsic parameters, and output data of the target network layer; The quantization coefficients include: a first quantization coefficient for the input data and a second quantization coefficient for the network layer intrinsic parameters; The step of performing quantization operations on the plurality of quantization objects based on the quantization coefficients to obtain the quantization results of the plurality of quantization objects in the current iteration cycle includes: For a first quantized object whose type is input data, a quantization operation is performed on the first quantized object based on a first quantization coefficient to obtain the first quantization result of the first quantized object in the current iteration cycle; For the second quantization object, which is a network layer intrinsic parameter, a quantization operation is performed on the second quantization object based on the second quantization coefficient to obtain the second quantization result of the second quantization object in the current iteration cycle; For a third quantized object whose type is output data, a quantization operation is performed on the third quantized object based on the third quantization coefficient to obtain the third quantization result of the third quantized object in the current iteration cycle; or, based on the first quantization result and the second quantization result, the third quantization result corresponding to the third quantized object in the current iteration cycle is determined.

5. The quantization method according to claim 4, characterized in that, The step of determining the loss weights corresponding to the quantization losses of the plurality of quantization objects based on the quantization loss includes: Normalize the quantization losses corresponding to quantization objects of the same type to obtain the loss weights corresponding to the quantization losses of the same type of quantization objects. or, Based on a preset monotonic function, quantization objects of the same type are processed separately to obtain the loss weights corresponding to the quantization loss of the quantization objects of the same type. or, A pre-trained weight prediction neural network is used to perform weight prediction processing on quantization objects of the same type, thereby obtaining the loss weights corresponding to the quantization loss of the quantization objects of the same type. or, For each quantization object, determine the target value range to which the quantization loss of each quantization object belongs among multiple value ranges; based on the weight corresponding to the target value range, determine the loss weight corresponding to the quantization loss of each quantization object.

6. The quantization method according to claim 1, characterized in that, The iteration stopping condition includes at least one of the following: The total quantitative loss is less than a preset loss threshold; the total quantitative loss is determined based on the loss weights corresponding to the multiple quantized objects respectively; The difference between the total quantitative loss corresponding to multiple target iteration cycles is less than a preset difference threshold; wherein, the target iteration cycle includes: the current iteration cycle and at least one historical iteration cycle that is continuous with the current iteration cycle.

7. The quantization method according to any one of claims 1-6, characterized in that, The step of quantizing the first neural network using the target quantization coefficient to generate the second neural network includes: Using the target quantization coefficients, the quantization object in the target network layer is quantized to obtain the target quantization object; The second neural network is generated based on the target quantization object.

8. A quantization device for a neural network, characterized in that, include: A processing module is configured to execute at least one of the following iteration cycles, and in each iteration cycle, perform the following: determine quantization coefficients corresponding to multiple quantization objects when quantizing a target network layer in a first neural network; wherein the quantization coefficients are determined based on original quantization coefficients, or based on quantization loss and loss weights determined in a previous iteration cycle; perform quantization operations on the multiple quantization objects based on the quantization coefficients to obtain quantization results corresponding to the multiple quantization objects respectively; determine quantization losses corresponding to the multiple quantization objects based on the quantization results corresponding to the multiple quantization objects respectively, and determine loss weights corresponding to the quantization losses of the multiple quantization objects respectively based on the quantization losses; wherein the quantization objects include input data and output data of the target network layer, the input data includes an original image, or a feature map output by the previous network layer of the target network layer; the output data is another feature map obtained by processing the input data; the processing module determines the quantization loss corresponding to the multiple quantization objects in the current iteration cycle based on the quantization results corresponding to the multiple quantization objects respectively, and determines the loss weights corresponding to the quantization losses of the multiple quantization objects respectively based on the quantization losses, for: targeting multiple quantization objects For each quantized object in the image, based on the quantization coefficient, the quantization result corresponding to each quantized object in the current iteration cycle is dequantized to obtain the dequantization result corresponding to each quantized object; based on each quantized object and the corresponding dequantization result, the quantization loss of each quantized object in the current iteration cycle is obtained; based on the quantization loss of each quantized object in the current iteration cycle, a corresponding loss weight is determined for each quantized object; after determining the quantization loss corresponding to each of the multiple quantized objects in the current iteration cycle based on the quantization result corresponding to each of the multiple quantized objects in the current iteration cycle, and determining the loss weight corresponding to the quantization loss of each of the multiple quantized objects based on the quantization loss, the processing module is further configured to: determine whether the current iteration cycle meets the iteration stop condition based on the loss weight corresponding to each of the multiple quantized objects; in response to the current iteration cycle not meeting the iteration stop condition, proceed to the next iteration cycle; when the processing module determines the quantization coefficient determined in the last iteration cycle as the target quantization coefficient, it is configured to: in response to the current iteration cycle meeting the iteration stop condition, determine the current iteration cycle as the last iteration cycle, and determine the quantization coefficient of the last iteration cycle as the target quantization coefficient; A quantization module is used to quantize the first neural network using the target quantization coefficient to generate a second neural network and store the second neural network; wherein the second neural network is suitable for deployment on a mobile device.

9. A computer device, characterized in that, include: A processor and a memory, the memory storing machine-readable instructions executable by the processor, the processor executing the machine-readable instructions stored in the memory, wherein when the machine-readable instructions are executed by the processor, the processor performs the steps of the quantization method of the neural network as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, which, when executed by a computer device, performs the steps of the quantization method for a neural network as described in any one of claims 1 to 7.

Images