Model quantization method, electronic device and computer-readable storage medium

Abstract

The present application discloses a model quantization method, an electronic device, and a computer-readable storage medium, the method comprising: obtaining candidate benchmark data and its corresponding data distribution, selecting at least part of the candidate benchmark data as reference benchmark data based on the data distribution; decomposing the model to be quantized into multiple sub-networks and obtaining the network information of the sub-networks, using the network information to index the quantization parameters in the knowledge base, and determining the sub-network indexed to the quantization parameters; wherein the knowledge base stores the network information and quantization parameters of the quantized sub-networks; in response to at least part of the sub-networks not being indexed to the quantization parameters, using the reference benchmark data to quantize the sub-networks that are not indexed to the quantization parameters; obtaining the quantized model, using the reference benchmark data to verify the quantized model, and obtaining the quantized model after the verification conditions are met, and updating the knowledge base. The above scheme can take into account both the model quantization efficiency and the effect of the quantized model.

Description

Technical Field

[0001] The present application relates to the technical field of model quantization, and in particular to a model quantization method, an electronic device, and a computer-readable storage medium. Background Art

[0002] With the widespread application of deep learning models, model quantization has received more and more attention as an important means to achieve lightweight models. Model quantization can reduce the size of the model and reduce power consumption to facilitate model deployment. Model quantization needs to consider model quantization efficiency and quantization effect. However, the existing model quantization methods will lead to poor quantization effect if they focus on improving quantization efficiency, and will lead to low quantization efficiency if they focus on quantization effect. In view of this, how to balance the model quantization efficiency and the effect of the quantized model has become an urgent problem to be solved. Summary of the invention

[0003] The main technical problem solved by the present application is to provide a model quantization method, an electronic device and a computer-readable storage medium, which can take into account both the model quantization efficiency and the effect of the quantized model.

[0004] To solve the above technical problems, the first aspect of the present application provides a model quantization method, which includes: obtaining candidate benchmark data and their corresponding data distribution, and selecting at least part of the candidate benchmark data as reference benchmark data based on the data distribution; decomposing the model to be quantized into multiple sub-networks and obtaining network information of the sub-networks, using the network information to index quantization parameters in a knowledge base, and determining the sub-network indexed to the quantization parameters; wherein the knowledge base stores the network information and quantization parameters of the quantized sub-networks; in response to at least part of the sub-networks not being indexed to the quantization parameters, using the reference benchmark data to quantize the sub-networks not being indexed to the quantization parameters; obtaining the quantized model, verifying the quantized model using the reference benchmark data, and obtaining the quantized model after the verification conditions are met, and updating the knowledge base.

[0005] To solve the above technical problems, the second aspect of the present application provides an electronic device, which includes: a memory and a processor coupled to each other, wherein the memory stores program data, and the processor calls the program data to execute the method described in the first aspect.

[0006] In order to solve the above technical problem, the third aspect of the present application provides a computer-readable storage medium on which program data is stored. When the program data is executed by a processor, the method described in the first aspect is implemented.

[0007] The above scheme obtains candidate benchmark data and data distribution corresponding to the candidate benchmark data, selects at least part of the candidate benchmark data as reference benchmark data from all candidate benchmark data based on the data distribution, and realizes the conversion of large batch data to small batch data. The model to be quantized is decomposed into multiple sub-networks, the network information of each sub-network is obtained, and the quantization parameters of the sub-network are indexed in the knowledge base using the network information, and the sub-network that can index the quantization parameters is determined, wherein the knowledge base stores the network information and quantization parameters of the quantized sub-network, so as to facilitate the reuse of the quantization parameters of the sub-networks with consistent network information matching, thereby improving the quantization efficiency. When at least part of the sub-networks are not indexed to the quantization parameters, the sub-networks that are not indexed to the quantization parameters are quantized using the reference benchmark data, so that the quantization parameters are first indexed in the knowledge base, and then the sub-networks that are not indexed to the quantization parameters are quantized, and the quantization parameters of each sub-network are determined step by step, and the sub-networks that are not indexed to the quantization parameters are individually quantized to improve the accuracy of sub-network quantization and enhance the effect of model quantization. The model is quantized based on the quantization parameters of the sub-network to obtain the quantized model. The quantized model is verified using the reference benchmark data to determine whether the verification conditions are met to verify the quantization effect. After the verification conditions are met, the quantized model is obtained and the knowledge base is updated. Therefore, the knowledge base stores the network information and quantization parameters of the quantized sub-network for easy reuse after indexing. The quantization parameters of the sub-network are quantized using the reference benchmark data. The reference benchmark data is selected based on the data distribution and can select more suitable data in the model deployment scenario. It can not only improve the efficiency of quantization and verification using the reference benchmark data, but also ensure the accuracy of the model when the model is adapted to the scenario, and obtain more suitable quantization parameters to reduce inference time, taking into account both the model quantization efficiency and the effect of the quantized model. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. Among them:

[0009] Figure 1 It is a flow chart of an implementation method of the model quantification method of the present application;

[0010] Figure 2 It is a flow chart of another implementation method of the model quantification method of the present application;

[0011] Figure 3 This is a schematic diagram of an application scenario of an implementation method of data distribution of this application;

[0012] Figure 4 It is a structural schematic diagram of an embodiment of the electronic device of the present application;

[0013] Figure 5 It is a structural schematic diagram of an implementation method of a computer-readable storage medium of the present application. DETAILED DESCRIPTION

[0014] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments, and different implementation methods can be adaptively combined. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0015] The terms "system" and "network" are often used interchangeably in this article. The term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship. In addition, "many" in this article means two or more than two.

[0016] The model quantization method provided in this application is used to quantify the model, and its corresponding execution subject is a processing terminal that can call the model.

[0017] See also Figure 1 , Figure 1 It is a flow chart of an implementation method of the model quantification method of the present application, and the method comprises:

[0018] S101: Acquire candidate benchmark data and their corresponding data distribution, and select at least part of the candidate benchmark data as reference benchmark data based on the data distribution.

[0019] Specifically, candidate benchmark data and data distribution corresponding to the candidate benchmark data are obtained, and at least part of the candidate benchmark data is selected from all candidate benchmark data as reference benchmark data based on the data distribution to achieve conversion of large batch data into small batch data.

[0020] It can be understood that the candidate benchmark data is collected from the scene where the model is to be deployed, and the candidate benchmark data is adapted to the scene. The data that needs to be collected in the specific scene can be at least one modality of image, video and audio.

[0021] In some implementation scenarios, candidate benchmark data and their data distribution are obtained, and the data interval with the largest number of data is screened out from the data distribution using a preset data interval length. Random sampling is performed in the data interval with the largest number of data to obtain some candidate benchmark data as reference benchmark data.

[0022] In some implementation scenarios, candidate benchmark data and data distribution of the candidate benchmark data are obtained, the data distribution is divided into multiple data intervals, and the distribution peak value corresponding to the data distribution is determined, and based on the difference between the median value of the data interval and the distribution peak value, part of the candidate benchmark data is selected from the multiple data intervals as reference benchmark data. The smaller the difference, the more candidate benchmark data is selected in the corresponding data interval.

[0023] It should be noted that the number of reference benchmark data needs to be greater than the preset total threshold. If the number of selected candidate benchmark data is less than the preset total threshold, the value range can be expanded for further selection. When the total number of candidate benchmark data is less than the preset total threshold, all candidate benchmark data can be selected as reference benchmark data.

[0024] S102: Decomposing the model to be quantized into multiple sub-networks and obtaining network information of the sub-networks, using the network information to index quantization parameters in a knowledge base, and determining the sub-network indexed to the quantization parameters; wherein the knowledge base stores the network information and quantization parameters of the sub-networks that have been quantized.

[0025] Specifically, the model to be quantized is decomposed into multiple sub-networks, the network information of each sub-network is obtained, the quantization parameters of the sub-network are indexed in the knowledge base using the network information, and the sub-network that can index the quantization parameters is determined.

[0026] It can be understood that the knowledge base stores the network information and quantization parameters of the quantized sub-networks, so as to facilitate the reuse of the quantization parameters of the sub-networks with consistent network information matching, thereby improving the quantization efficiency.

[0027] It should be noted that model quantization usually converts the floating-point algorithm of the neural network into fixed-point. The quantization parameter can characterize the conversion relationship between the parameters before quantization and the parameters after quantization in the network. The quantization parameter corresponds to the quantization bit width. A smaller quantization bit width can improve the compression rate of the model, and a larger quantization bit width can improve the accuracy of the model.

[0028] In some implementation scenarios, the model to be quantized is decomposed into multiple sub-networks, and the network information of each sub-network is obtained, wherein the network information is related to the network structure of the sub-network, and the similarity between the sub-network to be quantized and the network information of the sub-network that has been quantized in the knowledge base is determined, and the quantization parameters of the quantized sub-network that are greater than the similarity threshold and have the greatest similarity are used as the quantization parameters of the corresponding sub-network to be quantized, and the sub-network indexed to the quantization parameters is determined.

[0029] In some implementation scenarios, the model to be quantized is decomposed into multiple sub-networks, and the network information of each sub-network is obtained, wherein the network information is related to the network parameters in the sub-network. Sub-network matching is performed based on the network information to determine whether a matching quantized sub-network is obtained from the knowledge base. The quantization parameters of the matching quantized sub-network are used as the quantization parameters of the corresponding sub-network to be quantized, and the sub-network indexed to the quantization parameters is determined.

[0030] S103: In response to at least part of the sub-networks not being indexed to quantization parameters, quantize the sub-networks not being indexed to quantization parameters using reference benchmark data.

[0031] Specifically, when at least part of the sub-networks do not index the quantization parameters, the sub-networks that do not index the quantization parameters are quantized using the reference benchmark data.

[0032] It can be understood that the quantization parameters are first indexed in the knowledge base, and then the sub-networks that have not been indexed to the quantization parameters are quantized, and the quantization parameters of each sub-network are determined step by step. The sub-networks that have not been indexed to the quantization parameters are quantized separately to improve the accuracy of sub-network quantization and enhance the effect of the model after quantization.

[0033] Optionally, the sub-networks that are not indexed to the quantization parameters are quantized based on the quantization algorithm and the reference benchmark data to determine the quantization parameters of the corresponding sub-networks. The quantization algorithm includes but is not limited to a binarization algorithm, a linear quantization algorithm, and a logarithmic quantization algorithm.

[0034] S104: Obtain the quantized model, verify the quantized model using reference benchmark data, obtain the quantized model after the verification conditions are met, and update the knowledge base.

[0035] Specifically, the model is quantized based on the quantization parameters of the sub-network to obtain a quantized model, and the quantized model is verified using reference benchmark data to determine whether the verification conditions are met to verify the quantization effect. After the verification conditions are met, the quantized model is obtained and the knowledge base is updated.

[0036] In some implementation scenarios, reference benchmark data is input into the model before quantization to obtain a first output result, and the reference benchmark data is input into the model after quantization to obtain a second output result, and an estimated loss of the second output result compared to the first output result is determined, and whether a verification condition is met is determined based on the estimated loss, wherein, when the estimated loss converges, the network information and quantization parameters of the sub-network that are not indexed to the quantization parameters are updated to the knowledge base.

[0037] In some implementation scenarios, reference benchmark data is input into a model before quantization to obtain a first output result, and reference benchmark data is input into a model after quantization to obtain a second output result and an output response time, and an estimated loss of the second output result compared to the first output result is determined, and whether a verification condition is met is determined based on the estimated loss and the output response time, wherein, when the estimated loss converges and the output response time is less than a time threshold, the network information and quantization parameters of the sub-network that are not indexed to the quantization parameters are updated to the knowledge base.

[0038] It can be understood that the knowledge base stores the network information and quantization parameters of the quantized sub-network for easy reuse after indexing. The quantization parameters of the sub-network are obtained by quantization using reference benchmark data. The reference benchmark data is selected based on data distribution, and more suitable data can be selected in the model deployment scenario. This can not only improve the efficiency of quantization and verification using reference benchmark data, but also ensure the accuracy of the model when the model is adapted to the scenario, as well as obtain more suitable quantization parameters to reduce inference time, taking into account both the model quantization efficiency and the effect of the quantized model.

[0039] The above scheme obtains candidate benchmark data and data distribution corresponding to the candidate benchmark data, selects at least part of the candidate benchmark data as reference benchmark data from all candidate benchmark data based on the data distribution, and realizes the conversion of large batch data to small batch data. The model to be quantized is decomposed into multiple sub-networks, the network information of each sub-network is obtained, and the quantization parameters of the sub-network are indexed in the knowledge base using the network information, and the sub-network that can index the quantization parameters is determined, wherein the knowledge base stores the network information and quantization parameters of the quantized sub-network, so as to facilitate the reuse of the quantization parameters of the sub-networks with consistent network information matching, thereby improving the quantization efficiency. When at least part of the sub-networks are not indexed to the quantization parameters, the sub-networks that are not indexed to the quantization parameters are quantized using the reference benchmark data, so that the quantization parameters are first indexed in the knowledge base, and then the sub-networks that are not indexed to the quantization parameters are quantized, and the quantization parameters of each sub-network are determined step by step, and the sub-networks that are not indexed to the quantization parameters are individually quantized to improve the accuracy of sub-network quantization and enhance the effect of model quantization. The model is quantized based on the quantization parameters of the sub-network to obtain the quantized model. The quantized model is verified using the reference benchmark data to determine whether the verification conditions are met to verify the quantization effect. After the verification conditions are met, the quantized model is obtained and the knowledge base is updated. Therefore, the knowledge base stores the network information and quantization parameters of the quantized sub-network for easy reuse after indexing. The quantization parameters of the sub-network are quantized using the reference benchmark data. The reference benchmark data is selected based on the data distribution and can select more suitable data in the model deployment scenario. It can not only improve the efficiency of quantization and verification using the reference benchmark data, but also ensure the accuracy of the model when the model is adapted to the scenario, and obtain more suitable quantization parameters to reduce inference time, taking into account both the model quantization efficiency and the effect of the quantized model.

[0040] See also Figure 2 , Figure 2 : is a flow chart of another embodiment of the model quantification method of the present application, the method comprising:

[0041] S201: Acquire candidate benchmark data and their corresponding data distribution, and select at least part of the candidate benchmark data as reference benchmark data based on the data distribution.

[0042] Specifically, candidate benchmark data and data distribution corresponding to the candidate benchmark data are obtained, and at least part of the candidate benchmark data is selected from all candidate benchmark data as reference benchmark data based on the data distribution to achieve conversion of large batch data into small batch data.

[0043] It should be noted that the candidate reference data correspond to the same data format. Taking the candidate reference data as image data as an example, the image data have the same image size and adopt the same encoding method.

[0044] Furthermore, the data distribution status can provide feedback on the distribution trend of data in the scenario where the model is deployed, and the data distribution status is related to the data collected in the scenario.

[0045] In some implementation scenarios, candidate benchmark data and their corresponding data distribution are obtained to determine the distribution peak value corresponding to the data distribution; based on the distribution peak value in the data distribution and the distribution value within a preset range, at least part of the candidate benchmark data is selected from all candidate benchmark data as reference benchmark data.

[0046] Specifically, candidate benchmark data and their corresponding data distribution are obtained, a distribution peak is determined from the data distribution, a distribution value within a preset range is determined in the data distribution with the distribution peak as the center, and based on the distribution value within the preset range, at least part of the candidate benchmark data is selected from the candidate benchmark data as reference benchmark data.

[0047] It is understandable that the distribution peak can reflect the maximum value of the data distribution, and the data close to the distribution peak can reflect the overall distribution of the data. Based on the distribution peak in the data distribution and the distribution values ​​within a preset range, the selected reference benchmark data can be more adapted to the model deployment scenario. Quantification and verification with small batches of reference benchmark data can effectively improve efficiency and reduce the probability of a significant drop in accuracy due to a small amount of data.

[0048] In a specific implementation scenario, the preset range includes multiple value intervals, and the value intervals are determined based on the distance of the distribution value relative to the distribution peak.

[0049] Specifically, the preset range is provided with a plurality of value intervals for taking values ​​from different value intervals, and the value intervals are determined according to the distance of the distribution value relative to the distribution peak value.

[0050] Furthermore, based on the distribution peak in the data distribution and the distribution values ​​within a preset range, at least part of the candidate benchmark data is selected from all the candidate benchmark data as reference benchmark data, including: based on the distribution peak in the data distribution, at least part of the candidate benchmark data is selected from multiple value intervals in turn as reference benchmark data; wherein the number of candidate benchmark data selected within the value interval is negatively correlated with the distance of the distribution value relative to the distribution peak.

[0051] For illustration purposes, see Figure 3 , Figure 3 This is a schematic diagram of an application scenario of an implementation method of data distribution of this application, wherein: Figure 3 The position indicated by the middle dotted line is the distribution peak, and at least part of the candidate benchmark data is selected from multiple value intervals as reference benchmark data in turn, wherein the farther the value interval is from the distribution peak, the fewer the number of candidate benchmark data selected.

[0052] Optionally, candidate benchmark data in the range of 0~1t, 1t~2t, and 2t~3t before and after the distribution peak are selected respectively, and distributed in a ratio of 3:2:1 to obtain a small batch of reference benchmark data, so that the reference benchmark data can be concentrated before and after the distribution peak, making the reference benchmark data more representative. Wherein, t is a preset distribution value, the specific value of t can be customized, the number of value intervals and the ratio of values ​​can be customized, and this application does not impose specific restrictions on this.

[0053] S202: Decomposing the model to be quantized into multiple sub-networks and obtaining network information of the sub-networks, using the network information to index quantization parameters in a knowledge base, and determining the sub-network indexed to the quantization parameters; wherein the knowledge base stores the network information and quantization parameters of the sub-networks that have been quantized.

[0054] Specifically, the network information includes the network type and network parameters of the sub-network. The model to be quantized is decomposed into multiple sub-networks, the network type and network parameters of the sub-network are obtained as the network information, and the network information is used to index the quantization parameters of the sub-network in the knowledge base to determine the sub-network that can index the quantization parameters.

[0055] In some implementation scenarios, the model to be quantized is decomposed into multiple sub-networks and the network type and network parameters of each sub-network are obtained to obtain the network information of each sub-network, and an index identifier corresponding to the network information is generated; the index identifier is used to index the quantization parameters in the knowledge base to determine the sub-network matched by the index identifier and the quantization parameters of the sub-network in the knowledge base; wherein the network information stored in the knowledge base corresponds to the index identifier.

[0056] Specifically, the model is decomposed into multiple sub-networks and the network type and network parameters of each sub-network are obtained, the network type and network parameters of the sub-network are used as the network information of the sub-network, and an index identifier is generated based on the network information.

[0057] Optionally, the network type and network parameters are cached in the form of key-value pairs, so that the network type and network parameters are extracted in sequence to generate index identifiers, wherein the network information stored in the knowledge base corresponds to the index identifier, thereby reducing the consumption of storage space.

[0058] Furthermore, the quantization parameter is indexed in the knowledge base using the index identifier, the sub-network matched by the index identifier is determined, and the quantization parameter of the corresponding sub-network that has been quantized is obtained from the knowledge base, so that the sub-network indexed to the quantization parameter can be determined by a single indexing through the index identifier, thereby improving the indexing efficiency.

[0059] It should be noted that the model to be quantified is decomposed into multiple sub-networks and the network type and network parameters of each sub-network are obtained to obtain the network information of each sub-network, and an index identifier corresponding to the network information is generated, including: decomposing the model to be quantified into multiple sub-networks, and determining the network type of the sub-network based on the network structure of the sub-network; obtaining the minimum parameter value, the maximum parameter value and the parameter range under the network structure as the network parameters of the sub-network, and obtaining the network information of each sub-network; wherein the parameter range includes a preset proportion of parameter values ​​under the network structure; encoding the network information and generating an index identifier corresponding to the network information.

[0060] Specifically, the model to be quantized is decomposed into multiple sub-networks in layers, the network structure of the sub-network is determined, and the network type of the sub-network is divided based on the network structure to obtain different types of sub-networks.

[0061] Furthermore, the minimum parameter value, the maximum parameter value, and the parameter interval that can cover the preset proportion of parameter values ​​under the network structure are obtained as the network parameters of the sub-network, so that the network parameters can cover the endpoints and interval values ​​of the parameters in the network structure, thereby encoding the network information through a small number of parameter feedback parameter characteristics, generating an index identifier corresponding to the network information, reducing the complexity of generating the index identifier and improving the matching efficiency. Among them, the encoding method of the network information includes but is not limited to the fifth version of the message digest algorithm (Message Digest Algorithm 5, MD5), so as to generate the index identifier.

[0062] Optionally, the network type can correspond to the network serial number, the same network structure corresponds to the same serial number, different network structures have increasing serial numbers, and the network serial number can be cached and called. In addition, the preset ratio used in the parameter interval can be customized to set a specific value, and the network parameters can also include multiple parameter intervals covering different preset ratios, and this application does not make specific restrictions on this.

[0063] In a specific implementation scenario, network types and network parameters are cached in the form of key-value pairs, network parameters are converted into parameter vectors, and the similarity between parameter vectors is determined based on the Euclidean distance between parameter vectors corresponding to the same network type. When the similarity between parameter vectors is greater than a similarity threshold, the corresponding network parameters are merged, so that sub-networks with the same network type and parameter similarity greater than the similarity threshold are classified as the same network, thereby reducing the amount of data of sub-networks stored in the knowledge base and improving indexing efficiency.

[0064] S203: In response to at least part of the sub-networks not indexing quantization parameters, quantize the sub-networks not indexing quantization parameters using reference benchmark data.

[0065] Specifically, when at least part of the sub-networks do not index quantization parameters, the sub-networks that do not index quantization parameters are quantized using reference benchmark data, and the sub-networks that do not index quantization parameters are quantized separately to improve the accuracy of sub-network quantization and enhance the effect of model quantization.

[0066] In some implementation scenarios, a sub-network to which a quantization parameter is not indexed is quantized using reference benchmark data, including: quantizing a sub-network to which a quantization parameter is not indexed using reference benchmark data within a preset quantization time; in response to obtaining a quantization parameter within the preset quantization time, fixing the quantization parameter of the corresponding sub-network; in response to not obtaining a quantization parameter within the preset quantization time, marking the corresponding sub-network as a network to be corrected.

[0067] Specifically, the quantization parameters are first indexed in the knowledge base, and then the sub-networks that have not been indexed to the quantization parameters are quantized, forming a pre-quantization stage. In the pre-quantization stage, the quantization efficiency can be improved by reusing the quantization parameters and determining the quantization parameters of the sub-networks individually. In order to verify the quantization effect of the pre-quantization stage as soon as possible, a preset quantization time is set for the process of quantizing the sub-networks individually, and the sub-networks that have not been indexed to the quantization parameters are quantized using reference benchmark data within the preset quantization time. The quantization process includes starting from the minimum quantization bit width and trying to search for quantization parameters that meet the precision loss requirements.

[0068] Furthermore, it is determined whether the quantization parameters are obtained within the preset quantization time. If the quantization parameters are obtained within the preset quantization time, the quantization parameters of the corresponding sub-network are fixed. If the quantization parameters are not obtained within the preset quantization time, the corresponding sub-network is marked as a network to be corrected, thereby completing the pre-quantization stage with higher efficiency and determining the network to be corrected that requires subsequent focus.

[0069] S204: Obtain the quantized model, and verify the quantized model using reference benchmark data.

[0070] Specifically, the model is quantized based on the quantization parameters of the sub-network to obtain a quantized model, and the quantized model is verified using reference benchmark data to determine whether the verification conditions are met to verify the quantization effect.

[0071] S205: In response to the verification condition not being met, based on the reference benchmark data, the input features corresponding to each sub-network are determined, the processing accuracy of the sub-network is verified using the input features, and the network to be corrected is screened from all sub-networks based on the processing accuracy, and the quantization parameters of the network to be corrected are adjusted.

[0072] Specifically, when the verification conditions are not met, the model obtained after the pre-quantization stage is obtained, and each sub-network in the current model is fixed so that each sub-network can be verified separately. Based on the reference benchmark data, respective input features are prepared for each sub-network, so as to focus only on the sub-network itself and avoid the error of the previous sub-network from affecting the subsequent sub-network.

[0073] Furthermore, the processing accuracy of the corresponding sub-network is verified by using the input features corresponding to the sub-network, so as to screen out the network to be corrected based on the processing accuracy of all sub-networks, wherein the processing accuracy of the network to be corrected is lower than that of other sub-networks that have not been selected. The quantization parameters of the network to be corrected are adjusted separately, so as to re-determine the quantization parameters of the network to be corrected with lower processing accuracy, thereby improving the accuracy and efficiency of quantization.

[0074] It is understandable that, after the quantization parameters of the network to be corrected are determined, the model can be quantized again based on the latest quantization parameters of the sub-network to obtain the quantized model, and the quantized model can be verified using the reference benchmark data to determine whether the verification conditions are met, and step S205 is repeated if the verification conditions are not met, or step S206 is entered after the verification conditions are met. Therefore, in the pre-quantization stage and at least one round of screening of the network to be corrected for quantization, the quantization process can be exited immediately after the verification conditions are met, and multiple rounds of step-by-step quantization can be achieved to ensure the quantization effect. Each step can exit and complete the quantization as long as the target is reached, thereby improving the quantization efficiency.

[0075] In some implementation scenarios, the processing accuracy of the sub-network is verified using input features, and the network to be corrected is screened from all sub-networks based on the processing accuracy, and the quantization parameters of the network to be corrected are adjusted, including: inputting the input features into the sub-network before quantization to obtain a first output feature, and inputting the input features into the sub-network after quantization to obtain a second output feature; obtaining the similarity between the first output feature and the second output feature, sorting the sub-networks based on the similarity, and screening the network to be corrected from all the sorted sub-networks; adjusting the value or bit width of the quantization parameters of the network to be corrected to obtain the adjusted quantization parameters of the network to be corrected.

[0076] Specifically, for each subnetwork, the input feature is input into the subnetwork before quantization, that is, the subnetwork of the original model, to obtain the first output feature, and the input feature is input into the subnetwork after quantization to obtain the second output feature, and the similarity between the first output feature and the second output feature is obtained. The subnetworks are sorted based on the similarity, and the subnetworks with lower similarity are selected from all the sorted subnetworks as the networks to be corrected.

[0077] Furthermore, the network to be corrected is quantized using the reference benchmark data, and the value of the quantization parameter of the network to be corrected is adjusted or the bit width of the quantization parameter of the network to be corrected is adjusted until the similarity between the second output feature of the quantized sub-network output and the first output feature meets the output condition, thereby obtaining the adjusted quantization parameter of the network to be corrected.

[0078] In a specific implementation scenario, if the network to be corrected corresponds to a quantization parameter of the pre-quantization stage, the value of the quantization parameter of the pre-quantization stage is adjusted; if the network to be corrected does not correspond to a quantization parameter of the pre-quantization stage, the bit width of the quantization parameter is adjusted to be greater than the minimum bit width.

[0079] S206: In response to the verification condition being met, the current quantization parameters of each sub-network in the model are fixed to obtain a quantized model, and the knowledge base is updated.

[0080] Specifically, when the verification conditions are met, the current quantization parameters of each sub-network in the model are fixed to obtain a quantized model. The sub-networks that are not indexed to the quantization parameters and the sub-networks whose quantization parameters have been adjusted are used as sub-networks for updating the knowledge base, and the network information and quantization parameters of the corresponding sub-networks are obtained to update the knowledge base.

[0081] In this embodiment, the quantization parameter is first indexed in the knowledge base, and then the sub-network to which the quantization parameter is not indexed is quantized, forming a pre-quantization stage. In the pre-quantization stage, the quantization parameter can be reused and the quantization parameter can be determined individually by the sub-network to improve the quantization efficiency. When the model obtained after the pre-quantization stage does not meet the verification conditions, the network to be corrected is screened and the quantization parameters of the network to be corrected are adjusted. In the pre-quantization stage and at least one round of screening of the network to be corrected for quantization, the quantization process can be exited immediately after the verification conditions are met, thereby realizing multi-round step-by-step quantization and ensuring the quantization effect. Each step can exit and complete the quantization as long as the target is reached, thereby improving the quantization efficiency.

[0082] See also Figure 4 , Figure 4 It is a structural diagram of an embodiment of an electronic device of the present application, wherein the electronic device 40 includes a memory 401 and a processor 402 coupled to each other, wherein the memory 401 stores program data (not shown), and the processor 402 calls the program data to implement the method in any of the above embodiments. For descriptions of related contents, please refer to the detailed description of the above method embodiments, which will not be repeated here.

[0083] See also Figure 5 , Figure 5It is a structural diagram of an embodiment of a computer-readable storage medium of the present application. The computer-readable storage medium 50 stores program data 500. When the program data 500 is executed by a processor, the method in any of the above embodiments is implemented. For descriptions of related contents, please refer to the detailed description of the above method embodiments, which will not be repeated here.

[0084] It should be noted that the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present implementation scheme.

[0085] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0086] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) or a processor to execute all or part of the steps of each implementation method of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program code.

[0087] The above description is only an implementation method of the present application, and does not limit the protection scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly used in other related technical fields, are also included in the protection scope of the present application.

Claims

1. A model quantization method, characterized in that: The method comprises: Acquire candidate benchmark data and corresponding data distribution thereof, and select at least part of the candidate benchmark data as reference benchmark data based on the data distribution; Decomposing the model to be quantized into multiple sub-networks and obtaining network information of the sub-networks, using the network information to index quantization parameters in a knowledge base, and determining the sub-network indexed to the quantization parameters; wherein the knowledge base stores the network information and quantization parameters of the sub-networks that have been quantized; In response to at least part of the sub-networks not being indexed to quantization parameters, quantizing the sub-networks not being indexed to quantization parameters using the reference benchmark data; Acquire the quantized model, verify the quantized model using the reference benchmark data, obtain the quantized model after the verification condition is met, and update the knowledge base; specifically include: Acquire the quantized model, and verify the quantized model using the reference benchmark data; In response to the verification condition not being met, based on the reference benchmark data, the input features corresponding to each of the sub-networks are determined, the processing accuracy of the sub-network is verified using the input features, and the network to be corrected is screened from all the sub-networks based on the processing accuracy, and the quantization parameters of the network to be corrected are adjusted; wherein, after the quantization parameters of the network to be corrected are determined, the model can be quantized again based on the latest quantization parameters of the sub-network to obtain a quantized model, and the quantized model is verified using the reference benchmark data to determine whether the verification condition is met; In response to satisfying the verification condition, the current quantization parameters of each sub-network in the model are fixed to obtain a quantized model, and the knowledge base is updated.

2. The model quantization method according to claim 1, characterized in that: The acquiring candidate benchmark data and corresponding data distribution thereof, and selecting at least part of the candidate benchmark data as reference benchmark data based on the data distribution, comprises: Acquire candidate benchmark data and its corresponding data distribution, and determine a distribution peak value corresponding to the data distribution; Based on the distribution peak value in the data distribution and the distribution value within a preset range thereof, at least part of the candidate benchmark data is selected from all the candidate benchmark data as reference benchmark data.

3. The model quantization method according to claim 2, characterized in that: The preset range includes a plurality of value intervals, and the value intervals are determined based on the distance of the distribution value relative to the distribution peak value; The selecting at least part of the candidate benchmark data as reference benchmark data from all the candidate benchmark data based on the distribution peak value in the data distribution and the distribution value within the preset range thereof, comprises: Based on the distribution peak in the data distribution, at least part of the candidate benchmark data is selected in turn from the multiple value intervals as reference benchmark data; wherein the number of the candidate benchmark data selected in the value interval is negatively correlated with the distance of the distribution value relative to the distribution peak.

4. The model quantization method according to claim 1, characterized in that: The network information includes the network type and network parameters of the sub-network; The step of decomposing the model to be quantized into a plurality of sub-networks and obtaining network information of the sub-networks, using the network information to perform quantization parameter indexing in a knowledge base, and determining the sub-network indexed to the quantization parameter includes: Decomposing the model to be quantized into multiple sub-networks and obtaining the network type and network parameters of each sub-network, obtaining the network information of each sub-network, and generating an index identifier corresponding to the network information; The index identifier is used to perform quantization parameter indexing in the knowledge base, and the sub-network matched by the index identifier and the quantization parameter of the sub-network in the knowledge base are determined; wherein the network information stored in the knowledge base corresponds to the index identifier.

5. The model quantization method according to claim 4, characterized in that: The step of decomposing the model to be quantized into a plurality of sub-networks and acquiring a network type and a network parameter of each of the sub-networks, obtaining network information of each of the sub-networks, and generating an index identifier corresponding to the network information includes: Decomposing the model to be quantified into multiple sub-networks, and determining the network type of the sub-network based on the network structure of the sub-network; Obtaining the minimum value, maximum value and interval of the parameters under the network structure as the network parameters of the sub-network, and obtaining the network information of each of the sub-networks; wherein the parameter interval includes the parameter values ​​of a preset proportion under the network structure; The network information is encoded to generate an index identifier corresponding to the network information.

6. The model quantization method according to claim 1, characterized in that: The using the input feature to verify the processing accuracy of the sub-network, and selecting a network to be corrected from all the sub-networks based on the processing accuracy, and adjusting the quantization parameters of the network to be corrected, includes: Input the input feature into the sub-network before quantization to obtain a first output feature, and input the input feature into the sub-network after quantization to obtain a second output feature; Obtaining the similarity between the first output feature and the second output feature, sorting the sub-networks based on the similarity, and screening the network to be corrected from all the sorted sub-networks; The value or bit width of the quantization parameter of the network to be corrected is adjusted to obtain the adjusted quantization parameter of the network to be corrected.

7. The model quantization method according to claim 1, characterized in that: The step of quantizing the sub-network to which the quantization parameter is not indexed by using the reference benchmark data includes: quantizing the sub-network to which the quantization parameter is not indexed using the reference benchmark data within a preset quantization time; In response to obtaining a quantization parameter within the preset quantization time, fixing the corresponding quantization parameter of the sub-network; In response to not obtaining the quantization parameter within the preset quantization time, marking the corresponding sub-network as a network to be corrected.

8. An electronic device, characterized in that: include: A memory and a processor coupled to each other, wherein the memory stores program data, and the processor calls the program data to execute the method according to any one of claims 1 to 7.

9. A computer-readable storage medium having program data stored thereon, characterized in that: When the program data is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images