Model compression method, device, electronic device, storage medium and product

Abstract

The embodiment of the present application provides a model compression method, device, electronic device, storage medium and product, which relates to the field of deep learning technology. The method includes: for the same input data, using the model to be compressed and the quantization model for processing respectively; calculating the difference between the output data of the model to be compressed and the quantization model as the first difference corresponding to the quantization model; adjusting the quantization parameters of each network layer in the model to be compressed according to the first difference; calculating the continuous domain quantization bit width of the network layer based on the optimal quantization parameter of each network layer; determining the discrete domain quantization bit width that is larger than the continuous domain quantization bit width of the network layer and has the smallest difference with the continuous domain quantization bit width of the network layer from each discrete domain quantization bit width supported by the hardware platform where the model needs to be deployed; and determining the corresponding target compression model with the smallest second difference from each candidate compression model obtained by compressing using multiple sparsity combinations. The applicability of the model can be improved.

Description

Technical Field

[0001] The present application relates to the field of deep learning technology, and in particular to a model compression method, device, electronic device, storage medium and product. Background Art

[0002] With the development of technology, the application of neural network models is becoming more and more extensive. For example, neural network models can be applied to image detection, text detection, audio detection, etc. However, neural network models often contain more parameters, and the network structure is more complex, which requires more storage resources and computing resources. This places high requirements on the storage space and performance of electronic devices used to run neural network models, resulting in low applicability of neural network models. Summary of the invention

[0003] The purpose of the embodiments of the present application is to provide a model compression method, device, electronic device, storage medium and product to reduce the storage resources and computing resources consumed by the model and improve the applicability of the model. The specific technical solution is as follows:

[0004] In a first aspect of an embodiment of the present application, a model compression method is provided, the method comprising:

[0005] The same input data is processed using the model to be compressed and the current quantization model respectively; wherein the current quantization model is obtained by quantizing the first network parameter of the network layer according to the quantization bit width obtained based on the current quantization parameter of each network layer in the model to be compressed; the quantization parameter of any network layer includes the truncation coefficient and quantization interval of the network layer;

[0006] Calculating a difference between output data obtained by the model to be compressed and the current quantization model as a first difference corresponding to the current quantization model;

[0007] According to the first difference corresponding to the current quantization model, the current quantization parameter of each network layer in the model to be compressed is adjusted until the first convergence condition is reached, thereby obtaining the optimal quantization parameter of each network layer;

[0008] Based on the optimal quantization parameter of each network layer, the continuous domain quantization bit width of the network layer is calculated;

[0009] From the discrete domain quantization bit widths supported by the hardware platform where the model needs to be deployed, determine a bit width that is larger than the continuous domain quantization bit width of the network layer and has the smallest difference with the continuous domain quantization bit width of the network layer as the discrete domain quantization bit width of the network layer;

[0010] Determine, from among the candidate compression models obtained by respectively compressing using multiple sparsity combinations, a candidate compression model with the smallest corresponding second difference as a target compression model;

[0011] Among them, an alternative compression model is: using the discrete domain quantization bit width of each network layer to quantize the first network parameter of the network layer, and using a sparsity combination to sparsify the second network parameters of each network layer after quantization; the second difference corresponding to an alternative compression model is represented by: the difference between the output data obtained by the alternative compression model and the model to be compressed when processing the same input data; the size relationship of the sparsity of each network layer included in any sparsity combination is consistent with the size relationship of the redundancy of each network layer; the redundancy of any network layer is represented by: the difference between the discrete domain quantization bit width and the continuous domain quantization bit width of the network layer.

[0012] Optionally, determining, from among the candidate compression models obtained by respectively compressing using multiple sparsity combinations, a corresponding candidate compression model with the smallest second difference as the target compression model includes:

[0013] According to the discrete domain quantization bit width of each network layer and the current sparsity combination, the model to be compressed is quantized and sparsified to obtain the current candidate compression model; wherein each sparsity included in the sparsity combination used when the candidate compression model is obtained for the first time is a preset initial value;

[0014] For the same input data, the to-be-compressed model and the current candidate compression model are used for processing respectively;

[0015] Calculating a difference between output data obtained by the model to be compressed and the current candidate compression model as a second difference corresponding to the current candidate compression model;

[0016] When the second difference corresponding to the current candidate compression model is less than the current optimal second difference, the current sparsity combination is updated to obtain a new sparsity combination, and the current optimal second difference is updated to the second difference corresponding to the current candidate compression model; when the second difference corresponding to the current candidate compression model is not less than the current optimal second difference, the sparsity combination used to obtain the current optimal second difference is updated to obtain a new sparsity combination;

[0017] Return to the step of quantizing and sparsifying the model to be compressed according to the discrete domain quantization bit width of each network layer and the current sparsity combination to obtain the current candidate compression model, until the second convergence condition is reached to obtain the target compression model.

[0018] Optionally, the second convergence condition is that the difference between the second difference corresponding to the current candidate compression model and the current optimal second difference is less than a preset threshold, or the number of times the optimal second difference is updated is a preset number.

[0019] Optionally, calculating a difference between output data obtained by the model to be compressed and the current candidate compression model as a second difference corresponding to the current candidate compression model includes:

[0020] For the same input data, obtaining output data of the first specified network layer in the current candidate compression model, and output data of the network layer corresponding to each first specified network layer in the model to be compressed;

[0021] Calculate the difference between the output data of each first specified network layer in the current candidate compression model and the output data of the network layer corresponding to the first specified network layer in the to-be-compressed model, and obtain the sub-difference corresponding to the first specified network layer;

[0022] Based on the sub-differences corresponding to the first specified network layers, a second difference corresponding to the current candidate compression model is calculated.

[0023] Optionally, calculating a difference between output data obtained by the model to be compressed and the current quantization model as a first difference corresponding to the current quantization model includes:

[0024] For the same input data, obtaining output data of the second specified network layer in the current quantization model, and output data of the network layer corresponding to each second specified network layer in the model to be compressed;

[0025] Calculate the difference between the output data of each second specified network layer in the current quantization model and the output data of the network layer corresponding to the second specified network layer in the model to be compressed, and obtain the sub-difference corresponding to the second specified network layer;

[0026] Based on the sub-differences corresponding to the second specified network layers, a first difference corresponding to the current quantization model is calculated.

[0027] Optionally, the calculating the continuous domain quantization bit width of each network layer based on the optimal quantization parameter of the network layer includes:

[0028] Based on the optimal quantization parameter of each network layer, the continuous domain quantization bit width of the network layer is calculated according to a preset formula; wherein the preset formula is as follows:

[0029]

[0030] Indicates The continuous domain quantization bit width of each network layer is Indicates The optimal quantization parameters of the network layer include the truncation coefficients, Indicates The optimal quantization parameters of each network layer include the quantization interval.

[0031] Optionally, the first network parameter of any network layer includes: the weight and / or activation value of the network layer; the second network parameter of any network layer includes: the weight and / or activation value of the network layer.

[0032] According to a second aspect of the embodiment of the present application, a model compression device is provided, the device comprising:

[0033] A data processing module, used for processing the same input data using the model to be compressed and the current quantization model respectively; wherein the current quantization model is obtained by quantizing the first network parameter of the network layer according to the quantization bit width obtained based on the current quantization parameter of each network layer in the model to be compressed; the quantization parameter of any network layer includes the truncation coefficient and quantization interval of the network layer;

[0034] A first difference calculation module, used to calculate the difference between the output data obtained by the model to be compressed and the current quantization model as a first difference corresponding to the current quantization model;

[0035] A quantization parameter adjustment module, configured to adjust the current quantization parameter of each network layer in the model to be compressed according to the first difference corresponding to the current quantization model, until the first convergence condition is reached, thereby obtaining the optimal quantization parameter of each network layer;

[0036] A continuous domain quantization bit width calculation module, used to calculate the continuous domain quantization bit width of each network layer based on the optimal quantization parameter of the network layer;

[0037] A discrete domain quantization bit width determination module is used to determine, from among various discrete domain quantization bit widths supported by the hardware platform on which the model needs to be deployed, a bit width that is greater than the continuous domain quantization bit width of the network layer and has the smallest difference with the continuous domain quantization bit width of the network layer, as the discrete domain quantization bit width of the network layer;

[0038] A target compression model determination module, used to determine, from among the candidate compression models obtained by respectively compressing using multiple sparsity combinations, a corresponding candidate compression model with the smallest second difference as the target compression model;

[0039] Among them, an alternative compression model is: using the discrete domain quantization bit width of each network layer to quantize the first network parameter of the network layer, and using a sparsity combination to sparsify the second network parameters of each network layer after quantization; the second difference corresponding to an alternative compression model is represented by: the difference between the output data obtained by the alternative compression model and the model to be compressed when processing the same input data; the size relationship of the sparsity of each network layer included in any sparsity combination is consistent with the size relationship of the redundancy of each network layer; the redundancy of any network layer is represented by: the difference between the discrete domain quantization bit width and the continuous domain quantization bit width of the network layer.

[0040] Optionally, the target compression model determination module includes:

[0041] A model sparsification submodule, used to quantize and sparsify the model to be compressed according to the discrete domain quantization bit width of each network layer and the current sparsity combination, so as to obtain the current candidate compression model; wherein each sparsity included in the sparsity combination used when obtaining the candidate compression model for the first time is a preset initial value;

[0042] A data processing submodule, used for processing the same input data using the to-be-compressed model and the current candidate compression model respectively;

[0043] A second difference calculation submodule, used for calculating the difference between the output data obtained by the model to be compressed and the current candidate compression model as the second difference corresponding to the current candidate compression model;

[0044] The sparsity combination updating submodule is used to update the current sparsity combination to obtain a new sparsity combination when the second difference corresponding to the current alternative compression model is smaller than the current optimal second difference, and update the current optimal second difference to the second difference corresponding to the current alternative compression model; when the second difference corresponding to the current alternative compression model is not smaller than the current optimal second difference, update the sparsity combination used to obtain the current optimal second difference to obtain a new sparsity combination; trigger the model sparsification submodule until the second convergence condition is reached to obtain the target compression model.

[0045] Optionally, the second convergence condition is that the difference between the second difference corresponding to the current candidate compression model and the current optimal second difference is less than a preset threshold, or the number of times the optimal second difference is updated is a preset number.

[0046] Optionally, the second difference calculation submodule is specifically used to obtain, for the same input data, the output data of the first specified network layer in the current alternative compression model and the output data of the network layer corresponding to each first specified network layer in the model to be compressed; calculate the difference between the output data of each first specified network layer in the current alternative compression model and the output data of the network layer corresponding to the first specified network layer in the model to be compressed, to obtain the sub-difference corresponding to the first specified network layer; based on the sub-differences corresponding to each first specified network layer, calculate the second difference corresponding to the current alternative compression model.

[0047] Optionally, the first difference calculation module is specifically used to obtain, for the same input data, the output data of the second specified network layer in the current quantization model and the output data of the network layer corresponding to each second specified network layer in the model to be compressed; calculate the difference between the output data of each second specified network layer in the current quantization model and the output data of the network layer corresponding to the second specified network layer in the model to be compressed, to obtain the sub-difference corresponding to the second specified network layer; based on the sub-differences corresponding to each second specified network layer, calculate the first difference corresponding to the current quantization model.

[0048] Optionally, the continuous domain quantization bit width calculation module is specifically used to calculate the continuous domain quantization bit width of each network layer according to a preset formula based on the optimal quantization parameter of the network layer; wherein the preset formula is as follows:

[0049]

[0050] Indicates The continuous domain quantization bit width of each network layer is Indicates The optimal quantization parameters of the network layer include the truncation coefficients, Indicates The optimal quantization parameters of each network layer include the quantization interval.

[0051] Optionally, the first network parameter of any network layer includes: the weight and / or activation value of the network layer; the second network parameter of any network layer includes: the weight and / or activation value of the network layer.

[0052] According to a third aspect of the embodiments of the present application, there is further provided an electronic device, including:

[0053] Memory, used to store computer programs;

[0054] The processor is used to implement any of the above-mentioned model compression methods when executing the program stored in the memory.

[0055] An embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, any of the above-mentioned model compression methods is implemented.

[0056] An embodiment of the present application also provides a computer program product comprising instructions, which, when executed on a computer, enables the computer to execute any of the above-mentioned model compression methods.

[0057] Beneficial effects of the embodiments of the present application:

[0058] A model compression method provided by an embodiment of the present application can process the same input data using a model to be compressed and a current quantization model respectively; wherein the current quantization model is obtained by quantizing the network layer according to the quantization bit width obtained based on the current quantization parameter of each network layer in the model to be compressed; the quantization parameter of any network layer includes the truncation coefficient and quantization interval of the network layer; the difference between the output data obtained by the model to be compressed and the current quantization model is calculated as the first difference corresponding to the current quantization model; according to the first difference corresponding to the current quantization model, the current quantization parameter of each network layer in the model to be compressed is adjusted until the first convergence condition is reached, and the optimal quantization parameter of each network layer is obtained; based on the optimal quantization parameter of each network layer, the continuous domain quantization bit width of the network layer is calculated; the discrete domain quantization bit width supported by the hardware platform where the model needs to be deployed is calculated; Among the alternative compression models, a bit width that is larger than the continuous domain quantization bit width of the network layer and has the smallest difference with the continuous domain quantization bit width of the network layer is determined as the discrete domain quantization bit width of the network layer; from the alternative compression models obtained by compressing respectively using multiple sparsity combinations, the alternative compression model with the smallest corresponding second difference is determined as the target compression model; wherein, an alternative compression model is: obtained by quantizing and sparsifying the model to be compressed using the discrete domain quantization bit width of each network layer and a sparsity combination; the second difference corresponding to an alternative compression model is represented by: the difference between the output data obtained by processing the same input data by the alternative compression model and the model to be compressed; the size relationship of the sparsities of each network layer contained in any sparsity combination is consistent with the size relationship of the redundancy of each network layer; the redundancy of any network layer is represented by: the difference between the discrete domain quantization bit width and the continuous domain quantization bit width of the network layer.

[0059] Based on the above processing, the quantization parameters of each network layer in the model that currently needs to be compressed (i.e., the model to be compressed) can be optimized to obtain the optimal quantization parameters of each network layer. According to the quantization bit width obtained based on the optimal quantization parameters of each network layer, the performance difference between the quantized model obtained by quantizing the model to be compressed and the model to be compressed is minimized. Based on the optimal quantization parameters of each network layer, the continuous domain quantization bit width of the network layer can be calculated, and the quantization bit width (i.e., discrete domain quantization bit width) that can be used in actual quantization can be determined in combination with the discrete domain quantization bit widths supported by the hardware platform where the model needs to be deployed, so as to ensure that the model to be compressed can be quantized according to the discrete domain quantization bit width. And because the discrete domain quantization bit width is: the bit width that is greater than the continuous domain quantization bit width and has the smallest difference with the continuous domain quantization bit width among the discrete domain quantization bit widths supported by the hardware platform, it is possible to ensure that the compression ratio of the model quantized is optimal while obtaining the performance of the model by quantizing the model to be compressed according to the discrete domain quantization bit width.

[0060] The difference (i.e., redundancy) between the discrete domain quantization bit width and the continuous domain quantization bit width of any network layer can represent the redundancy of the expression capability of the network layer obtained by quantizing the first network parameter of the network layer according to the discrete domain quantization bit width of the network layer, relative to the network layer obtained by quantizing the first network parameter of the network layer according to the continuous domain quantization bit width of the network layer. The higher the redundancy, the higher the degree to which the network layer obtained by quantizing the network layer according to the discrete domain quantization bit width of the network layer can be further compressed, that is, the higher the degree to which the quantized network layer can be subsequently sparse. Furthermore, multiple sparsity combinations can be determined, and the size relationship of the sparsity of each network layer contained in each sparsity combination is consistent with the size relationship of the redundancy of each network layer. From the various alternative compression models obtained by compressing the model to be compressed using each sparsity combination, the target compression model is determined. The performance difference between the target compression model and the model to be compressed is minimal. In this way, the model to be compressed can be compressed by combining quantization and sparsification, which can effectively improve the degree of compression of the model while ensuring the performance of the compressed model, reduce the storage and computing resources required by the model, and improve the applicability of the model.

[0061] Of course, implementing any product or method of the present application does not necessarily require achieving all of the advantages described above at the same time. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application, and for ordinary technicians in this field, other embodiments can also be obtained based on these drawings.

[0063] Figure 1 A schematic diagram of a first flow chart of a model compression method provided in an embodiment of the present application;

[0064] Figure 2 A schematic diagram of a process for optimizing sparsity combination in a model compression method provided in an embodiment of the present application;

[0065] Figure 3 A second flow chart of the model compression method provided in the embodiment of the present application;

[0066] Figure 4 A third flow chart of the model compression method provided in the embodiment of the present application;

[0067] Figure 5 A schematic diagram of the structure of a model compression device provided in an embodiment of the present application;

[0068] Figure 6 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0069] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field based on the present application belong to the scope of protection of the present application.

[0070] With the development of technology, the application of neural network models is becoming more and more extensive. For example, neural network models can be applied to image detection, text detection, audio detection, etc. However, neural network models often contain more parameters, and the network structure is more complex, which requires more storage resources and computing resources. This places high requirements on the storage space and performance of electronic devices used to run neural network models, resulting in low applicability of neural network models.

[0071] In order to improve the applicability of the neural network model, the present application embodiment provides a model compression method, see Figure 1 , Figure 1 A first flow chart of a model compression method provided in an embodiment of the present application, the method comprising:

[0072] Step S101: for the same input data, use the model to be compressed and the current quantization model to process respectively.

[0073] Among them, the current quantization model is: according to the quantization bit width obtained based on the current quantization parameter of each network layer in the model to be compressed, the first network parameter of the network layer is quantized; the quantization parameter of any network layer includes the truncation coefficient and quantization interval of the network layer.

[0074] Step S102: Calculate the difference between the output data obtained by the model to be compressed and the current quantization model as the first difference corresponding to the current quantization model.

[0075] Step S103: According to the first difference corresponding to the current quantization model, the current quantization parameter of each network layer in the compression model is adjusted until the first convergence condition is reached, thereby obtaining the optimal quantization parameter of each network layer.

[0076] Step S104: Based on the optimal quantization parameter of each network layer, the continuous domain quantization bit width of the network layer is calculated.

[0077] Step S105: From the discrete domain quantization bit widths supported by the hardware platform where the model needs to be deployed, determine the bit width that is larger than the continuous domain quantization bit width of the network layer and has the smallest difference with the continuous domain quantization bit width of the network layer as the discrete domain quantization bit width of the network layer.

[0078] Step S106: Determine the alternative compression model with the smallest corresponding second difference from the alternative compression models obtained by respectively compressing using multiple sparsity combinations, as the target compression model; wherein an alternative compression model is obtained by quantizing the first network parameter of each network layer using the discrete domain quantization bit width of the network layer, and sparsifying the second network parameters of each network layer after quantization using a sparsity combination; the second difference corresponding to an alternative compression model is represented by: the difference between the output data obtained by processing the same input data by the alternative compression model and the model to be compressed; the size relationship of the sparsity of each network layer included in any sparsity combination is consistent with the size relationship of the redundancy of each network layer; the redundancy of any network layer is represented by: the difference between the discrete domain quantization bit width and the continuous domain quantization bit width of the network layer.

[0079] Based on the above processing, the quantization parameters of each network layer in the model that currently needs to be compressed (i.e., the model to be compressed) can be optimized to obtain the optimal quantization parameters of each network layer. According to the quantization bit width obtained based on the optimal quantization parameters of each network layer, the performance difference between the quantized model obtained by quantizing the model to be compressed and the model to be compressed is minimized. Based on the optimal quantization parameters of each network layer, the continuous domain quantization bit width of the network layer can be calculated, and the quantization bit width (i.e., discrete domain quantization bit width) that can be used in actual quantization can be determined in combination with the discrete domain quantization bit widths supported by the hardware platform where the model needs to be deployed, so as to ensure that the model to be compressed can be quantized according to the discrete domain quantization bit width. And because the discrete domain quantization bit width is: the bit width that is greater than the continuous domain quantization bit width and has the smallest difference with the continuous domain quantization bit width among the discrete domain quantization bit widths supported by the hardware platform, it is possible to ensure that the compression ratio of the model quantized is optimal while obtaining the performance of the model by quantizing the model to be compressed according to the discrete domain quantization bit width.

[0080] The difference (i.e., redundancy) between the discrete domain quantization bit width and the continuous domain quantization bit width of any network layer can represent the redundancy of the expression capability of the network layer obtained by quantizing the first network parameter of the network layer according to the discrete domain quantization bit width of the network layer, relative to the network layer obtained by quantizing the first network parameter of the network layer according to the continuous domain quantization bit width of the network layer. The higher the redundancy, the higher the degree to which the network layer obtained by quantizing the network layer according to the discrete domain quantization bit width of the network layer can be further compressed, that is, the higher the degree to which the quantized network layer can be subsequently sparse. Furthermore, multiple sparsity combinations can be determined, and the size relationship of the sparsity of each network layer contained in each sparsity combination is consistent with the size relationship of the redundancy of each network layer. From the various alternative compression models obtained by compressing the model to be compressed using each sparsity combination, the target compression model is determined. The performance difference between the target compression model and the model to be compressed is minimal. In this way, the model to be compressed can be compressed by combining quantization and sparsification, which can effectively improve the degree of compression of the model while ensuring the performance of the compressed model, reduce the storage and computing resources required by the model, and improve the applicability of the model.

[0081] For step S101, the model that currently needs to be compressed (i.e., the model to be compressed) is a pre-trained neural network model, which may include multiple network layers. For example, the network layers in the model to be compressed may include: convolutional layer, activation layer, fully connected layer, pooling layer, etc. The type of the model to be compressed can be set as needed without specific limitation. For example, the model to be compressed may be a pre-trained image detection model, text detection model, or audio detection model. The quantization parameters of any network layer may include the truncation coefficient and quantization interval of the network layer. The quantization bit width of the network layer can be obtained based on the truncation coefficient and quantization interval of each network layer. For the specific method of obtaining the quantization bit width, please refer to the relevant description of subsequent steps S104 and S105.

[0082] According to the quantization bit width obtained based on the current quantization parameter of each network layer, the first network parameter of the network layer is quantized to obtain a quantization model. The first network parameter of any network layer may include at least one of the weight and activation value of the network layer. The activation value of any network layer may represent the output data of the network layer. Accordingly, the quantization parameter of any network layer may include: the truncation coefficient and quantization interval of the weight of the network layer, and / or, the truncation coefficient and quantization interval of the activation value of the network layer. Based on the truncation coefficient and quantization interval corresponding to the weight of a network layer, the quantization bit width (which may be referred to as the weight quantization bit width) used to quantize the weight of the network layer can be obtained. Based on the truncation coefficient and quantization interval corresponding to the activation value of a network layer, the quantization bit width (which may be referred to as the activation value quantization bit width) used to quantize the activation value of the network layer can be obtained.

[0083] Furthermore, in the process of obtaining the quantized model, for each network layer, the weight of the network layer can be quantized according to the weight quantization bit width, and / or the activation value of the network layer can be quantized according to the activation value quantization bit width. That is, when quantizing each network layer, at least one of the weight and activation value of the network layer can be quantized. While ensuring that the storage resources and computing resources required for the model can be reduced through quantization, the efficiency of quantizing the model can be improved by adjusting the part of each network layer that needs to be quantized, which can also improve the efficiency of compressing the model.

[0084] The current quantization parameter is the latest quantization parameter. The quantization parameter of each network layer used when the quantization model is first obtained can be a preset initial quantization parameter. For the same input data, the model to be compressed and the quantization model can be used for processing respectively. The input data can be determined according to the type of the model to be compressed. For example, if the model to be compressed is an image detection model, the input data can be an image; or, if the model to be compressed is a text detection model, the input data can be text; or, if the model to be compressed is an audio detection model, the input data can be audio.

[0085] For step S102, the difference between the output data obtained by the model to be compressed and the quantized model can be calculated, and the obtained difference can be called the first difference corresponding to the quantized model. The output data obtained by the model to be compressed and the quantized model can include: the output data of each network layer in the model to be compressed and the quantized model.

[0086] In one embodiment, step S102 includes:

[0087] Step 1: For the same input data, obtain the output data of the second specified network layer in the current quantization model, and the output data of the network layer corresponding to each second specified network layer in the model to be compressed.

[0088] Step 2: Calculate the difference between the output data of each second specified network layer in the current quantization model and the output data of the network layer corresponding to the second specified network layer in the model to be compressed, and obtain the sub-difference corresponding to the second specified network layer.

[0089] Step 3: Based on the sub-differences corresponding to each second specified network layer, calculate the first difference corresponding to the current quantization model.

[0090] In an embodiment of the present application, the second designated network layer may be all network layers in the quantization model, or may be part of the network layers included in the quantization model. The number of second designated network layers can be set as needed without specific limitation. For example, the quantization model includes ten network layers, and the second designated network layer may be any five of the ten network layers, such as the 1st, 3rd, 5th, 7th and 9th network layers of the ten network layers. The quantization model corresponds one to one with the network layers in the model to be compressed. For a second designated network layer in the quantization model, the second designated network layer is quantized based on the network layer corresponding to the second designated network layer in the model to be compressed.

[0091] For the same input data, the output data of the second specified network layer in the latest quantization model and the output data of the network layer corresponding to each second specified network layer in the model to be compressed can be obtained. The difference between the output data of each second specified network layer in the current quantization model and the output data of the network layer corresponding to the second specified network layer in the model to be compressed can be calculated to obtain the sub-difference corresponding to the second specified network layer. Furthermore, the first difference corresponding to the current quantization model can be calculated based on the sub-difference corresponding to each second specified network layer. For example, the weighted sum of the sub-differences corresponding to each second specified network layer can be calculated to obtain the first difference corresponding to the current quantization model.

[0092] Based on the above processing, when the second designated network layer is all the network layers in the quantization model, the sub-differences corresponding to all the network layers in the latest quantization model can be combined to obtain the first difference. In this case, since the sub-differences corresponding to all the network layers in the latest quantization model are combined in the process of calculating the first difference, the accuracy of optimizing the quantization parameters of each network layer according to the first difference is high, so as to ensure that the quantization parameters of each network layer can be effectively optimized, and the effectiveness of the parameters obtained by optimizing the quantization parameters of each network layer according to the first difference is ensured. When the second designated network layer is part of the network layer in the quantization model, the number of output data used in calculating the first difference can be reduced, the amount of calculation of the first difference can be reduced, and the efficiency of calculating the first difference can be improved. Then, the efficiency of optimizing the quantization parameters of each network layer according to the first difference can be improved, and the efficiency of compressing the model to be compressed can be improved.

[0093] In this way, the second designated network layer can be set as needed. While ensuring that the basis for adjusting the quantization parameters of each network layer can be calculated, the accuracy and efficiency of optimizing the quantization parameters of each network layer can be comprehensively considered. By adjusting the number of second designated network layers, that is, adjusting the number of output data used when calculating the first difference, while ensuring the effectiveness of the parameters obtained by subsequently optimizing the quantization parameters of each network layer according to the first difference, the efficiency of subsequently optimizing the quantization parameters of each network layer according to the first difference is improved, thereby improving the efficiency of compressing the model to be compressed.

[0094] For step S103, according to the first difference corresponding to the current quantization model, the current quantization parameter of each network layer in the compression model is adjusted until the first convergence condition is reached to obtain the optimal quantization parameter of each network layer. For example, based on back propagation and gradient descent, according to the first difference corresponding to the current quantization model, the current quantization parameter of each network layer in the compression model can be adjusted. The first convergence condition can be that the number of times the quantization parameter is adjusted reaches a preset number of adjustments, or the first convergence condition can also be that the first difference corresponding to the current quantization model is less than a preset difference threshold.

[0095] For example, according to the first difference corresponding to the current quantization model, the optimization objective function for adjusting the current quantization parameters of each network layer in the compression model can be expressed as:

[0096]

[0097] in, Indicates the first difference corresponding to the current quantization model, Represents the weights of each network layer in the model to be compressed, represents the input data processed by the model to be compressed and the current quantization model, and Respectively represent The truncation coefficients and quantization intervals of the weights of each network layer, and Respectively represent The truncation coefficient and quantization interval of the activation value of each network layer. The optimization goal is to minimize the first difference corresponding to the current quantization model. The optimization object is , that is, to adjust the current quantization parameters of each network layer in the compression model, that is, Make adjustments.

[0098] For step S104, based on the truncation coefficient and quantization interval of each network layer included in the optimal quantization parameter of the network layer, the continuous domain quantization bit width of the network layer can be calculated. The relationship between the truncation coefficient, quantization interval and continuous domain quantization bit width of any network layer can be expressed as:

[0099]

[0100] in, Indicates the continuous domain quantization bit width of the network layer, represents the truncation coefficient of the network layer, Indicates the quantization interval of the network layer. The truncation coefficient and quantization interval of any network layer are greater than 0, and the truncation coefficient of the network layer is greater than the quantization interval of the network layer.

[0101] Accordingly, step S104 may include: based on the optimal quantization parameter of each network layer, calculating the continuous domain quantization bit width of the network layer according to a preset formula.

[0102] The preset formula is as follows:

[0103]

[0104] Indicates The continuous domain quantization bit width of each network layer is Indicates The optimal quantization parameters of the network layer include the truncation coefficients, Indicates The optimal quantization parameters of each network layer include the quantization interval.

[0105] In an embodiment of the present application, the continuous domain quantization bit width of the network layer can be calculated according to the preset formula based on the optimal quantization parameter of each network layer. The calculated continuous domain quantization bit width is often a floating point value, and the continuous domain quantization bit width can also be called a floating point quantization bit width. That is, the calculated continuous domain quantization bit width belongs to the continuous domain, and based on the above steps S101-S103, the quantization parameters of each network layer can be optimized in the continuous domain.

[0106] In one implementation, the continuous domain quantization bit width of any network layer may include: the continuous domain quantization bit width of the weight and / or activation value of the network layer. Based on the truncation coefficient and quantization interval of the weight in the optimal quantization parameter of each network layer, the continuous domain quantization bit width of the weight of the network layer is calculated according to the first formula. The first formula is as follows:

[0107]

[0108] in, Indicates The continuous domain quantization bit width of the weights of each network layer, and Respectively represent The truncation coefficients and quantization intervals for the weights of each network layer.

[0109] Based on the truncation coefficient and quantization interval of the activation value in the optimal quantization parameter of each network layer, the continuous domain quantization bit width of the weight of the network layer is calculated according to the second formula. The second formula is as follows:

[0110]

[0111] in, Indicates The continuous domain quantization bit width of the activation value of each network layer is and Respectively represent The truncation coefficients and quantization intervals of the activation values ​​of each network layer.

[0112] Based on the above processing, the continuous domain quantization bit width of each network layer can be calculated according to the preset formula, and the discrete domain quantization bit width of each network layer can be determined based on the continuous domain quantization bit width of each network layer to achieve quantization of the model to be compressed. In addition, it is ensured that the model to be compressed can be compressed by combining quantization and sparsification, and while ensuring the performance of the compressed model, the degree of compression of the model is effectively improved, the storage resources and computing resources required by the model are reduced, and the applicability of the model is improved.

[0113] Regarding step S105, since the hardware platform on which the model to be compressed is deployed often has certain limitations on its support for bit width, for example, the discrete domain quantization bit widths supported by the hardware platform are 2 bits, 4 bits, 8 bits, 16 bits, etc. Therefore, in order to ensure that the model to be compressed can be quantized, it is necessary to align the continuous domain quantization bit width of the floating-point type of each network layer to the discrete domain quantization bit widths supported by the hardware platform, so as to obtain the discrete domain quantization bit width of the network layer that the hardware platform can support.

[0114] In order to ensure that the performance of the model is obtained by quantizing the compression model according to the discrete domain quantization bit width of each network layer, the continuous domain quantization bit width of each network layer floating point type is aligned upward to the discrete domain quantization bit width supported by the hardware platform to obtain the discrete domain quantization bit width of the network layer. In order to ensure that the degree of compression of the model is high when the compression model is quantized according to the discrete domain quantization bit width of each network layer, the continuous domain quantization bit width of each network layer floating point type can be aligned upward to the bit width supported by the hardware platform and closest to the continuous domain quantization bit width of the network layer. That is, from the discrete domain quantization bit widths supported by the hardware platform where the model needs to be deployed, the bit width that is greater than the continuous domain quantization bit width of the network layer and has the smallest difference with the continuous domain quantization bit width of the network layer can be determined as the discrete domain quantization bit width of the network layer. For example, if the continuous domain quantization bit width of a network layer is 3.4 bits, the discrete domain quantization bit width of the network layer can be determined to be 4 bits; if the continuous domain quantization bit width of a network layer is 10.2 bits, the discrete domain quantization bit width of the network layer can be determined to be 16 bits.

[0115] For step S106, the difference between the discrete domain quantization bit width and the continuous domain quantization bit width of any network layer (i.e., redundancy) can represent: relative to the network layer obtained by quantizing the network layer according to the continuous domain quantization bit width of the network layer, the redundancy of the expression capability of the network layer obtained by quantizing the network layer according to the discrete domain quantization bit width of the network layer. The higher the redundancy, the higher the degree to which the network layer obtained by quantizing the network layer according to the discrete domain quantization bit width of the network layer can be further compressed, that is, the degree to which the quantized network layer can be subsequently sparsed is higher.

[0116] Furthermore, the size relationship of the sparsity of each network layer can be determined according to the size relationship of the redundancy of each network layer to obtain multiple sparsity combinations including the sparsity of each network layer. The sparsity of any network layer can be represented by: the proportion of the weight and / or activation value of the network layer that becomes 0 when the network layer is sparsified. The size relationship of the sparsity of each network layer is consistent with the size relationship of the redundancy of each network layer, which means that the position of the redundancy of any network layer in the redundancy sorting of each network layer is the same as the position of the sparsity of the network layer in the sparsity sorting of each network layer, and the sorting method of the redundancy sorting and the sparsity sorting of each network layer is consistent (e.g., from high to low or from low to high). For example, the continuous domain quantization bit width of each network layer is (3.4, 10.2, 6.2) bits, and the discrete domain quantization bit width is (4, 16, 8) bits, then the redundancy of each network layer can be (0.6, 5.8, 1.8), that is, the redundancy of the first network layer is the lowest, and the redundancy of the second network layer is the highest. Correspondingly, the sparsity of the first network layer is the lowest, and the sparsity of the second network layer is the highest.

[0117] An alternative compression model is: the first network parameter of each network layer is quantized using the discrete domain quantization bit width of the network layer, and the second network parameter of each network layer after quantization is sparsely obtained using a sparsity combination. The second network parameter of any network layer may include at least one of the weight and activation value of the network layer. Accordingly, a sparsity combination may include: the sparsity of the weights of each network layer, and / or the sparsity of the activation value of the network layer. In the process of sparsifying each quantized network layer using a sparsity combination, for each network layer, the weight of the network layer may be sparsely according to the sparsity of the weight of the network layer, and / or, the activation value of the network layer may be sparsely according to the sparsity of the activation value of the network layer. That is, when sparsely performing each quantized network layer, at least one of the weight and activation value of the network layer may be sparsely performed. While ensuring that sparsification can be combined to reduce the storage and computing resources required by the model, the efficiency of sparsification of the model can be improved by adjusting the part of each network layer that needs to be sparsified, which can further improve the efficiency of compressing the model.

[0118] For the same input data, an alternative compression model and a model to be compressed can be used for processing respectively. Accordingly, the difference between the output data obtained by the alternative compression model and the model to be compressed can be calculated as the second difference corresponding to the alternative compression model. According to the second differences corresponding to each alternative compression model, the target compression model can be determined. The second difference corresponding to the target compression model is the smallest, that is, the performance difference between the target compression model and the model to be compressed is the smallest.

[0119] In one embodiment, the target compression model may be an image detection model, which is used to detect a specified object in an image. For example, the specified object may be a vehicle or an animal, etc., and the input data may be an image. For the same image, the image may be input into the model to be compressed and the current quantization model for processing. Correspondingly, the output data of the last network layer in the model to be compressed and the current quantization model may be the detection result of the input image, such as the detection result may represent the position of the specified object in the input image. The output data of other network layers except the last network layer may represent the feature data of the input image. Furthermore, the difference between the output data obtained by the model to be compressed and the current quantization model may be calculated in combination with the detection results and / or feature data obtained by the model to be compressed and the current quantization model as the first difference corresponding to the current quantization model, so as to obtain the basis for optimizing the quantization parameters of each network layer.

[0120] In the process of optimizing the sparsity combination, the input data is also an image. For the same image, the image can be input into the model to be compressed and the alternative compression model for processing. Correspondingly, the output data of the last network layer in the model to be compressed and the alternative compression model can be the detection result of the input image. The output data of other network layers except the last network layer can represent the feature data of the input image. Furthermore, the difference between the output data obtained by the model to be compressed and the alternative compression model can be calculated in combination with the detection results and / or feature data obtained by the model to be compressed and the alternative compression model as the second difference corresponding to the alternative compression model, so as to obtain the basis for optimizing the sparsity combination.

[0121] The target compression model can also be a text detection model, which is used to detect specified content in the text. For example, the specified content can be text that does not meet the preset specifications, or content that expresses specific emotions, etc., and the input data can be text. For the same text, the text can be input into the model to be compressed and the current quantization model for processing. Correspondingly, the output data of the last network layer in the model to be compressed and the current quantization model can be the detection result of the input text, such as the detection result can represent the location of the specified type of content in the input text. The output data of other network layers except the last network layer can represent the feature data of the input text. Furthermore, the difference between the output data obtained by the model to be compressed and the current quantization model can be calculated in combination with the detection results and / or feature data obtained by the model to be compressed and the current quantization model, as the first difference corresponding to the current quantization model, to obtain the basis for optimizing the quantization parameters of each network layer.

[0122] In the process of optimizing the sparsity combination, the input data is also text. For the same text, the text can be input into the model to be compressed and the alternative compression model for processing. Correspondingly, the output data of the last network layer in the model to be compressed and the alternative compression model can be the detection result of the input text. The output data of other network layers except the last network layer can represent the feature data of the input text. Furthermore, the difference between the output data obtained by the model to be compressed and the alternative compression model can be calculated in combination with the detection results and / or feature data obtained by the model to be compressed and the alternative compression model as the second difference corresponding to the alternative compression model, so as to obtain the basis for optimizing the sparsity combination.

[0123] After obtaining the target compression model, the target compression model can be used to process the data that currently needs to be detected. For example, if the model to be compressed is an image detection model, then the target compression model is also an image detection model. The image that currently needs to be detected (which can be referred to as the image to be detected) can be input into the target compression model to obtain the image detection result. For example, the image detection result can indicate the type of the image to be detected or the type of the object contained in the image to be detected.

[0124] In one implementation, multiple sparsity combinations can be randomly generated in advance according to the size relationship of the redundancy of each network layer. The number of sparsity combinations can be set as needed and is not specifically limited. The model to be compressed is quantized using the discrete domain quantization bit width of each network layer, and the quantized model is sparsified using each sparsity combination to obtain an alternative compression model corresponding to the sparsity combination. The difference between each alternative compression model and the output data obtained by processing the same input data with the model to be compressed is calculated to obtain the second difference corresponding to the alternative compression model. From each alternative compression model, the alternative compression model with the smallest corresponding second difference is determined as the target compression model.

[0125] In another implementation, step S106 includes:

[0126] Step 1: According to the discrete domain quantization bit width of each network layer and the current sparsity combination, the compression model is quantized and sparsified to obtain the current candidate compression model.

[0127] The sparsities included in the sparsity combination used when obtaining the candidate compression model for the first time are preset initial values.

[0128] Step 2: For the same input data, use the model to be compressed and the current candidate compression model to process them respectively.

[0129] Step 3: Calculate the difference between the output data obtained by the model to be compressed and the current candidate compression model as the second difference corresponding to the current candidate compression model.

[0130] Step 4: When the second difference corresponding to the current alternative compression model is smaller than the current optimal second difference, the current sparsity combination is updated to obtain a new sparsity combination, and the current optimal second difference is updated to the second difference corresponding to the current alternative compression model; when the second difference corresponding to the current alternative compression model is not smaller than the current optimal second difference, the sparsity combination used to obtain the current optimal second difference is updated to obtain a new sparsity combination; return to execute step 1 until the second convergence condition is reached to obtain the target compression model.

[0131] In an embodiment of the present application, an initial sparsity combination can be set in advance according to the size relationship of the redundancy of each network layer. That is, the sparsities contained in the initial sparsity combination are preset initial values, and the size relationship of each sparsity is consistent with the size relationship of the redundancy of each network layer. The model to be compressed is quantized according to the discrete domain quantization bit width of each network layer, and the initial sparsity combination can be used to sparsify the quantized model to obtain an initial compression model, which is the alternative compression model obtained for the first time. For the same input data, the model to be compressed and the initial compression model are used for processing respectively. The difference between the output data obtained by the model to be compressed and the initial compression model is calculated as the second difference corresponding to the initial compression model. The second difference corresponding to the initial compression model is used as the current optimal second difference, and the initial sparsity combination is updated.

[0132] Updating the initial sparsity combination can also be called mutating the initial sparsity combination. For example, the sparsity of any N network layers contained in the initial sparsity combination can be randomly transformed, N is greater than 0 and not greater than the number of network layers contained in the model to be compressed, and the sparsity contained in the initial sparsity combination and the transformed sparsity combination are both between 0 and 1. For example, the initial sparsity combination is (0.2, 0.1, 0.7), and the transformed sparsity combination can be (0.4, 0.3, 0.55).

[0133] It can also be determined whether the magnitude relationship of each sparsity contained in the transformed sparsity combination is consistent with the magnitude relationship of the redundancy of each network layer. If consistent, the transformed sparsity combination is used as a new sparsity combination. If inconsistent, the initial sparsity combination is transformed again. Correspondingly, the magnitude relationship of each sparsity contained in the new sparsity combination is also consistent with the magnitude relationship of the redundancy of each network layer.

[0134] Subsequently, the model to be compressed can be quantized according to the discrete domain quantization bit width of each network layer, and the quantized model can be sparsely processed using a new sparsity combination to obtain a new alternative compression model. For the same input data, the model to be compressed and the new alternative compression model are used for processing respectively. The difference between the output data obtained by the model to be compressed and the new alternative compression model is calculated as the second difference corresponding to the new alternative compression model. When the second difference corresponding to the new alternative compression model is less than the current optimal second difference, the current sparsity combination is updated, and the current optimal second difference is updated to the second difference corresponding to the new alternative compression model. When the second difference corresponding to the new alternative compression model is not less than the current optimal second difference, the sparsity combination used to obtain the current optimal second difference (i.e., the initial sparsity combination) is updated to obtain a new sparsity combination.

[0135] Furthermore, the compression model can be quantized according to the discrete domain quantization bit width of each network layer, and the quantized model can be sparsely populated using the latest sparsity combination to obtain the latest candidate compression model. Repeat the above steps until the second convergence condition is reached to obtain the target compression model. For example, the second convergence condition is: the difference between the second difference corresponding to the current candidate compression model and the current optimal second difference is less than a preset threshold, or the optimal second difference is updated a preset number of times.

[0136] The current optimal second difference is the minimum value among the calculated second differences. The difference between the second difference corresponding to the current alternative compression model and the current optimal second difference is less than the preset threshold, which means that the second difference corresponding to the current alternative compression model is smaller. When the number of times the optimal second difference is updated is the preset number, the second difference corresponding to the current alternative compressibility is often smaller. The preset number of times can be determined according to the number of network layers of the model to be compressed. For example, the more network layers of the model to be compressed, the greater the preset number of times. Subsequently, the performance improvement of the alternative compression model obtained by optimizing the sparsity combination and then using the new sparsity combination may not be large. Accordingly, it can be determined that the second convergence condition is met and the target compression model is obtained. In this way, the efficiency of optimizing the sparsity combination can be improved while optimizing the sparsity combination, and thus, the efficiency of compressing the model can be improved.

[0137] Based on the above processing, the sparsity combination can be optimized by evolutionary search, and finally the optimal sparsity combination and the alternative compression model (i.e., the target compression model) obtained by sparsification using the optimal sparsity combination are determined. And each time a new sparsity combination is obtained by updating the previous sparsity combination, there may be a situation where the sparsity of some network layers included remains unchanged, that is, the sparsity of some network layers remains unchanged. Since the relevant calculations for sparsification have been performed on this part of the network layer before, the amount of calculation when sparsification is performed according to the updated sparsity combination is also small. In this way, the efficiency of quantizing and sparsifying the compression model to be treated according to the discrete domain quantization bit width and sparsity combination of each network layer to obtain the alternative compression model can be improved, thereby improving the efficiency of optimizing the sparsity combination, and correspondingly, the efficiency of compressing the model can be improved.

[0138] In one embodiment, see Figure 2 , Figure 2 A schematic diagram of a process for optimizing sparsity combinations in a model compression method provided in an embodiment of the present application.

[0139] Step S201: Initialize the model sparsity list. The sparsity list is the sparsity combination in the above embodiment. That is, the initial sparsity combination is set according to the size relationship of the redundancy of each network layer.

[0140] Step S202: compress the model using the optimal quantization bit width and initial sparsity, perform forward inference on a given calibration data set, and compare the loss with the original model inference result to obtain loss1. The optimal quantization bit width is the discrete domain quantization bit width in the above embodiment. That is, steps 1 to 3 in the above embodiment. Loss1 is the current optimal second difference.

[0141] Step S203: mutate the sparsity, that is, update the current sparsity combination.

[0142] Step S204: Whether the mutation result conforms to the optimal sparsity order. If yes, execute step S205; if no, execute step S203. The optimal sparsity order is the order of the size relationship of the redundancy of each network layer. That is, determine whether the size relationship of each sparsity contained in the transformed sparsity combination is consistent with the size relationship of the redundancy of each network layer.

[0143] Step S205: compress the model with the optimal quantization bit width and given sparsity on the calibration data set, and perform forward reasoning, compare the reasoning result loss with the original model, and obtain loss2. That is, return to execute step 1 in the above embodiment. Loss2 is the second difference corresponding to the current candidate compression model.

[0144] Step S206: Is loss2 smaller than loss1? If yes, execute step S207; if no, execute step S203. That is, determine whether the second difference corresponding to the current candidate compression model is smaller than the current optimal second difference.

[0145] Step S207: Update the loss1 value to the loss2 value, that is, update the current optimal second difference to the second difference corresponding to the current candidate compression model.

[0146] Step S208: Whether the maximum number of iterations has been reached. If yes, execute step S209; if no, execute step S203. That is, determine whether the second convergence condition has been met.

[0147] Step S209: Outputting the optimal sparsity list. That is, when the second convergence condition is reached, the current sparsity combination is the optimal sparsity list (also referred to as the optimal sparsity combination), and the current candidate compression model is the target compression model.

[0148] In one embodiment, step 3 includes:

[0149] Step 31: For the same input data, obtain the output data of the first specified network layer in the current candidate compression model, and the output data of the network layer corresponding to each first specified network layer in the model to be compressed.

[0150] Step 32: Calculate the difference between the output data of each first designated network layer in the current candidate compression model and the output data of the network layer corresponding to the first designated network layer in the model to be compressed, and obtain the sub-difference corresponding to the first designated network layer.

[0151] Step 33: Based on the sub-differences corresponding to each first specified network layer, calculate the second difference corresponding to the current candidate compression model.

[0152] In an embodiment of the present application, the first designated network layer may be all network layers in the alternative compression model, or may be part of the network layers included in the alternative compression model. The number of first designated network layers may be set as needed without specific limitation. For example, the alternative compression model includes 11 network layers, and the first designated network layer may be any 4 network layers of the 11 network layers, such as the 2nd, 5th, 8th and 10th network layers of the 11 network layers. The network layers in the alternative compression model and the model to be compressed correspond one to one. For a first designated network layer in the alternative compression model, the first designated network layer is obtained by quantizing and sparsifying the network layer corresponding to the first designated network layer in the model to be compressed.

[0153] For the same input data, the output data of the first specified network layer in the latest alternative compression model and the output data of the network layer corresponding to each first specified network layer in the model to be compressed can be obtained. The difference between the output data of each first specified network layer in the current alternative compression model and the output data of the network layer corresponding to the first specified network layer in the model to be compressed can be calculated to obtain the sub-difference corresponding to the first specified network layer. Furthermore, the second difference corresponding to the current alternative compression model can be calculated based on the sub-difference corresponding to each first specified network layer. For example, the weighted sum of the sub-differences corresponding to each first specified network layer can be calculated to obtain the second difference corresponding to the current alternative compression model.

[0154] Based on the above processing, when the first designated network layer is all the network layers in the alternative compression model, the sub-differences corresponding to all the network layers in the latest alternative compression model can be combined to obtain the second difference. In this case, since the sub-differences corresponding to all the network layers in the latest alternative compression model are combined in the process of calculating the second difference, the accuracy of optimizing the sparsity combination according to the second difference is high, so as to ensure that the sparsity combination can be effectively optimized, and the effectiveness of the sparsity combination obtained by optimizing the sparsity combination according to the second difference is ensured. When the first designated network layer is part of the network layer in the alternative compression model, the number of output data used in calculating the second difference can be reduced, the amount of calculation of the second difference can be reduced, and the efficiency of calculating the second difference can be improved. Then, the efficiency of optimizing the sparsity combination according to the second difference can be improved, and the efficiency of compressing the model to be compressed can be improved.

[0155] In this way, the first designated network layer can be set as needed. While ensuring that the basis for adjusting the sparsity combination can be calculated, the accuracy and efficiency of optimizing the sparsity combination can be comprehensively considered. By adjusting the number of the first designated network layers, that is, adjusting the number of output data used when calculating the second difference, while ensuring the effectiveness of the sparsity combination obtained by subsequently optimizing the sparsity combination according to the second difference, the efficiency of subsequently optimizing the sparsity combination according to the second difference is improved, thereby improving the efficiency of compressing the model to be compressed.

[0156] In one embodiment, see Figure 3 , Figure 3 A second flow chart of the model compression method provided in an embodiment of the present application.

[0157] Step S301: Gradient-based continuous domain bit optimization, that is, steps S101 to S103 in the above embodiment.

[0158] Step S302: Obtaining an optimized floating point bit combination, that is, step S104 in the above embodiment.

[0159] Step S303: Align the floating point bits (upward alignment) to the bits supported by the hardware (2 bits, 4 bits, 8 bits, 16 bits, etc.) to obtain an optimal quantization bit width list, that is, step S105 in the above embodiment.

[0160] Step S304: Obtain the redundancy difference of the expression power of the aligned bit combination between different layers under the hardware constraint condition, that is, calculate the difference between the discrete domain quantization bit width and the continuous domain quantization bit width of any network layer to obtain the redundancy of the network layer.

[0161] Step S305: using the redundancy difference, determine the theoretical order of sparsity between different layers. That is, the size relationship of the redundancy of each network layer is determined as the size relationship of the sparsity of each network layer.

[0162] Step S306: Evolutionary search (based on theoretical sorting of sparsity) to obtain an optimal sparsity list, that is, steps 1 to 4 in the above embodiment.

[0163] Step S307: synthesize the sparse quantization model with the best compression ratio, that is, obtain the target compression model.

[0164] The model compression method provided in the embodiment of the present application can integrate sparsification and quantization to achieve compression of large models to accelerate the reasoning of large models. The balance problem of the compression ratio of sparsification and quantization is comprehensively considered, and while ensuring the performance of the model, a sparse quantization model (i.e., a target compression model) with the highest overall compression ratio is achieved. And because the model compression method provided in the embodiment of the present application can align the continuous domain quantization bit width of the floating-point type upward to the discrete domain quantization bit width supported by the hardware platform, and the sparsity can be adjusted, it can flexibly support hardware platforms with any quantization specifications and sparse specifications, and has high compatibility.

[0165] In one embodiment, see Figure 4 , Figure 4A third flow chart of the model compression method provided in an embodiment of the present application. Taking the example of a model to be compressed comprising three network layers, the continuous domain quantization bit width (which can be expressed as float_bits) of each network layer (which can be expressed as layer) is (3.4, 10.2, 6.2) bits. By rounding the bits and aligning them to the hardware, that is, in step S105 in the above embodiment, the discrete domain quantization bit width (which can be expressed as bits) of each network layer is obtained to be (4, 16, 8) bits. The bit rounding operation produces redundancy of expressive power. The difference between the discrete domain quantization bit width and the continuous domain quantization bit width of any network layer can be calculated to obtain the redundancy of the network layer (also called expressive power redundancy), such as Figure 4 As shown in , the redundancy of each network layer is (0.6, 5.8, 1.8). According to the redundancy of the quantization domain, the optimal sparsity of each layer (also called the optimal sparsity) is obtained through evolutionary search, that is, the optimal sparsity combination containing the optimal sparsity of each network layer is obtained. Figure 4 As shown, the optimal sparsity combination is (10%, 40%, 20%), that is, (0.1, 0.4, 0.2). According to the discrete domain quantization bit width and the optimal sparsity combination of each network layer, the compression model is quantized and sparsed to obtain a sparse quantization model with the best comprehensive compression ratio (that is, the target compression model in the above embodiment).

[0166] Based on the same inventive concept, the present application also provides a model compression device, see Figure 5 , Figure 5 A schematic diagram of the structure of a model compression device provided in an embodiment of the present application, the device comprising:

[0167] The data processing module 501 is used to process the same input data using the model to be compressed and the current quantization model respectively; wherein the current quantization model is obtained by quantizing the first network parameter of the network layer according to the quantization bit width obtained based on the current quantization parameter of each network layer in the model to be compressed; the quantization parameter of any network layer includes the truncation coefficient and quantization interval of the network layer;

[0168] A first difference calculation module 502, used to calculate the difference between the output data obtained by the model to be compressed and the current quantization model as a first difference corresponding to the current quantization model;

[0169] The quantization parameter adjustment module 503 is used to adjust the current quantization parameter of each network layer in the model to be compressed according to the first difference corresponding to the current quantization model, until the first convergence condition is reached, so as to obtain the optimal quantization parameter of each network layer;

[0170] A continuous domain quantization bit width calculation module 504, configured to calculate the continuous domain quantization bit width of each network layer based on the optimal quantization parameter of the network layer;

[0171] The discrete domain quantization bit width determination module 505 is used to determine, from the discrete domain quantization bit widths supported by the hardware platform on which the model needs to be deployed, a bit width that is larger than the continuous domain quantization bit width of the network layer and has the smallest difference with the continuous domain quantization bit width of the network layer, as the discrete domain quantization bit width of the network layer;

[0172] A target compression model determination module 506 is used to determine, from among the candidate compression models obtained by respectively compressing using multiple sparsity combinations, a corresponding candidate compression model with the smallest second difference as the target compression model;

[0173] Among them, an alternative compression model is: using the discrete domain quantization bit width of each network layer to quantize the first network parameter of the network layer, and using a sparsity combination to sparsify the second network parameters of each network layer after quantization; the second difference corresponding to an alternative compression model is represented by: the difference between the output data obtained by the alternative compression model and the model to be compressed when processing the same input data; the size relationship of the sparsity of each network layer included in any sparsity combination is consistent with the size relationship of the redundancy of each network layer; the redundancy of any network layer is represented by: the difference between the discrete domain quantization bit width and the continuous domain quantization bit width of the network layer.

[0174] Based on the model compression device provided by the embodiment of the present application, the quantization parameters of each network layer in the model that currently needs to be compressed (i.e., the model to be compressed) can be optimized to obtain the optimal quantization parameters of each network layer. According to the quantization bit width obtained based on the optimal quantization parameters of each network layer, the performance difference between the quantization model obtained by quantizing the compressed model and the model to be compressed is minimized. Based on the optimal quantization parameters of each network layer, the continuous domain quantization bit width of the network layer can be calculated, and the quantization bit width (i.e., discrete domain quantization bit width) that can be used in actual quantization can be determined in combination with the discrete domain quantization bit widths supported by the hardware platform where the model needs to be deployed, so as to ensure that the compressed model can be quantized according to the discrete domain quantization bit width. And because the discrete domain quantization bit width is: the bit width that is greater than the continuous domain quantization bit width and has the smallest difference with the continuous domain quantization bit width among the discrete domain quantization bit widths supported by the hardware platform, it is possible to ensure that the compression ratio of the model quantized is optimal while quantizing the model to be compressed according to the discrete domain quantization bit width to obtain the performance of the model.

[0175] The difference (i.e., redundancy) between the discrete domain quantization bit width and the continuous domain quantization bit width of any network layer can represent the redundancy of the expression capability of the network layer obtained by quantizing the first network parameter of the network layer according to the discrete domain quantization bit width of the network layer, relative to the network layer obtained by quantizing the first network parameter of the network layer according to the continuous domain quantization bit width of the network layer. The higher the redundancy, the higher the degree to which the network layer obtained by quantizing the network layer according to the discrete domain quantization bit width of the network layer can be further compressed, that is, the higher the degree to which the quantized network layer can be subsequently sparse. Furthermore, multiple sparsity combinations can be determined, and the size relationship of the sparsity of each network layer contained in each sparsity combination is consistent with the size relationship of the redundancy of each network layer. From the various alternative compression models obtained by compressing the model to be compressed using each sparsity combination, the target compression model is determined. The performance difference between the target compression model and the model to be compressed is minimal. In this way, the model to be compressed can be compressed by combining quantization and sparsification, which can effectively improve the degree of compression of the model while ensuring the performance of the compressed model, reduce the storage and computing resources required by the model, and improve the applicability of the model.

[0176] In one embodiment, the target compression model determination module 506 includes:

[0177] A model sparsification submodule, used to quantize and sparsify the model to be compressed according to the discrete domain quantization bit width of each network layer and the current sparsity combination, so as to obtain the current candidate compression model; wherein each sparsity included in the sparsity combination used when obtaining the candidate compression model for the first time is a preset initial value;

[0178] A data processing submodule, used for processing the same input data using the to-be-compressed model and the current candidate compression model respectively;

[0179] A second difference calculation submodule, used for calculating the difference between the output data obtained by the model to be compressed and the current candidate compression model as the second difference corresponding to the current candidate compression model;

[0180] The sparsity combination updating submodule is used to update the current sparsity combination to obtain a new sparsity combination when the second difference corresponding to the current alternative compression model is smaller than the current optimal second difference, and update the current optimal second difference to the second difference corresponding to the current alternative compression model; when the second difference corresponding to the current alternative compression model is not smaller than the current optimal second difference, update the sparsity combination used to obtain the current optimal second difference to obtain a new sparsity combination; trigger the model sparsification submodule until the second convergence condition is reached to obtain the target compression model.

[0181] In one embodiment, the second convergence condition is that the difference between the second difference corresponding to the current candidate compression model and the current optimal second difference is less than a preset threshold, or the number of times the optimal second difference is updated is a preset number.

[0182] In one embodiment, the second difference calculation submodule is specifically used to obtain, for the same input data, the output data of the first specified network layer in the current alternative compression model and the output data of the network layer corresponding to each first specified network layer in the model to be compressed; calculate the difference between the output data of each first specified network layer in the current alternative compression model and the output data of the network layer corresponding to the first specified network layer in the model to be compressed, to obtain the sub-difference corresponding to the first specified network layer; based on the sub-differences corresponding to each first specified network layer, calculate the second difference corresponding to the current alternative compression model.

[0183] In one embodiment, the first difference calculation module 502 is specifically used to obtain, for the same input data, the output data of the second specified network layer in the current quantization model and the output data of the network layer corresponding to each second specified network layer in the model to be compressed; calculate the difference between the output data of each second specified network layer in the current quantization model and the output data of the network layer corresponding to the second specified network layer in the model to be compressed, and obtain the sub-difference corresponding to the second specified network layer; based on the sub-differences corresponding to each second specified network layer, calculate the first difference corresponding to the current quantization model.

[0184] In one embodiment, the continuous domain quantization bit width calculation module 504 is specifically used to calculate the continuous domain quantization bit width of each network layer according to a preset formula based on the optimal quantization parameter of the network layer; wherein the preset formula is as follows:

[0185]

[0186] Indicates The continuous domain quantization bit width of each network layer is Indicates The optimal quantization parameters of the network layer include the truncation coefficients, Indicates The optimal quantization parameters of each network layer include the quantization interval.

[0187] In one embodiment, the first network parameter of any network layer includes: the weight and / or activation value of the network layer; the second network parameter of any network layer includes: the weight and / or activation value of the network layer.

[0188] The present application also provides an electronic device, such as Figure 6 As shown, including:

[0189] Memory 601, used for storing computer programs;

[0190] The processor 602 is used to execute the program stored in the memory 601 to implement the following steps:

[0191] The same input data is processed using the model to be compressed and the current quantization model respectively; wherein the current quantization model is obtained by quantizing the first network parameter of the network layer according to the quantization bit width obtained based on the current quantization parameter of each network layer in the model to be compressed; the quantization parameter of any network layer includes the truncation coefficient and quantization interval of the network layer;

[0192] Calculating a difference between output data obtained by the model to be compressed and the current quantization model as a first difference corresponding to the current quantization model;

[0193] According to the first difference corresponding to the current quantization model, the current quantization parameter of each network layer in the model to be compressed is adjusted until the first convergence condition is reached, thereby obtaining the optimal quantization parameter of each network layer;

[0194] Based on the optimal quantization parameter of each network layer, the continuous domain quantization bit width of the network layer is calculated;

[0195] From the discrete domain quantization bit widths supported by the hardware platform where the model needs to be deployed, determine a bit width that is larger than the continuous domain quantization bit width of the network layer and has the smallest difference with the continuous domain quantization bit width of the network layer as the discrete domain quantization bit width of the network layer;

[0196] Determine, from among the candidate compression models obtained by respectively compressing using multiple sparsity combinations, a candidate compression model with the smallest corresponding second difference as a target compression model;

[0197] Among them, an alternative compression model is: using the discrete domain quantization bit width of each network layer to quantize the first network parameter of the network layer, and using a sparsity combination to sparsify the second network parameters of each network layer after quantization; the second difference corresponding to an alternative compression model is represented by: the difference between the output data obtained by the alternative compression model and the model to be compressed when processing the same input data; the size relationship of the sparsity of each network layer included in any sparsity combination is consistent with the size relationship of the redundancy of each network layer; the redundancy of any network layer is represented by: the difference between the discrete domain quantization bit width and the continuous domain quantization bit width of the network layer.

[0198] Furthermore, the electronic device may further include a communication bus and / or a communication interface, and the processor 602, the communication interface, and the memory 601 communicate with each other via the communication bus.

[0199] The communication bus mentioned in the above electronic device can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0200] The communication interface is used for communication between the above electronic device and other devices.

[0201] The memory may include a random access memory (RAM) or a non-volatile memory (NVM), such as at least one disk memory. Optionally, the memory may also be at least one storage device located away from the aforementioned processor.

[0202] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0203] In another embodiment provided in the present application, a computer-readable storage medium is provided, in which a computer program is stored. When the computer program is executed by a processor, the steps of any of the above-mentioned model compression methods are implemented.

[0204] In another embodiment provided by the present application, a computer program product including instructions is also provided, which, when executed on a computer, enables the computer to execute any model compression method in the above embodiments.

[0205] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from a website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a solid-state hard disk (SSD), etc.

[0206] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0207] Each embodiment in this specification is described in a related manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device, electronic device, computer-readable storage medium, and computer program product embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiments.

[0208] The above description is only a preferred embodiment of the present application and is not intended to limit the protection scope of the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application are included in the protection scope of the present application.

Claims

1. A model compression method, characterized in that: The method comprises: The same input data is processed using the model to be compressed and the current quantization model respectively; wherein the current quantization model is obtained by quantizing the first network parameter of the network layer according to the quantization bit width obtained based on the current quantization parameter of each network layer in the model to be compressed; the quantization parameter of any network layer includes the truncation coefficient and quantization interval of the network layer; Calculating a difference between output data obtained by the model to be compressed and the current quantization model as a first difference corresponding to the current quantization model; According to the first difference corresponding to the current quantization model, the current quantization parameter of each network layer in the model to be compressed is adjusted until the first convergence condition is reached, thereby obtaining the optimal quantization parameter of each network layer; Based on the optimal quantization parameter of each network layer, the continuous domain quantization bit width of the network layer is calculated; From the discrete domain quantization bit widths supported by the hardware platform where the model needs to be deployed, determine a bit width that is larger than the continuous domain quantization bit width of the network layer and has the smallest difference with the continuous domain quantization bit width of the network layer as the discrete domain quantization bit width of the network layer; Determine, from among the candidate compression models obtained by respectively compressing using multiple sparsity combinations, a candidate compression model with the smallest corresponding second difference as a target compression model; Among them, an alternative compression model is: using the discrete domain quantization bit width of each network layer to quantize the first network parameter of the network layer, and using a sparsity combination to sparsify the second network parameters of each network layer after quantization; the second difference corresponding to an alternative compression model is represented by: the difference between the output data obtained by the alternative compression model and the model to be compressed when processing the same input data; the size relationship of the sparsity of each network layer included in any sparsity combination is consistent with the size relationship of the redundancy of each network layer; the redundancy of any network layer is represented by: the difference between the discrete domain quantization bit width and the continuous domain quantization bit width of the network layer.

2. The method according to claim 1, characterized in that The step of determining, from among the candidate compression models obtained by respectively compressing using multiple sparsity combinations, a candidate compression model with the smallest corresponding second difference as the target compression model, includes: According to the discrete domain quantization bit width of each network layer and the current sparsity combination, the model to be compressed is quantized and sparsified to obtain the current candidate compression model; wherein each sparsity included in the sparsity combination used when the candidate compression model is obtained for the first time is a preset initial value; For the same input data, the to-be-compressed model and the current candidate compression model are used for processing respectively; Calculating a difference between output data obtained by the model to be compressed and the current candidate compression model as a second difference corresponding to the current candidate compression model; When the second difference corresponding to the current candidate compression model is less than the current optimal second difference, the current sparsity combination is updated to obtain a new sparsity combination, and the current optimal second difference is updated to the second difference corresponding to the current candidate compression model; when the second difference corresponding to the current candidate compression model is not less than the current optimal second difference, the sparsity combination used to obtain the current optimal second difference is updated to obtain a new sparsity combination; Return to the step of quantizing and sparsifying the model to be compressed according to the discrete domain quantization bit width of each network layer and the current sparsity combination to obtain the current candidate compression model, until the second convergence condition is reached to obtain the target compression model.

3. The method according to claim 2, characterized in that The second convergence condition is that the difference between the second difference corresponding to the current candidate compression model and the current optimal second difference is less than a preset threshold, or the number of times the optimal second difference is updated is a preset number.

4. The method according to claim 2, characterized in that: Calculating a difference between output data obtained by the model to be compressed and the current candidate compression model as a second difference corresponding to the current candidate compression model includes: For the same input data, obtaining output data of the first specified network layer in the current candidate compression model and output data of the network layer corresponding to each first specified network layer in the model to be compressed; Calculate the difference between the output data of each first specified network layer in the current candidate compression model and the output data of the network layer corresponding to the first specified network layer in the to-be-compressed model, and obtain the sub-difference corresponding to the first specified network layer; Based on the sub-differences corresponding to the first specified network layers, a second difference corresponding to the current candidate compression model is calculated.

5. The method according to claim 1, characterized in that Calculating a difference between output data obtained by the model to be compressed and the current quantization model as a first difference corresponding to the current quantization model includes: For the same input data, obtaining output data of the second specified network layer in the current quantization model, and output data of the network layer corresponding to each second specified network layer in the model to be compressed; Calculate the difference between the output data of each second specified network layer in the current quantization model and the output data of the network layer corresponding to the second specified network layer in the model to be compressed, and obtain the sub-difference corresponding to the second specified network layer; Based on the sub-differences corresponding to the second specified network layers, a first difference corresponding to the current quantization model is calculated.

6. The method according to any one of claims 1 to 5, characterized in that: The step of calculating the continuous domain quantization bit width of each network layer based on the optimal quantization parameter of the network layer includes: Based on the optimal quantization parameter of each network layer, the continuous domain quantization bit width of the network layer is calculated according to a preset formula; wherein the preset formula is as follows: ； Indicates The continuous domain quantization bit width of each network layer is Indicates The optimal quantization parameters of the network layer include the truncation coefficients, Indicates The optimal quantization parameters of each network layer include the quantization interval.

7. The method according to any one of claims 1 to 5, characterized in that: The first network parameter of any network layer includes: the weight and / or activation value of the network layer; the second network parameter of any network layer includes: the weight and / or activation value of the network layer.

8. A model compression device, characterized in that: The device comprises: A data processing module, used for processing the same input data using the model to be compressed and the current quantization model respectively; wherein the current quantization model is obtained by quantizing the first network parameter of the network layer according to the quantization bit width obtained based on the current quantization parameter of each network layer in the model to be compressed; the quantization parameter of any network layer includes the truncation coefficient and quantization interval of the network layer; A first difference calculation module, used to calculate the difference between the output data obtained by the model to be compressed and the current quantization model as a first difference corresponding to the current quantization model; A quantization parameter adjustment module, configured to adjust the current quantization parameter of each network layer in the model to be compressed according to the first difference corresponding to the current quantization model, until the first convergence condition is reached, thereby obtaining the optimal quantization parameter of each network layer; A continuous domain quantization bit width calculation module, used to calculate the continuous domain quantization bit width of each network layer based on the optimal quantization parameter of the network layer; A discrete domain quantization bit width determination module is used to determine, from among various discrete domain quantization bit widths supported by the hardware platform on which the model needs to be deployed, a bit width that is greater than the continuous domain quantization bit width of the network layer and has the smallest difference with the continuous domain quantization bit width of the network layer, as the discrete domain quantization bit width of the network layer; A target compression model determination module, used to determine, from among the candidate compression models obtained by respectively compressing using multiple sparsity combinations, a corresponding candidate compression model with the smallest second difference as the target compression model; Among them, an alternative compression model is: using the discrete domain quantization bit width of each network layer to quantize the first network parameter of the network layer, and using a sparsity combination to sparsify the second network parameters of each network layer after quantization; the second difference corresponding to an alternative compression model is represented by: the difference between the output data obtained by the alternative compression model and the model to be compressed when processing the same input data; the size relationship of the sparsity of each network layer included in any sparsity combination is consistent with the size relationship of the redundancy of each network layer; the redundancy of any network layer is represented by: the difference between the discrete domain quantization bit width and the continuous domain quantization bit width of the network layer.

9. The device according to claim 8, characterized in that The target compression model determination module includes: A model sparsification submodule, used to quantize and sparsify the model to be compressed according to the discrete domain quantization bit width of each network layer and the current sparsity combination, so as to obtain the current candidate compression model; wherein each sparsity included in the sparsity combination used when obtaining the candidate compression model for the first time is a preset initial value; A data processing submodule, used for processing the same input data using the to-be-compressed model and the current candidate compression model respectively; A second difference calculation submodule, used to calculate the difference between the output data obtained by the model to be compressed and the current candidate compression model as the second difference corresponding to the current candidate compression model; The sparsity combination updating submodule is used to update the current sparsity combination to obtain a new sparsity combination when the second difference corresponding to the current candidate compression model is less than the current optimal second difference, and update the current optimal second difference to the second difference corresponding to the current candidate compression model; when the second difference corresponding to the current candidate compression model is not less than the current optimal second difference, update the sparsity combination used to obtain the current optimal second difference to obtain a new sparsity combination; trigger the model sparsification submodule until the second convergence condition is reached to obtain the target compression model; The second convergence condition is that: the difference between the second difference corresponding to the current candidate compression model and the current optimal second difference is less than a preset threshold, or the number of times the optimal second difference is updated is a preset number; The second difference calculation submodule is specifically used to obtain, for the same input data, the output data of the first specified network layer in the current candidate compression model and the output data of the network layer corresponding to each first specified network layer in the model to be compressed; calculate the difference between the output data of each first specified network layer in the current candidate compression model and the output data of the network layer corresponding to the first specified network layer in the model to be compressed, to obtain the sub-difference corresponding to the first specified network layer; based on the sub-differences corresponding to the first specified network layers, calculate the second difference corresponding to the current candidate compression model; The first difference calculation module is specifically used to obtain, for the same input data, the output data of the second specified network layer in the current quantization model and the output data of the network layer corresponding to each second specified network layer in the model to be compressed; calculate the difference between the output data of each second specified network layer in the current quantization model and the output data of the network layer corresponding to the second specified network layer in the model to be compressed, to obtain the sub-difference corresponding to the second specified network layer; based on the sub-differences corresponding to each second specified network layer, calculate the first difference corresponding to the current quantization model; The continuous domain quantization bit width calculation module is specifically used to calculate the continuous domain quantization bit width of each network layer according to a preset formula based on the optimal quantization parameter of the network layer; wherein the preset formula is as follows: ； Indicates The continuous domain quantization bit width of each network layer is Indicates The optimal quantization parameters of the network layer include the truncation coefficients, Indicates The optimal quantization parameters of each network layer include the quantization interval; The first network parameter of any network layer includes: the weight and / or activation value of the network layer; the second network parameter of any network layer includes: the weight and / or activation value of the network layer.

10. An electronic device, characterized in that: include: Memory, used to store computer programs; A processor, for implementing any of the methods described in claims 1-7 when executing a program stored in a memory.

11. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

12. A computer program product, characterized in that When the computer program product is run on a computer, the computer is enabled to execute the method according to any one of claims 1 to 7.

Images