Model compression method, system, terminal and storage medium

Abstract

The application provides a model compression method, system, terminal and storage medium, the method comprises the following steps: model training is carried out on a to-be-compressed model, a regularization term and singular value decomposition are added in the to-be-compressed model, a singular value matrix is obtained, the step of executing model training on the to-be-compressed model and subsequent steps are returned according to the singular value matrix, until the to-be-compressed model meets a performance decline condition, and the to-be-compressed model is output; parameter clustering is carried out according to a weight tensor of the to-be-compressed model, a weight parameter matrix is obtained, weight quantization is carried out on the weight parameter matrix, and a clustered quantization matrix is obtained; parameter setting is carried out on the to-be-compressed model according to the clustered quantization matrix, and a compressed model is obtained. The application realizes maximum model compression on the premise of ensuring accuracy by jointly compressing the model from the global perspective based on the joint model compression mode of sparse regularization, iterative pruning and clustered quantization.

Description

Technical Field

[0001] This invention relates to the field of model processing technology, and in particular to a model compression method, system, terminal, and storage medium. Background Technology

[0002] Machine learning is now widely used across various industries. While complex models offer better performance, their high storage space and computational resource consumption are major factors limiting their application efficiency. Therefore, model compression methods are receiving increasing attention.

[0003] In existing model compression processes, unimportant channels, convolutional kernels, or certain layers are typically removed directly from the model, thereby reducing the number of parameters and computational load. However, because these channels, convolutional kernels, or network layers are directly removed, the compressed model has lower performance, which reduces the user experience. Summary of the Invention

[0004] The purpose of this invention is to provide a model compression method, system, terminal, and storage medium, aiming to solve the problem of low performance of compressed models in existing model compression processes.

[0005] The present invention is implemented as follows: a model compression method, the method comprising:

[0006] Obtain the model to be compressed and train the model on the model to be compressed;

[0007] After model training, a regularization term is added to the model to be compressed, and the weight tensor of the model to be compressed after adding the regularization term is decomposed into singular value matrix. The regularization term is used to sparsify the weight parameters in the model to be compressed.

[0008] Based on the singular value matrix, return to execute the steps of training the model to be compressed and subsequent steps until the model to be compressed meets the performance degradation condition, and output the model to be compressed.

[0009] Based on the weight tensor of the output model to be compressed, parameter clustering is performed to obtain a weight parameter matrix, and the weight parameter matrix is ​​then weighted to obtain a clustering quantization matrix.

[0010] The parameters of the model to be compressed are set according to the clustering quantization matrix to obtain the compressed model.

[0011] Furthermore, before the step of performing model training on the model to be compressed and subsequent steps based on the singular value matrix, the method further includes:

[0012] The test data is input into the trained model to be compressed for data processing to obtain output data, and the standard data of the test data is obtained.

[0013] Based on the standard data and the output data, the performance of the model to be compressed is tested to obtain subjective evaluation index, audio intelligibility index and speech quality evaluation index.

[0014] If the subjective evaluation index, the audio intelligibility index, and the speech quality evaluation index meet the preset index conditions, then the model to be compressed is determined to meet the performance degradation condition.

[0015] Furthermore, the step of performing performance testing on the model to be compressed based on the standard data and the output data includes:

[0016] Obtain the output image from the output data, and obtain the standard image from the standard data;

[0017] The signal-to-noise ratio is calculated based on the standard image and the output image to obtain the subjective evaluation index;

[0018] Obtain the output audio from the output data, and obtain the standard audio from the standard data;

[0019] The audio intelligibility index is obtained by performing a short-time objective intelligibility calculation based on the output audio and the standard audio.

[0020] Obtain the output speech from the output data, and obtain the standard speech from the standard data;

[0021] Based on the output speech and the standard speech, a perceptual speech quality evaluation calculation is performed to obtain the speech quality evaluation index.

[0022] Furthermore, adding a regularization term to the model to be compressed after model training includes:

[0023] Query the encoding layers in the model to be compressed, and add a first regularization term to the normalization layer of each encoding layer;

[0024] Query the long short-term memory network layers in the model to be compressed, and add a second regularization term to the layer of each long short-term memory network layer;

[0025] The first regular expression term includes:

[0026]

[0027] The second regular expression term includes:

[0028]

[0029] Where L1 is the first regularization term, L2 is the second regularization term, and μ β and δ β denoted as the mean and variance of the activation values ​​in the model to be compressed, respectively; γ and β are affine transformation parameters; and λ is the penalty coefficient factor.

[0030] Furthermore, the formula used for singular value decomposition of the weight tensor of the model to be compressed after adding the regularization term includes:

[0031] A=UΣV T

[0032] Where U is an m×m matrix, Σ is an m×n matrix, Σ has all elements except those on the main diagonal as zero, and each element on the main diagonal of Σ is a singular value, V is an n×n matrix, and both U and V are unitary matrices satisfying UU T =I,VV T =I, where A is the weight tensor.

[0033] Furthermore, the step of weight quantization of the weight parameter matrix to obtain a clustering quantization matrix includes:

[0034] The weight parameters in the weight parameter matrix are matched with the pre-stored clustering reference table to obtain the cluster centroid values. The clustering quantization matrix is ​​then constructed based on the cluster centroid values. The clustering reference table stores the correspondence between different weight parameters and their corresponding cluster centroid values.

[0035] Furthermore, the formula used for parameter clustering based on the output weight tensor of the model to be compressed includes:

[0036]

[0037] Among them, w min and w max These represent the minimum and maximum values ​​of the weight tensor, respectively.

[0038] Another objective of this invention is to provide a model compression system, the system comprising:

[0039] The model training module is used to acquire the model to be compressed and to train the model on the model to be compressed.

[0040] The singular value decomposition module is used to add a regularization term to the model to be compressed after model training, and to perform singular value decomposition on the weight tensor of the model to be compressed after adding the regularization term to obtain a singular value matrix. The regularization term is used to sparsify the weight parameters in the model to be compressed.

[0041] The model output module is used to return the steps of training the model to be compressed and subsequent steps based on the singular value matrix until the model to be compressed meets the performance degradation condition, and then output the model to be compressed.

[0042] The parameter clustering module is used to perform parameter clustering based on the weight tensor of the output model to be compressed, to obtain a weight parameter matrix, and to perform weight quantization on the weight parameter matrix to obtain a clustering quantization matrix.

[0043] The parameter setting module is used to set the parameters of the model to be compressed based on the clustering quantization matrix to obtain the compressed model.

[0044] Another objective of this invention is to provide a terminal device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the method described above.

[0045] Another objective of this invention is to provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the above-described method.

[0046] In this embodiment of the invention, by adding a regularization term to the model to be compressed after model training, the sparsity of the weight tensor in the model to be compressed is increased, thereby achieving a higher pruning rate without sacrificing model performance. This ensures the performance of the compressed model output after compression. By returning the steps of model training and subsequent steps through the singular value matrix, the sparsity of the weight parameters in the model to be compressed can be continuously improved, thereby achieving an iterative pruning effect and improving the pruning effect of the model to be compressed. By performing parameter clustering through the weight tensor of the model to be compressed, the parameters in the model to be compressed can be effectively clustered and quantized, further improving the model compression effect. Attached Figure Description

[0047] Figure 1 This is a flowchart of the model compression method provided in the first embodiment of the present invention;

[0048] Figure 2 This is a schematic diagram of the structure of the model to be compressed provided in the first embodiment of the present invention;

[0049] Figure 3 This is a schematic diagram of the parameter clustering process of the weight tensor provided in the first embodiment of the present invention;

[0050] Figure 4 This is a flowchart of the model compression method provided in the second embodiment of the present invention;

[0051] Figure 5 This is a schematic diagram of the model compression system provided in the third embodiment of the present invention;

[0052] Figure 6 This is a schematic diagram of the structure of the terminal device provided in the fourth embodiment of the present invention. Detailed Implementation

[0053] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0054] To illustrate the technical solution described in this invention, specific embodiments are described below.

[0055] Example 1

[0056] Please see Figure 1 This is a flowchart of a model compression method provided in the first embodiment of the present invention. This model compression method can be applied to any terminal device or system, and includes the following steps:

[0057] Step S10: Obtain the model to be compressed and train the model to be compressed;

[0058] The model to be compressed can be configured according to requirements; please refer to [link / reference]. Figure 2 The model to be compressed in this step is a Diffused Convolutional Recurrent Neural Network (DCRNN) model. The DCRNN model consists of a seven-layer encoder layer, a two-layer long short-term memory network layer, and a one-layer fully connected layer. The encoder layer includes a convolutional neural network (CNN), BatchNorm2d, and PReLU. The long short-term memory network layer includes a long short-term memory network (LSTM) and LayerNorm. BatchNorm2d and LayerNorm are merged into one layer.

[0059] In this step, the importance of the weights in the model to be compressed is not distinguished before model training. By training the model, the important and unimportant weights in the model to be compressed can be distinguished, so as to preserve the optimal number of model parameters and facilitate the subsequent sparsification process of the model to be compressed.

[0060] Step S20: Add a regularization term to the model to be compressed after model training, and perform singular value decomposition on the weight tensor of the model to be compressed after adding the regularization term to obtain a singular value matrix.

[0061] The regularization term is used to sparsify the weight parameters in the model to be compressed. By adding the regularization term to the model to be compressed after training, the unimportant weights in the model to be compressed can be sparsified without affecting the accuracy of the model to be compressed, so as to achieve the effect of sparsity regularization and ensure the effect of subsequent model pruning. In this step, based on sparsity regularization, the number of model parameters and computation can be minimized to the maximum extent, so that the compressed model to be compressed can be deployed on latency-sensitive applications or resource-limited devices.

[0062] Optionally, in this step, adding a regularization term to the model to be compressed after model training includes:

[0063] Query the encoding layers in the model to be compressed, and add a first regularization term to the normalization layer of each encoding layer;

[0064] Query the long short-term memory network layers in the model to be compressed, and add a second regularization term to the layer of each long short-term memory network layer;

[0065] Specifically, for the encoding layer, a first regularization term is introduced in the BatchNorm layer. The BatchNorm layer operates on channels and can sparsify the weights of less important channels to 0, which is beneficial for pruning. For pruning of the Long Short-Term Memory network layer, a second regularization term is introduced in the LayerNorm layer. The LayerNorm layer operates on hidden-to-hidden and can sparsify the weights of less important hidden-to-hidden channels to 0, which is also convenient for pruning.

[0066] Furthermore, the first regularization term includes:

[0067]

[0068] The second regular expression term includes:

[0069]

[0070] Where L1 is the first regularization term, L2 is the second regularization term, and μ β δ and β are the mean and variance of the activation values ​​in the model to be compressed, respectively; γ and β are affine transformation parameters; and λ is the penalty coefficient factor.

[0071] In this step, by adding a first regularization term to the normalization layer of each encoding layer and a second regularization term to the normalization layer of each Long Short-Term Memory (LSTM) network layer, the problem that the compressed model cannot be pruned from a global perspective in existing technologies is effectively solved. In this step, when performing sparsity training on the compressed model, λ is 10 when the epoch is 50%. -4When the epoch is greater than 50%, λ is 10. -6 This enables the rapid recovery of precision loss caused by sparsity. One epoch represents the process of all data being fed into the network, completing one forward computation and backpropagation.

[0072] Furthermore, in this step, the formula used for singular value decomposition of the weight tensor of the model to be compressed after adding the regularization term includes:

[0073] A=UΣV T

[0074] Where U is an m×m matrix, Σ is an m×n matrix, Σ has all elements except those on the main diagonal as zero, and each element on the main diagonal of Σ is a singular value, V is an n×n matrix, and both U and V are unitary matrices satisfying UU T =I,VV T =I, where A is the weight tensor.

[0075] Furthermore, for each layer of the network in the model to be compressed, the pruning ratio is determined by performing singular value decomposition on all mean tensors to obtain eigenvalues, thus achieving the effect of tensor-by-tensor pruning sensitivity analysis and determining the pruning rate. Singular values ​​are arranged in descending order in the singular value matrix, and the decrease in singular values ​​is relatively rapid. Generally, the sum of the first 10% or even 1% of singular values ​​accounts for more than 99% of the sum of all singular values. In this embodiment, the singular value corresponding to 97% is used as the threshold to determine the pruning rate of each layer.

[0076] Step S30: Based on the singular value matrix, return to the step of training the model to be compressed and subsequent steps until the model to be compressed meets the performance degradation condition, and output the model to be compressed;

[0077] Among them, by returning the steps of training the model to be compressed and subsequent steps through the singular value matrix, the sparsity of the weight parameters in the model to be compressed can be continuously improved, so as to achieve the iterative pruning effect of the model to be compressed and improve the pruning effect of the model to be compressed.

[0078] In this step, the performance degradation condition is used to detect whether the performance of the model to be compressed has degraded after model training. In this step, when the performance of the model to be compressed after model training is detected to have degraded, it is determined that the model to be compressed meets the performance degradation condition.

[0079] Step S40: Perform parameter clustering based on the output weight tensor of the model to be compressed to obtain a weight parameter matrix, and perform weight quantization on the weight parameter matrix to obtain a clustering quantization matrix.

[0080] In this step, the formula used for parameter clustering based on the output weight tensor of the model to be compressed includes:

[0081]

[0082] Among them, w min and w max These represent the minimum and maximum values ​​of the weight tensor in the model to be compressed, respectively, using a uniform interval in [w min ,w max The weight tensor within the interval is used to initialize the cluster centers. When the clustering algorithm converges, all weights belonging to the same cluster are reset to the values ​​of the corresponding centroids, resulting in the weight parameter matrix. Please refer to [reference needed]. Figure 3 The original weights are approximated by the parameters in the weight parameter matrix. Based on the parameter clustering of the weight tensor of the model to be compressed, the number of effective weight values ​​that need to be stored in the model to be compressed is effectively reduced. In the weight parameter matrix, each weight can be represented as a clustering index.

[0083] Optionally, in this step, the weight parameter matrix is ​​weighted to obtain a clustering quantization matrix, which includes:

[0084] The weight parameters in the weight parameter matrix are matched with the pre-stored clustering lookup table to obtain the cluster centroid values, and the clustering quantization matrix is ​​obtained by constructing a matrix based on the cluster centroid values.

[0085] To further compress the model, cluster quantization is proposed, which divides the weights of each weight tensor into K clusters S1, S2, ..., S... k K-means clustering is used. The cluster lookup table stores the correspondence between different weight parameters and their corresponding cluster centroid values. In the cluster lookup table, each weight parameter is bound to a corresponding cluster index, which is used to determine the corresponding cluster centroid value in the cluster lookup table.

[0086] Specifically, each weight parameter is quantized to log2Kbit. Each weight parameter requires log2Kbits to store its corresponding cluster index. For example, if the initial weight parameter is a 32-bit floating-point number, an additional 32Kbits are needed to store the code. Therefore, the compression ratio of the quantization is calculated as follows:

[0087]

[0088] Wherein, represents the number of weights in the weight tensor N, and K represents the number of clusters. For cluster-based quantization, choosing a suitable value of K is the key to achieving quantization. In this embodiment, K is set to 8 bits, which reduces the number of parameters of the model to be compressed by nearly 5 times and the amount of computation by 1 time without losing model performance, thus improving the model compression effect.

[0089] Step S50: Set the parameters of the model to be compressed according to the clustering quantization matrix to obtain the compressed model;

[0090] In this method, the weight parameters in the model to be compressed are set by using the clustering quantization matrix, which effectively compresses the weight parameters in the model and reduces the number of parameters and computational cost in the model to be compressed.

[0091] This embodiment provides a joint model compression method based on sparse regularization, iterative pruning, and cluster quantization. Sparse regularization increases the sparsity of the weight tensors in the model to be compressed, thereby achieving a higher pruning rate without sacrificing model performance, ensuring the performance of the compressed model output. The model to be compressed is trained through alternating iterative pruning and fine-tuning. Finally, the remaining weights in the model to be compressed are quantized based on k-means clustering, and pruning and quantization are performed based on single tensor sensitivity analysis. When the weight distribution varies significantly among tensors, it is beneficial for selecting the pruning rate. In this embodiment, the joint model compression method based on sparse regularization, iterative pruning, and cluster quantization compresses the model from a global perspective, achieving maximum model compression without sacrificing accuracy. Parameter clustering using the weight tensors of the model to be compressed effectively clusters and quantizes the parameters in the model to be compressed, further improving the model compression effect.

[0092] Example 2

[0093] Please see Figure 4 This is a flowchart of the model compression method provided in the second embodiment of the present invention. This embodiment is used to further refine step S30 in the first embodiment, including the following steps:

[0094] Step S31: Input the test data into the trained model to be compressed for data processing to obtain output data, and obtain the standard data of the test data.

[0095] The test data includes various types of data, such as image data, speech data, and audio data. The image data, speech data, and audio data are input into the trained model to be compressed for data processing to obtain the output data.

[0096] In this step, the data identifiers of each data in the test data are obtained, and the obtained data identifiers are matched with the pre-stored data lookup table to obtain the standard data. The data lookup table stores the correspondence between different data identifiers and the corresponding standard data. The standard data is used to detect the accuracy of the output data and to evaluate the model performance of the model to be compressed.

[0097] Step S32: Based on the standard data and the output data, perform performance testing on the model to be compressed to obtain subjective evaluation index, audio intelligibility index and speech quality evaluation index.

[0098] Optionally, in this step, the performance testing of the model to be compressed based on the standard data and the output data includes:

[0099] Obtain the output image from the output data, and obtain the standard image from the standard data;

[0100] The subjective evaluation index is obtained by calculating the signal-to-noise ratio based on the standard image and the output image; wherein, the subjective evaluation index used to evaluate the image processing performance of the model to be compressed is obtained by calculating the signal-to-noise ratio of the standard image and the output image.

[0101] Obtain the output audio from the output data, and obtain the standard audio from the standard data;

[0102] The audio intelligibility index is obtained by performing a short-time objective intelligibility calculation on the output audio and the standard audio; wherein, by performing a short-time objective intelligibility calculation on the output audio and the standard audio, an audio intelligibility index is obtained to evaluate the audio processing performance of the model to be compressed.

[0103] Obtain the output speech from the output data, and obtain the standard speech from the standard data;

[0104] The perceptual speech quality evaluation is calculated based on the output speech and the standard speech to obtain the speech quality evaluation index; wherein, by performing perceptual speech quality evaluation calculation on the output speech and the standard speech, a speech quality evaluation index for evaluating the speech processing performance of the model to be compressed is obtained.

[0105] Step S33: If the subjective evaluation index, the audio intelligibility index, and the speech quality evaluation index meet the preset index conditions, then it is determined that the model to be compressed meets the performance degradation condition.

[0106] The preset index conditions can be set according to requirements. For example, in this step, it is detected whether the subjective evaluation index, audio intelligibility index, and speech quality evaluation index are less than the corresponding preset index thresholds. The index thresholds can be set according to requirements. If any evaluation index is less than the corresponding preset threshold, it is determined that the model to be compressed meets the performance degradation condition. That is, when it is detected that the performance of any model of the image processing performance, audio processing performance, or speech processing performance of the model to be compressed has degraded, it is determined that the model to be compressed meets the performance degradation condition.

[0107] In this embodiment, test data is input into the trained model to be compressed for data processing to obtain output data. The output data provides a data foundation for performance testing of the model to be compressed. By obtaining standard data from the test data, the performance of the model to be compressed can be effectively tested. By testing the performance of the model to be compressed, subjective evaluation indicators, audio intelligibility indicators, and speech quality evaluation indicators are obtained. Based on these indicators, the image processing performance, audio processing performance, and speech processing performance of the model to be compressed can be effectively characterized. When a decrease in the performance of any one of the image processing performance, audio processing performance, or speech processing performance of the model to be compressed is detected, the model to be compressed is determined to meet the performance degradation condition.

[0108] Example 3

[0109] Please see Figure 5 This is a schematic diagram of the structure of the model compression system 100 provided in the third embodiment of the present invention, including: a model training module 10, a singular value decomposition module 11, a model output module 12, a parameter clustering module 13, and a parameter setting module 14, wherein:

[0110] The model training module 10 is used to acquire the model to be compressed and to train the model to be compressed.

[0111] The singular value decomposition module 11 is used to add a regularization term to the model to be compressed after model training, and to perform singular value decomposition on the weight tensor of the model to be compressed after adding the regularization term to obtain a singular value matrix. The regularization term is used to sparsify the weight parameters in the model to be compressed.

[0112] The singular value decomposition module 11 is also used to: query the coding layer in the model to be compressed, and add a first regularization term to the normalization layer of each coding layer;

[0113] Query the long short-term memory network layers in the model to be compressed, and add a second regularization term to the layer of each long short-term memory network layer;

[0114] The first regular expression term includes:

[0115]

[0116] The second regular expression term includes:

[0117]

[0118] Where L1 is the first regularization term, L2 is the second regularization term, and μ β δβ and δβ are the mean and variance of the activation values ​​in the model to be compressed, respectively; γ and β are affine transformation parameters; and λ is the penalty coefficient factor.

[0119] Optionally, the formula used for singular value decomposition of the weight tensor of the model to be compressed after adding the regularization term includes:

[0120] A=UΣV T

[0121] Where U is an m×m matrix, Σ is an m×n matrix, Σ has all elements except those on the main diagonal as zero, and each element on the main diagonal of Σ is a singular value, V is an n×n matrix, and both U and V are unitary matrices satisfying UU T =I,VV T =I, where A is the weight tensor.

[0122] The model output module 12 is used to return the steps of training the model to be compressed and subsequent steps based on the singular value matrix until the model to be compressed meets the performance degradation condition, and then output the model to be compressed.

[0123] The model output module 12 is further configured to: input test data into the trained model to be compressed for data processing, obtain output data, and acquire standard data of the test data;

[0124] Based on the standard data and the output data, the performance of the model to be compressed is tested to obtain subjective evaluation index, audio intelligibility index and speech quality evaluation index.

[0125] If the subjective evaluation index, the audio intelligibility index, and the speech quality evaluation index meet the preset index conditions, then the model to be compressed is determined to meet the performance degradation condition.

[0126] Optionally, the model output module 12 is further configured to: acquire the output image in the output data, and acquire the standard image in the standard data;

[0127] The signal-to-noise ratio is calculated based on the standard image and the output image to obtain the subjective evaluation index;

[0128] Obtain the output audio from the output data, and obtain the standard audio from the standard data;

[0129] The audio intelligibility index is obtained by performing a short-time objective intelligibility calculation based on the output audio and the standard audio.

[0130] Obtain the output speech from the output data, and obtain the standard speech from the standard data;

[0131] Based on the output speech and the standard speech, a perceptual speech quality evaluation calculation is performed to obtain the speech quality evaluation index.

[0132] The parameter clustering module 13 is used to perform parameter clustering based on the weight tensor of the output model to be compressed, to obtain a weight parameter matrix, and to perform weight quantization on the weight parameter matrix to obtain a clustering quantization matrix.

[0133] The parameter clustering module 13 is further configured to: match the weight parameters in the weight parameter matrix with a pre-stored clustering lookup table to obtain cluster centroid values, and construct a matrix based on the cluster centroid values ​​to obtain the clustering quantization matrix. The clustering lookup table stores the correspondence between different weight parameters and corresponding cluster centroid values.

[0134] Optionally, the formula used for parameter clustering based on the weight tensor of the output model to be compressed includes:

[0135]

[0136] Among them, w min and w max These represent the minimum and maximum values ​​of the weight tensor, respectively.

[0137] The parameter setting module 14 is used to set parameters for the model to be compressed based on the clustering quantization matrix to obtain the compressed model.

[0138] In this embodiment, by adding a regularization term to the model to be compressed after model training, the sparsity of the weight tensor in the model to be compressed is increased, thereby achieving a higher pruning rate without sacrificing model performance. This ensures the performance of the compressed model output after compression. By returning the singular value matrix to execute the steps of model training and subsequent steps of the model to be compressed, the sparsity of the weight parameters in the model to be compressed can be continuously improved to achieve an iterative pruning effect, thus improving the pruning effect of the model to be compressed. By performing parameter clustering on the weight tensor of the model to be compressed, the parameters in the model to be compressed can be effectively clustered and quantized, further improving the model compression effect of the model to be compressed.

[0139] Example 4

[0140] Figure 6 This is a structural block diagram of a terminal device 2 provided in the fourth embodiment of this application. For example... Figure 6 As shown, the terminal device 2 in this embodiment includes a processor 20, a memory 21, and a computer program 22 stored in the memory 21 and executable on the processor 20, such as a program for a model compression method. When the processor 20 executes the computer program 22, it implements the steps in the various embodiments of the above-described model compression methods.

[0141] For example, the computer program 22 may be divided into one or more modules, which are stored in the memory 21 and executed by the processor 20 to complete this application. The one or more modules may be a series of computer program instruction segments capable of performing specific functions, which describe the execution process of the computer program 22 in the terminal device 2. The terminal device may include, but is not limited to, the processor 20 and the memory 21.

[0142] The processor 20 may be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0143] The memory 21 can be an internal storage unit of the terminal device 2, such as a hard drive or memory of the terminal device 2. The memory 21 can also be an external storage device of the terminal device 2, such as a plug-in hard drive, Smart Media Card (SMC), Secure Digital (SD) card, or Flash Card equipped on the terminal device 2. Furthermore, the memory 21 can include both internal and external storage units of the terminal device 2. The memory 21 is used to store the computer program and other programs and data required by the terminal device. The memory 21 can also be used to temporarily store data that has been output or will be output.

[0144] Furthermore, the functional modules in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0145] If an integrated module is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. This computer-readable storage medium can be non-volatile or volatile. Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable storage medium can include: any entity or device capable of carrying computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the contents of a computer-readable storage medium may be appropriately added to or subtracted from the contents as required by the legislation and patent practice in a jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, a computer-readable storage medium may not include electrical carrier signals and telecommunication signals.

[0146] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A model compression method, characterized in that, The method includes: Obtain the model to be compressed and train the model on the model to be compressed; After model training, a regularization term is added to the model to be compressed, and the weight tensor of the model to be compressed after adding the regularization term is decomposed into singular value matrix. The regularization term is used to sparsify the weight parameters in the model to be compressed. Based on the singular value matrix, return to execute the steps of training the model to be compressed and subsequent steps until the model to be compressed meets the performance degradation condition, and output the model to be compressed. Based on the weight tensor of the output model to be compressed, parameter clustering is performed to obtain a weight parameter matrix, and the weight parameter matrix is ​​then weighted to obtain a clustering quantization matrix. The parameters of the model to be compressed are set according to the clustering quantization matrix to obtain the compressed model. Before the step of performing model training on the model to be compressed and subsequent steps based on the singular value matrix, the method further includes: The test data is input into the trained model to be compressed for data processing to obtain output data, and the standard data of the test data is obtained. Based on the standard data and the output data, the performance of the model to be compressed is tested to obtain subjective evaluation index, audio intelligibility index and speech quality evaluation index. If the subjective evaluation index, the audio intelligibility index, and the speech quality evaluation index meet the preset index conditions, then the model to be compressed is determined to meet the performance degradation condition. Adding a regularization term to the model to be compressed after model training includes: Query the encoding layers in the model to be compressed, and add a first regularization term to the normalization layer of each encoding layer; Query the long short-term memory network layers in the model to be compressed, and add a second regularization term to the layer of each long short-term memory network layer; The first regular expression term includes: The second regular expression term includes: in, For the first regular expression term, The second regular expression term, and These are the mean and variance of the activation values ​​in the model to be compressed, respectively. and These are affine transformation parameters. It is the penalty coefficient factor.

2. The model compression method as described in claim 1, characterized in that, The step of performing performance testing on the model to be compressed based on the standard data and the output data includes: Obtain the output image from the output data, and obtain the standard image from the standard data; The signal-to-noise ratio is calculated based on the standard image and the output image to obtain the subjective evaluation index; Obtain the output audio from the output data, and obtain the standard audio from the standard data; The audio intelligibility index is obtained by performing a short-time objective intelligibility calculation based on the output audio and the standard audio. Obtain the output speech from the output data, and obtain the standard speech from the standard data; Based on the output speech and the standard speech, a perceptual speech quality evaluation calculation is performed to obtain the speech quality evaluation index.

3. The model compression method as described in claim 1, characterized in that, The formula used for singular value decomposition of the weight tensor of the model to be compressed after adding the regularization term includes: in, It is m matrix, It is n matrix, Except for the elements on the main diagonal, which are all zero. Every element on the main diagonal is a singular value. It is n matrix, and They are all unitary matrices, satisfying , A is the weight tensor.

4. The model compression method as described in claim 1, characterized in that, The step of quantizing the weight parameter matrix to obtain the clustering quantization matrix includes: The weight parameters in the weight parameter matrix are matched with the pre-stored clustering reference table to obtain the cluster centroid values. The clustering quantization matrix is ​​then constructed based on the cluster centroid values. The clustering reference table stores the correspondence between different weight parameters and their corresponding cluster centroid values.

5. The model compression method according to any one of claims 1 to 4, characterized in that, The formula used for parameter clustering based on the weight tensor of the output model to be compressed includes: in, and These represent the minimum and maximum values ​​of the weight tensor, respectively.

6. A model compression system, characterized in that, The system includes: The model training module is used to acquire the model to be compressed and to train the model on the model to be compressed. The singular value decomposition module is used to add a regularization term to the model to be compressed after model training, and to perform singular value decomposition on the weight tensor of the model to be compressed after adding the regularization term to obtain a singular value matrix. The regularization term is used to sparsify the weight parameters in the model to be compressed. The model output module is used to return the steps of training the model to be compressed and subsequent steps based on the singular value matrix until the model to be compressed meets the performance degradation condition, and then output the model to be compressed. The parameter clustering module is used to perform parameter clustering based on the weight tensor of the output model to be compressed, to obtain a weight parameter matrix, and to perform weight quantization on the weight parameter matrix to obtain a clustering quantization matrix. The parameter setting module is used to set the parameters of the model to be compressed according to the clustering quantization matrix to obtain the compressed model; The model output module is also used to: input test data into the trained model to be compressed for data processing, obtain output data, and acquire standard data of the test data; Based on the standard data and the output data, the performance of the model to be compressed is tested to obtain subjective evaluation index, audio intelligibility index and speech quality evaluation index. If the subjective evaluation index, the audio intelligibility index, and the speech quality evaluation index meet the preset index conditions, then the model to be compressed is determined to meet the performance degradation condition. The singular value decomposition module is also used to: query the coding layer in the model to be compressed, and add a first regularization term to the normalization layer of each coding layer; Query the long short-term memory network layers in the model to be compressed, and add a second regularization term to the layer of each long short-term memory network layer; The first regular expression term includes: The second regular expression term includes: in, For the first regular expression term, The second regular expression term, and These are the mean and variance of the activation values ​​in the model to be compressed, respectively. and These are affine transformation parameters. It is the penalty coefficient factor.

7. A terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method as described in any one of claims 1 to 5.

8. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method as described in any one of claims 1 to 5.

Images